Multi-centre validation and clinical experience of EzaPredictive for the Emergency Room and clinical departments

Patient flow management has become increasingly important for hospital systems.
Novel machine learning (ML) methods may aid by giving information that supports
health care professionals in their patient flow management at the emergency
room (ER) and clinical departments. However, few of these prediction models
have been implemented in clinical practice and we know little about the added
value of these tools for supporting health care professionals in their patient
flow management. Moreover, few studies have investigated the clinical
experience and acceptance of these ML support tools.
We therefore propose to study these aspects for the EzaPredictive 1.0 tool.
EzaPredictive 1.0 provides explainable ML predictions for hospital admission,
ER length of stay (LOS), and hospital LOS.

Objective 1: Compare the predictions of the EzaPredictive 1.0 ML models for
hospital admission, ER LOS, and hospital LOS with healthcare professional
predictions.
Objective 2: Assess healthcare professional experience and acceptance of
EzaPredictive 1.0 for supporting their patient flow management.

For objective 1, we plan to perform a multi-centre observational study in at
least two Dutch general hospitals that are different from the hospitals where
the EzaPredictive 1.0 software was developed and two Dutch general hospitals
that participated in the development of EzaPredictive 1.0. In this study, we
will compare the predictive performance and inter-rater reliability of the
(individual) healthcare professionals, and the ML algorithms contained in the
EzaPredictive 1.0 software. Predictions will be collected across two phases.
During the first phase, healthcare professionals are blinded to the ML
outcomes. During the second phase, ML outcomes for the patient are shown to
healthcare professionals after they*ve reported their predictions for that
patient.
For objective 2, we give the healthcare professionals the full experience of
EzaPredictive 1.0 in a third phase of the study. In this phase the overview
page with ML predictions of all patients of a department are shown to
healthcare professionals, in addition to the ML predictions per patient. A the
end of this third phase, we distribute questionnaires to gather healthcare
professional feedback on the acceptability and usability of EzaPredictive 1.0
in supporting them in their patient flow management.

Risks: The current investigation will not directly affect patient care. During
the second phase of the observational study, real-time ML predictions of a
patient will be shown after healthcare professionals have provided their own
estimates for that patient to encourage interaction with and learning from the
tool*s insights. During the third phase of the study, real-time ML predictions
of a patient as well as the overview of real-time ML predictions for all
patients of a department are shown to assess healthcare professional experience
of the EzaPredictive 1.0 software in a qualitative questionnaire study.
The intended use of the EzaPredictive 1.0 software is limited to support on
patient flow management, explicitly excluding diagnostic or therapeutic use
(MDR class I). Healthcare professionals will be clearly instructed on the
intended and correct use of the EzaPredictive 1.0 software. Any effect during
this study will therefore be logistic in nature and at most affect the timing
of certain decisions if permissible (e.g., arranging a hospital admission). The
risks for patients associated with this study are therefore negligible. The
risks for healthcare professionals are minimal with proper training.