Implementing wait-time reductions under Ontario government benchmarks (Pay-for-Results): a Cluster Randomized Trial of the Effect of a Physician-Nurse Supplementary Triage Assistance team (MDRNSTAT) on emergency department patient wait times

Background

Internationally, emergency departments are struggling with crowding and its associated morbidity, mortality, and decreased patient and health-care worker satisfaction. The objective was to evaluate the addition of a MDRNSTAT (Physician (MD)-Nurse (RN) Supplementary Team At Triage) on emergency department patient flow and quality of care.

Methods

Pragmatic cluster randomized trial. From 131 weekday shifts (8:00–14:30) during a 26-week period, we randomized 65 days (3173 visits) to the intervention cluster with a MDRNSTAT presence, and 66 days (3163 visits) to the nurse-only triage control cluster. The primary outcome was emergency department length-of-stay (EDLOS) for patients managed and discharged only by the emergency department. Secondary outcomes included EDLOS for patients initially seen by the emergency department, and subsequently consulted and admitted, patients reaching government-mandated thresholds, time to initial physician assessment, left-without being seen rate, time to investigation, and measurement of harm.

Results

The intervention’s median EDLOS for discharged, non-consulted, high acuity patients was 4:05 [95^th% CI: 3:58 to 4:15] versus 4:29 [95^th% CI: 4:19–4:38] during comparator shifts. The intervention’s median EDLOS for discharged, non-consulted, low acuity patients was 1:55 [95^th% CI: 1:48 to 2:05] versus 2:08 [95^th% CI: 2:02–2:14]. The intervention’s median physician initial assessment time was 0:55 [95^th% CI: 0:53 to 0:58] versus 1:21 [95^th% CI: 1:18 to 1:25]. The intervention’s left-without-being-seen rate was 1.5% versus 2.2% for the control (p = 0.06). The MDRNSTAT subgroup analysis resulted in significant decreases in median EDLOS for discharged, non-consulted high (4:01 [95^th% CI: 3:43–4:16]) and low acuity patients (1:10 95^th% CI: 0:58–1:19]), as well as physician initial assessment time (0:25 [95^th% CI: 0:23–0:26]). No patients returned to the emergency department after being discharged by the MDRNSTAT at triage.

Conclusions

The intervention reduced delays and left-without-being-seen rate without increased return visits or jeopardizing urgent care of severely ill patients.

Trial registration number

NCT00991471 ClinicalTrials.gov

Emergency department (ED) crowding occurs when the demand for emergency services exceeds the ability to provide care in a reasonable amount of time [1]. The main cause of crowding is access block [1–3], where no ward bed is available for patients assessed by the ED requiring admission. Consequently, patients requesting ED care accumulate in the waiting room. In Ontario, discharged patients with longer ED length-of-stay (EDLOS) had higher subsequent hospital admission rates and short-term mortality [4]. Crowding increases both mortality and costs [4–9]. This problem is international [2, 5, 10, 11] in scope and impacts [1, 2, 5, 10–13] governments, [10, 14] insurers [7], hospitals, health care workers [11], and patients [4].

In Ontario, there are 163 emergency departments with 5.25 million visits per year [14]. In November 2007, the Ontario Ministry of Health and Long-Term Care declared that reducing EDLOS was a government priority. It established provincial targets, public reporting, and a number of policy initiatives including a Pay-for Results program with financial incentives to reduce EDLOS for targeted hospitals. [15] Our hospital, Sunnybrook Health Sciences Center (Sunnybrook), was one of the targeted hospitals.

The goal of Ontario’s Pay for Results program was for individual hospitals to improve their EDLOS targets by 10% from the previous year (2008) until > =90% of all ED patients reach the set targets [15]. For Sunnybrook, the 2009–10 Pay for Results target was to have > =38% of admitted and > =71% of discharged high acuity patients (Canadian Triage Acuity Scale [CTAS] 1–3) with EDLOS < =8 hrs. For discharged low acuity patients (CTAS 4–5), the Pay for Results target was to have > = 75% of patients with an EDLOS of < =4 hrs. The 2009–10 90^th percentile target for physician initial assessment time was set at 5:24, with the future goal of reaching the ideal target of 3:48 [15].

Despite widespread concerns, there are few rigorous evaluations of interventions to reduce EDLOS. The usual triage process has a nurse taking a brief assessment of non-critical patients arriving to the ED and assigning a triage score representing the urgency for emergency physician assessment. Patients who are critically ill (cardiac arrest) bypass triage, and enter a resuscitation room for immediate management. When ED stretchers are full, the rest of the patients, even those with high acuity scores, remain in the waiting room for lengthy periods of time before seeing an emergency physician.

Triage by a nurse alone is intended to prioritize care for the most severely ill patients by assigning acuity scores (CTAS). It is known that correlation of acuity scoring between a triage nurse and physician is poor [16]; therefore, physicians do not triage. The MDRNSTAT (Physician (MD)-Nurse (RN) Supplementary Team At Triage) does not triage. Its role is to interact and start interventions on patients without an ED care space after interacting with the triage nurse. At least 10 studies testing the impact of adding a physician to the existing nurse at triage have shown some benefit [17–26] but only one of these was a randomized trial [20]. We modified an Australian model of physician-triage and report the evaluation of a MDRNSTAT.

The primary objective of this study is to examine the impact and limitations of adding 6.5 hours of MDRNSTAT on EDLOS among non-consulted, discharged patients seen by the emergency physician.

Setting

Sunnybrook is a 1200-bed academic tertiary level hospital in central Toronto. It is a trauma, regional stroke, interventional cardiology, neurosurgical and oncology center. During the study period, Sunnybrook Emergency Department received 45000-patient visits per year with an admission rate of 22%.

Design

We conducted a cluster, randomized-controlled trial of the impact of a Physician-Nurse Supplementary Triage Assessment Team (MDRNSTAT) on EDLOS at Sunnybrook Emergency Department over a 26-week period (October 1, 2009 - April 1, 2010). Clusters were ED patients arriving from 8:00–14:30. During control clusters, ED patients were triaged by a standard nurse for acuity scoring, registered, and assigned an ED stretcher (Figure 1 light arrows). If no stretcher was available, patients stayed in the waiting room until one became available. Afterwards, an emergency physician would assess, order investigations, manage and determine disposition. During intervention clusters, we added a MDRNSTAT without changing time to triage (Figure 1 - bold arrows). Both ambulatory and ambulance patients could be seen by the MDRNSTAT. After being assigned an acuity score by the standard triage nurse, the patient was assessed by the MDRNSTAT physician in a room behind the triage bay where orders were administered instead of waiting for a stretcher in the ED. The MDRNSTAT nurse completed lab and medication orders; whereas, the radiology service completed diagnostic imaging orders. The MDRNSTAT nurse could assist triage or emergency if required. The MDRNSTAT physician could also request consults and, possibly, discharge the patient. If not discharged, the patient would return to the waiting room to wait for a stretcher where the usual ED physician would take over and follow up on the results or consultations. If an ED stretcher was immediately available, patients bypassed the MDRNSTAT and were seen by the usual ED physician.

Pictorial representation of patient management from arrival to discharge in the emergency department. The control group is outlined with light arrows. The MDRNSTAT intervention is outlined with bold arrows.

Full size image

Inclusion and exclusion criteria

We included weekday ED patients arriving from 8:00–14:30. We excluded critically ill (CTAS 1) patients requiring immediate resuscitation (i.e. cardiac arrest) since any delays for MDRNSTAT assessment at triage would be unethical and harmful. Patients directed (“directs”) to the ED for an admitting service (eg. oncology) assessment and not an emergency physician, were also excluded from the study.

Randomization and allocation

Randomization was conducted centrally, before study initiation, using a computer-generated algorithm by the study statistician. The unit of allocation was the ED shift. The randomization schedule ensured 2 control days between interventions to minimize any carryover MDRNSTAT effect. To anticipate predictable surges in the number of ED patients arriving at 11:00, the MDRNSTAT was assigned on weekdays from 8:00–14:30. Our rationale for this timing was to prevent queuing rather than reacting to increased waiting times.

In the daytime weekday study period, there were 65 MDRNSTAT shifts divided amongst 14 physicians and 14 nurses who were currently on staff. There were 66 usual nurse-triage alone (control) shifts. Physicians and nurses voluntarily signed up for paid MDRNSTAT shifts in addition to their regularly scheduled ED shifts. The MDRNSTAT added one physician and one nurse to the 3 physicians and 15–19 nurses typically working each dayshift.

The usual emergency physicians were instructed to not work at triage. The MDRNSTAT could not be blinded to staff; however, patients were unaware of the MDRNSTAT because its schedule was not publicized. Time recordings of primary outcomes were uninfluenced since they were automatically collected by a computerized patient-tracking emergency department information system.

Data collection procedure

Time data, CTAS levels, and patient disposition were retrieved from two different computerized information systems: Electronic Patient Records and Emergency Department Information Systems. Electronic Patient Records collected diagnostic imaging and laboratory order times. The Emergency Department Information System collected the rest of the data. Ethics approval was obtained from the Sunnybrook Hospital Research Ethics Board (August 2009). The requirement for formal informed consent from patients was waived.

Outcome measures

The primary cluster outcome measure was median EDLOS for discharged (non-admitted) patients directly seen by emergency physicians (MDRNSTAT or usual ED physician) with no consultation (“non-consulted”) because emergency department interventions could not influence consultation duration or hospital bed availability for admissions. According to the provincial thresholds, EDLOS was the interval between triage time to ED discharge. We analyzed EDLOS according to patient acuity, disposition, and consultation.

Secondary outcome measures were: triage time to start of emergency physician or assessment, EDLOS among patients seen by emergency physician and referred for consultation/admission or left-without-being seen, percentage of CTAS 2–5 patients who were seen within Ontario’s Pay for Results wait-time target thresholds, and left-without-being-seen rate, a widely used measure of safety and satisfaction. [2, 5, 27, 28] Other secondary outcome measures were triage time to: laboratory, diagnostic imaging, consultation, and bed request order time.

To measure unintended harm, we searched our database for patients who returned to our hospital within 48 hours after being discharged from triage by MDRNSTAT. These patients’ charts were reviewed (IC) for a change in management, admission or death. Management change occurred if diagnosis or treatment was different between the first and second visit. No time parameters or external hospital data could be collected for patients who left-without-being-seen. All outcome measures were assigned a priori.

Sample size and statistical analysis

We estimated that a sample size of 50 clusters (shifts) per group, with 32 individuals per cluster, would have greater than 90% power to detect a difference of 30 minutes between the group medians. We used 30 minutes as a clinically significant reduction, because previous trials determined this difference was associated with decreased 7-day mortality, re-admission of discharged patients [4] and left-without-being seen rate [27]. The Ontario Ministry of Health’s target was a 10% reduction [14] of Sunnybrook’s baseline physician initial assessment time of 324 minutes, which also equals 30 minutes. We used a standard deviation of 70 minutes, intra-cluster correlation of 0.1 [29] using a two-sided t-test, and a significance level of 0.05.

The analyses were carried out comparing patients in groups, control versus treatment, adjusting for the clusters (shifts) in which the patients appeared. This resulted in 66 control clusters (for a total of 3163 visits) compared to 65 intervention clusters (for a total of 3137 visits) and therefore the visits were not treated as independent observations for analysis purposes. The number of clusters was increased to 65 in order to compensate for the variable number of individuals per cluster.

Descriptive statistics were calculated for variables of interest with categorical measures summarized using counts and percentages. The primary outcome, EDLOS, as well as the other secondary time variables were summarized using medians. For analysis purposes, the outcomes were log transformed and the mean group differences of EP’s, MDRNSTAT, the combination of the two and the control group were compared using linear models adjusting for the correlation among observation taken from the same cluster (shift). Confidence intervals for medians were determined with 1000 bootstrap simulations. The 2.5^th and 97.5^th percentile respectively were reported as the 95% confidence interval. For variables with no events, such as rate of harm we calculated one-sided 95% confidence intervals using the Hanley estimate of 3/n where n is the total sample size of interest [30]. All analyses were carried out using SAS Version 9.1 (SAS Institute, Cary, NC, USA).

Participants

There were 17,034 weekday emergency department visits (8531 randomized to intervention and 8503 to control) during the study. After excluding patients arriving outside 8:00–14:30, critically ill, or “directs”, there were 66 control clusters (3163 visits) group and 65 intervention clusters (3137 visits). Of the 3137 intervention cluster patients, 750 (24%) patients waiting for an ED stretcher were initially seen by the MDRNSTAT. The regularly scheduled emergency physician (EP) solely managed the remaining 2387 patients (Figure 2) only after the patient obtained a stretcher. The two study groups were similar with respect to baseline characteristics of sex, age, and CTAS (Table 1). The percentages of discharged patients were the same in each group (79.1%).

Randomization and encounter allocation with associated inclusion and exclusion criteria.

Full size image

The combined (MDRNSTAT + EP) and MDRNSTAT group had improved primary outcomes. For the combined group, EDLOS was significantly reduced by 24’ for non-consulted higher acuity patients (CTAS 2–3) (p = .005) but was not significant for discharged, non-consulted lower acuity patients (CTAS 4–5). The MDRNSTAT significantly reduced the discharged, non-consulted lower acuity EDLOS by 56’ (p < .0001) (Table 2).

For the secondary outcomes among non-consulted CTAS 2–3 patients, the combined group met its 10% improvement target, and came close to the ideal threshold of 90%, with 89.1% of combined patients reaching the Pay for Results EDLOS target of < =8 h. For non-consulted CTAS 4–5 patients, the combined group reached the 10% improvement threshold, but did not exceed the ideal target (83.3%); however, the MDRNSTAT did (92.1%).

For patients who required admission or consultation, the intervention made little difference.

The combined group decreased physician initial assessment time by 26-minutes with a 90^th percentile of 3:31. The MDRNSTAT decreased physician initial assessment time by 56-minutes with a 90^th percentile of 1:08. The ideal provincial 90^th percentile physician initial assessment time threshold is 3:48. In 2007–8, Sunnybrook hospital’s 90^th percentile physician initial assessment time was 5:42.

The combined group and MDRNSTAT reduced the left-without-being-seen rate (Table 2).

Some of the secondary outcomes including laboratory, diagnostic imaging, and consultation times, for the control and trial groups improved (Table 3). For discharges and admissions, the combined group had shorter ordering times for all the three modalities than control. For admissions, MDRNSTAT hospital bed request time was faster than control. For discharges, the combined group ordered less bloodwork and imaging than control.

The MDRNSTAT discharged 26.1% (196/750) of patients from triage. Only 3 of these patients returned to the ED within 48 hours. None of these patients were admitted or died. Their charts were reviewed and were found to have the same discharge diagnoses between the first and second visit: staple reassessment, urinary tract infection and social concerns. All 3 patients were discharged on the second visit without change in management or treatment. None met our definition of harm. The Hanley 95% CI estimate [30] for harmful events was 0–1.53%.

This was a pragmatic study. Because the only changes were the added MDRNSTAT resource, its benefits could be applied to other hospitals similar to Sunnybrook.

This study had a number of limitations. It is a single-center study, and one of two randomized control trials [20]. To increase its generalizability from academic tertiary level centers, a multi-center trial is recommended. Coverage on weekdays from 8:00–14:30 meant that patients arriving on weekends and evenings could not benefit. Focusing MDRNSTAT shifts during times of crowding or adaptively engaging MDRNSTAT when bed blockage occurs may benefit from future study.

Because physicians and nurses entered time data, measurement bias was possible. This study was not blinded, but all health care workers were under the same Pay for Performance program. Additionally, the health care workers who volunteered to be MDRNSTAT also worked in the department as control or part of the intervention on different days. Consequently, all physicians would have the same pressure to discharge patients within targets. However, wait times could have been shortened because the MDRNSTAT wanted to discharge patients quicker, or faster physicians/nurses volunteered to be the MDRNSTAT. The differences between the MDRNSTAT and the control could be attributed to patient selection; however, time gains were observed with the combined group as well.

It could be argued that the benefits of this trial arose from the extra staff, and would have occurred had they been deployed in the ED rather than at triage. However, Ontario hospitals run lean with fewer ward beds per capita than many US settings [12, 31] creating access block. Consequently, ED admissions can occupy ED stretchers for a median 10 hours creating a bottleneck of physical space for assessment. Under these conditions, added staff capacity would be optimized at triage since patients do not have to wait for an available ED stretcher in order to see a physician. To further explore this issue, future studies could compare a control physician-nurse team in the department to the MDRNSTAT.

We only reviewed Sunnybrook hospital’s database for harm. Patients returning to a different hospital, or dying outside of the hospital would have been missed. However, distribution should have been similar in both arms given randomization. This suggests no detectable harm, but it is important to bear in mind that this study was not powered to detect mortality differences.

For patients managed solely by the ED, MDRNSTAT significantly reduced EDLOS and physician initial assessment time. The MDRNSTAT initiated actions (laboratory, diagnostic imaging, and consultation request times) that shortened emergency physician decisions and patient delays. Because waiting times to disposition were reduced, left-without-being-seen rates declined, resulting in greater patient throughput.

For patients whose EDLOS was dependent upon factors external to the ED (i.e. consultation), the gains with shortened physician initial assessment and request times were diluted by the time required to complete these external requests by non-ED hospital staff.

Serious illnesses seen in ED’s, such as ischemia, have time-dependant treatments [32]. Skilled, timely care [10, 33] is crucial; hence the focus of performance indicators on waiting times [34]. However, rules focusing solely on wait-times, without thoughtful improvement in care processes may compromise quality of care [35] and increase costs [36]. The 4-hrs to disposition rule in the UK was estimated to cost an extra £820 million without consistent health improvement [36]. None of 750 patients assessed in this prospective, randomized-controlled trial experienced harm according to our measures; suggesting that this method to speed critical decisions does not compromise quality of care.

Emergency departments have successfully implemented fast-track areas. [37, 38]. However, the MDRNSTAT can see mixed acuity patients, whereas the fast-track population will be mainly ambulatory, lower acuity patients. While fast-track might be useful in community hospitals, Sunnybrook has few (27%) CTAS 4–5 patients compared to the provincial average (48.2%) [39].

Some have argued that it is inappropriate to have the “the ED compensate(s) for deficiencies in the larger organization by subordinating its own needs and priorities” [40]. The ED is squeezed between rising demand for emergency access to beds and increasing block as hospitals beds are closed or occupied by more lucrative, elective patients. MDRNSTAT cannot address systemic emergency department [41] or health economic problems [42]. Thus, we support the conclusion of multiple studies that emphasize the need for “throughput-output” solutions, such as hospital bed availability [11, 43, 44], long-term care, diagnostic imaging, and associated funding. However, the MDRNSTAT may provide an additional solution to those requiring government co-operation or hospital resources [8, 45–47].

The MDRNSTAT was effective in decreasing EDLOS, physician initial assessment time and left-without-being-seen rates of discharged patients requiring ED services in an urban, academic tertiary-level hospital, without compromising the quality of patient care. Given that the risk of admission and death for ED patients is associated with increasing EDLOS [4], the time gains associated with MDRNSTAT could improve patient outcomes. We are currently analyzing data on the economic value of the MDRNSTAT program.

Team: Mr. Darren Hefferon: acquisition of data, technical support. Mr. Ben DeMendonca: acquisition of data. Mr. Marko Katic: statistical analysis. MDRNSTAT: Drs. Badali, Peerbaye, Cleghorn, Freedman, Robles, Tyberg, Shelton, Brzozowski, Cheng, Fulton, McDonnell, Atzema, Thurgur, Chenkin and Nurses Woodard, Spina, Michaelson, Nguyen, Giordano, Thorne, Desi, Cosby, Hincapie, Constant and Stevens, Marsh, Adam and Caruana.

Financial Support: AFP (Alternative Funding Plan) Innovation Fund. This fund had no role in the design, conduct of this study; collection, management, analysis, or interpretation of the data; preparation, review or approval of manuscript.

Data Sharing: Data sharing: technical appendix, statistical code, and dataset available from the corresponding author at ivy.cheng@sunnybrook.ca.

Affiliations

1. Emergency Services, Sunnybrook Health Sciences Center, Toronto, Canada

   • Ivy Cheng
   •  & Jeffrey Tyberg

2. Karolinska Institutet, Stockholm, Sweden

   • Ivy Cheng

3. Clinical Epidemiology Unit, Sunnybrook Health Sciences Center, Toronto, Canada

   • Jacques Lee

4. HOPE Research Centre, Sunnybrook Health Sciences Centre, Toronto, Canada

   • Nicole Mittmann

5. Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada

   • Sharon Ramagnano

6. Institute of Clinical Evaluative Sciences, Sunnybrook Health Sciences Centre, Toronto, Canada

   • Alex Kiss
   •  & Michael Schull

7. Department of Emergency Medicine, Austin Health, Heidelberg, Australia

   • Fergus Kerr

8. Department of Family Medicine - Schulich School of Medicine and Dentistry, Western University, London, Canada

   • Merrick Zwarenstein

Corresponding author

Correspondence to Ivy Cheng.

Competing interests

The authors declare that there have no competing interests.

Authors’ contributions

IC: public responsibility for the whole content, conception and design, acquisition of data, analysis and interpretation of data, drafting of the manuscript, critical revision of the manuscript, statistical analysis, obtaining funding, administrative, technical, material support. IC had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. MZ: public responsibility for part of the content, conception and design, interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual contact, statistical analysis supervision. JT: public responsibility for part of the content, conception, critical revision of the manuscript for important intellectual content, obtaining funding, administrative support. JL: public responsibility for part of the content, conception and design, interpretation of data, critical revision of the manuscript, supervision. NM: public responsibility for part of the content, design, critical revision of the manuscript, supervision. SR: public responsibility for part of the content, acquisition of data, critical revision of the manuscript, technical/material support. AK: public responsibility for part of the content, design, analysis of data, statistical analysis, critical revision of the manuscript. MS: public responsibility for part of the content, design, critical revision of the manuscript, supervision. FK: public responsibility for part of the content, conception and design, critical revision of the manuscript, technical support. All authors read and approved the final manuscript.

Below are the links to the authors’ original submitted files for images.

Reprints and Permissions