This article has Open Peer Review reports available.
A generic method for evaluating crowding in the emergency department
© The Author(s). 2016
Received: 14 July 2015
Accepted: 22 May 2016
Published: 14 June 2016
Crowding in the emergency department (ED) has been studied intensively using complicated non-generic methods that may prove difficult to implement in a clinical setting. This study sought to develop a generic method to describe and analyse crowding from measurements readily available in the ED and to test the developed method empirically in a clinical setting.
We conceptualised a model with ED patient flow divided into separate queues identified by timestamps for predetermined events. With temporal resolution of 30 min, queue lengths were computed as Q(t + 1) = Q(t) + A(t) – D(t), with A(t) = number of arrivals, D(t) = number of departures and t = time interval. Maximum queue lengths for each shift of each day were found and risks of crowding computed. All tests were performed using non-parametric methods. The method was applied in the ED of Aarhus University Hospital, Denmark utilising an open cohort design with prospectively collected data from a one-year observation period.
By employing the timestamps already assigned to the patients while in the ED, a generic queuing model can be computed from which crowding can be described and analysed in detail. Depending on availability of data, the model can be extended to include several queues increasing the level of information. When applying the method empirically, 41,693 patients were included. The studied ED had a high risk of bed occupancy rising above 100 % during day and evening shift, especially on weekdays. Further, a ‘carry over’ effect was shown between shifts and days.
The presented method offers an easy and generic way to get detailed insight into the dynamics of crowding in an ED.
Crowding is a major health concern and an increasing problem for Emergency Departments (EDs) internationally [1–3]. Crowding is defined as a situation where treatment demands exceed available resources and is associated with increased mortality, increased treatment costs and over-all reduced quality of care [4–8].
Crowding can be conceptualised as a model of patient flow, which incorporates factors that influence on or are affected by crowding . Any such model can be classified on a continuum from high generalisability with abstract information outputs (i.e. “level 1”) to highly specific with low generalisability (i.e. “level 4”) . When developing a new model, researchers should acknowledge at which level they want their model to operate; balancing the need for application in various settings (level 1) against the need for a model drawing very specific conclusions (level 4). A simple generic method for evaluation of crowding that allows easy implementation into various EDs is in high demand [11–13].
The aim of this study was to develop a method that meets the criteria for an ideal universal measure for crowding being generalisable in measurement, definition and validity as described by Dr. Pines . The proposed method was tested empirically in a clinical setting.
Study design and population
An emergency department (ED) has been defined as a hospital department accessible to patients with medical emergencies staffed by multidisciplinary clinical personnel ready to provide immediate stabilisation and care . The processes that a patient must undergo from the time of arrival to departure characterise patient flow in the ED . We defined a patient as being present in the ED from the time of electronic registration at the front desk (arrival) to the time of electronic registration of departure from the ED unit.
Variables obtained for each patient in each 30-min interval
Aggregated data set, 30-min intervals
Mean age of patients present in the ED
Time of arrival to ED
Number of patients waiting to be examined (Q1)
Time of triage
Number of patients waiting to be evaluated by a doctor (Q2)
Time of first TOKS
Number of patients waiting to have completed examination/treatment (Q3)
Time of last TOKS
Number of patients waiting to leave the ED (Q4)
Time of departure from ED
Number of patients in the ED triaged red
First TOKS score
Last TOKS score
Number of trauma patients in the ED
For every shift (day, evening and night) of each day in the study period the maximum queue (max queue) length was found. Days were defined as beginning by the day shift at 7 a.m. with every shift lasting eight hours, thus reflecting the clinical setting. To allow for steady state in the queuing system, all analysis of queues were made discarding the first 24-h, i.e. the first three shifts, of the study period.
To explore possible extensions of the model, thus increasing the level of information on throughput processes, we conceptualised an idealised patient flow by dividing the stay in the ED into four queues based on clinical relevance and structural challenges: A queue between arrival and start of initial assessment by an ED-nurse (Q1), a queue before evaluation by a doctor (Q2), a queue before completion of examination and treatment in the ED (Q3) and finally a queue arising for patients who are waiting to leave the ED (Q4). With few exceptions, e.g. trauma call patients, every patient presenting in the ED would go through these steps in that exact order. This is in line with the conceptual model proposed by Asplin and colleagues for ED throughput . Since it was not possible to get verbatim timestamps for some of these factors (i.e. start of initial assessment, first evaluation by a doctor and completion of treatment), a number of surrogate-markers were decided upon: time of first triage, first re-assessment of vital signs “TOKS” value (a Danish early warning system based on systematic assessment of vital signs ) and last TOKS value. Figure 1b offers a graphical representation of this model.
In applying the method we used an open cohort design with prospectively collected data on every patient presenting to the ED from January 1st to December 31st 2013. The number of arrivals during the study period determined the sample size and patients were followed from ED admission to ED discharge as registered in the Electronic Health Records (EHR). EHR contains patient administrative data (e.g. age, gender, time of admission and discharge) and clinical patient data (e.g. triage score and diagnosis code). Utilising the “black-box” model for patient flow we computed the aggregate data set (N = 17,520) and max queue length for each shift (3) of each day (365, i.e. 3 * 365 values of max queues). Likewise, queue lengths for each of the four queues in the extended model of idealised patient flow in the ED were calculated for each day and shift (i.e. 4 * 3 * 365 values of max queues).
We applied the method to the ED at Aarhus University Hospital, Denmark. The hospital had an uptake area of about 300,000 people and a census of approximately 70,000 annually divided on several units. The ED unit under study received about 40,000 patients in total divided on orthopaedic injuries, trauma patients and unstable medical patients. Other surgical patients and patients with medical conditions were also received although not exclusively in this unit. Some patient categories (including medical paediatric patients, patients with psychiatric emergencies and patients with heart related events) were not attended to in the studied ED but were transferred to specialised departments. Patients were either treated and discharged or admitted to the hospital. Physicians working in the ED had the right to admit patients to an appropriate inpatient bed, but in case of bed shortage on the wards, patients could be found boarding in the ED. Boarding time could not be defined since decision to admit were not registered. Unique department and location codes were used to record time of arrival and discharge from the studied ED unit. Hereinafter, the studied ED unit will be referred to simply as “the ED”.
Characteristics of the emergency department and the patients
Arrivals to the ED, 2013
Age < 18 years old
Age > 65 years old
Length of stay ≤ 30 min
Length of stay ≥ 5 h
Number of beds a
7–11.59 am: 7
7–10.59 am: 7
12–07.59 pm: 8
11 am–07.59 pm: 8
8–10.59 pm: 7
08–10.59 pm: 7
11 pm–6.59 am: 4
11 pm–6.59 am: 5
Due to highly skewed distributions, we used non-parametric statistical methods to evaluate queue lengths: Kruskal-Wallis rank sum test to compare the distributions of the maximum daily queue length in each of the three shifts and between weekday and weekends and Spearman’s rank sum correlation test (Spearman’s rho) to evaluate correlations between maximum queue lengths in successive shifts. We applied Fisher’s index of dispersion to evaluate Poisson distribution of the data. Data management and statistics were done using R, version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).
From the timestamps of arrivals and departures a simple method could be set up and risk of crowding and its predictors computed. It was possible to extend the model by separating the patient flow in the ED into clinically meaningful queues allowing for increased level of information. All codes necessary for the data management and analyses presented including an example data set are freely available (see section Availability of data and materials).
Characteristics of maximum queue lengths per day in 2013
% of time with ≥19 patients
Median patient/nurse ratio
Day shift (7 am–02.59 pm)
2.9 (2.5 when 8 nurses)
2.1 (1.9 when 8 nurses)
Evening shift (3 pm–10.59 pm)
2.5 (2.9 when 7 nurses)
2.1 (2.4 when 7 nurses)
Night shift (11 pm–6.59 am)
The main purpose of our study was to develop a generic method to describe and estimate crowding in an ED. By utilising only time of arrival and time of departure it was possible to get detailed insight into the dynamics of the queue and the resulting crowding in the ED. Since the method only relies on timestamps and patient census, it is the question at hand and data availability that sets the limit to the practical implementation of the method: A finer temporal resolution can be chosen, other time stamps can be used to define the queue of interest and several queues be combined to elaborate on the queues during patient flow (e.g. Fig. 1b). Likewise, the max queue can be related to one or more groups of staff in the ED and/or to patients of interest. We encourage fellow researchers to build on and modify the programming code for the method to suit the research question at hand.
The max queues represent the 30-min time period of each shift on each day with most patients in the ED. The interpretation is exemplified by the empirical cumulative distribution function (Fig. 4): It expresses the risk of at least one interval with a given queue length. Similarly, the positive correlation of max queues between shifts following each other means that, if the maximum queue during a shift was long, it was likely that a long queue would arise at least once during the following shift. When analysing crowding we are interested in what happens at times with prolonged queues. The observations with minor queues can be thought of as noise to the signal of interest. On the other hand if only observations with queue lengths above a certain threshold were chosen, variation would be lost and with it our ability to make inferences. To the knowledge of the authors, no other research has been done on maximum queues in the ED.
Crowding has been measured in a number of ways ranging from subjective assessment by ED staff  to objective measurements [19–24] and combinations of the two [12, 15]. To add to this, the threshold defining when crowding occurs is absolute in some studies [15, 19–21] using objective measurements for crowding and relative in others . When crowding is defined subjectively the external generalisability should be questioned: What is perceived as crowding in one ED might be considered otherwise in another [25–27]. Objective definitions however, can be criticised for oversimplifying a complex interplay between numerous factors . When the threshold for crowding is relative it is inevitable that crowding will occur a set percentage of the time, disregarding that the ED might never reach full bed capacity, experience the adverse effects of crowding etc. This limits possible research questions to concern effects of “above-normal” number of patients in the ED. Finally, dynamic measurement of crowding (i.e. several measurements during a patient’s stay in the ED) is preferable to a static measurement (i.e. a single measurement e.g. when arriving at the ED) . Methods requiring highly detailed administrative data and/or specialised insight into statistical programming are seldom implemented in a clinical setting, are found to have low external validity and are not found to be superior to ED occupancy level in describing crowding [11, 13, 21, 28–30]. Likewise, if a method relies on specific assumptions about data, researchers must ensure that such assumptions are met before implementing the method. Failing to do so could result in deceptive conclusions. As an example, we found that arrivals did not follow a Poisson distribution, but rather a negative binominal distribution. This is in opposition to what is assumed in many studies on ED crowding [23, 24, 31] and could render such methods error-prone in our setting. Therefore, we sought to develop a novel method for quantifying crowding in an ED from simple and readily available parameters with occupancy level as a central measurement of crowding and a temporal resolution of 30-min.
One example of direct actions based on our proposed method is the ability to anticipate crowding: If the max queue in a day shift is long the ‘carry over’ effect mean that a long queue are likely to arise during the following evening shift. This will allow interventions to counter crowding before it arises. Such prediction tool could be implemented and evaluated using the Plan-Do-Study-Act tool .
Some limitations must be addressed. Arrival and departure time were recorded for every patient in the study and are generally considered very accurate, and possible errors in these timestamps were most likely random. If they were systematic, e.g. if at busy times the arrivals were systematically collected and registered with delay all at the same time, it could seriously bias the findings towards longer queues. Other indicators of flow were considered, amongst others time of diagnosis, time of first medicine prescription, time of ordering of x-ray and time of ordering blood tests. All of these were found to be particularly liable to “measurement” bias in that they would very often, and depending on the individual doctor, be registered later than actually effectuated, and were thus rejected. As with any standardised analysis model, this is an approximation of real life allowing a detailed overview of the system. Although very few assumptions are necessary with this method, complex organisational and psychological interactions between the factors are likely to play important roles in an ED; not least in times with crowding . Such interactions have not been taken into account.
For the results of applying the method, it must be noted that EHR is a secondary data source primarily for clinical use. Since the data collection is not under the control of the researchers the quality of EHR data for research can be questioned . On the other hand, the fact that data were collected as part of the clinical work without this research project in mind could be argued to lower the risk of introducing selection and/or information bias and thus strengthen the validity of the results. Further, the number of nurses on duty was derived from the duty roster and not the actual number of nurses meeting in each shift of each day. These numbers could differ e.g. due to illness. With a study period of one year we found it not to be feasible to allow for such variations but if investigating shorter periods of time this could be an option to strengthen the internal validity of the results. Finally, only the nurse capacity was included in the presented results leaving other valuable resources in the ED - such as doctors - unaccounted for. We encourage future research to focus on the possibility to extend the method to include several sub-queues (e.g. as proposed in Fig. 1b) and to explore the method in a multicentre trial.
In summary, building on the recommendations for dynamic measurement of crowding put forth by McCarthy et al.  we present a highly generic method relating ED capacity (beds and nurses) to dynamically measured ED census, setting no threshold for crowding. Our model is highly generalisable and easy to implement in diverse settings.
We put forward a generic method for evaluating crowding in an ED from readily available data. This allows for detailed analysis of crowding: its impact and associations in any specified ED. It offers insight on the dynamics of crowding and allows for further investigation of predictors of crowding.
ED, emergency department; EHR, Electronic Health Records; IQR, interquartile range
The authors thank Jonas Rosendal Bager-Elsborg for his help in retrieving the data, and Malene Østerby Skov for her assistance in applying the method to the ED.
Aarhus Universitets Forskningsfond (DK); Fonden af 1870; Familien Hede Nielsens Fond. The funding bodies played no role in the design of the study, the collection, analysis, interpretation of data, or in writing the manuscript.
Availability of data and materials
Under Danish law raw data obtained from EHR cannot be published. The R-code for the data management as well as the analysis is available together with a constructed example of the dataset in the GitHub repository, https://github.com/eiset/Crowding_code.git
All authors participated in conceiving and design of the study and helped to draft the manuscript. AHE coordinated the project, performed all data management and analysis and drafted the manuscript. ME set up the analysis plan. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
In accordance with Danish law permission to conduct the study has been granted from the Danish Data Protection Agency (2007-58-0010) and the Danish Health and Medical Authorities (3-3013-552/1/). Register based studies requires no further approval in Denmark. We report in adherence with the Strengthening the Reporting of Observational Studies in Epidemiology statement .
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Derlet RW. Overcrowding in U.S. EDS: a critical condition. Cal J Emerg Med. 2002;3(2):24–6.PubMedPubMed CentralGoogle Scholar
- Pines JM, Pilgrim RL, Schneider SM, Siegel B, Viccellio P. Practical implications of implementing emergency department crowding interventions: Summary of a moderated panel. Acad Emerg Med. 2011;18(12):1278–82. doi:10.1111/j.1553-2712.2011.01227.x.View ArticlePubMedGoogle Scholar
- Schiff GD. System dynamics and dysfunctionalities: levers for overcoming emergency department overcrowding. Acad Emerg Med. 2011;18(12):1255–61. doi:10.1111/j.1553-2712.2011.01225.x.View ArticlePubMedGoogle Scholar
- Fee C, Hall K, Morrison JB, Stephens R, Cosby K, Fairbanks RT, et al. Consensus-based recommendations for research priorities related to interventions to safeguard patient safety in the crowded emergency department. Acad Emerg Med. 2011;18(12):1283–8. doi:10.1111/j.1553-2712.2011.01234.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Schuur JD, Hsia RY, Burstin H, Schull MJ, Pines JM. Quality measurement in the emergency department: past and future. Health Aff (Millwood). 2013;32(12):2129–38. doi:10.1377/hlthaff.2013.0730.View ArticleGoogle Scholar
- Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10. doi:10.1111/j.1553-2712.2008.00295.x.View ArticlePubMedGoogle Scholar
- Carter EJ, Pouch SM, Larson EL. The relationship between emergency department crowding and patient outcomes: a systematic review. J Nurs Scholarsh. 2014;46(2):106–15. doi:10.1111/jnu.12055.View ArticlePubMedGoogle Scholar
- Moskop JC, Sklar DP, Geiderman JM, Schears RM, Bookman KJ. Emergency department crowding, part 1--concept, causes, and moral consequences. Ann Emerg Med. 2009;53(5):605–11. doi:10.1016/j.annemergmed.2008.09.019.View ArticlePubMedGoogle Scholar
- Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo Jr CA. A conceptual model of emergency department crowding. Ann Emerg Med. 2003;42(2):173–80. doi:10.1067/mem.2003.302.View ArticlePubMedGoogle Scholar
- Fletcher A, Worthington D. What is a ‘generic’ hospital model?--a comparison of ‘generic’ and ‘specific’ hospital models of emergency patient flows. 2009(1386–9620 (Print)).Google Scholar
- Lewis RJ. Modeling complex systems: gaining valid insights and avoiding mathematical delusions. Acad Emerg Med. 2007;14(9):795–8. doi:10.1197/j.aem.2007.06.008.View ArticlePubMedGoogle Scholar
- Hwang U, McCarthy ML, Aronsky D, Asplin B, Crane PW, Craven CK, et al. Measures of crowding in the emergency department: a systematic review. Acad Emerg Med. 2011;18(5):527–38. doi:10.1111/j.1553-2712.2011.01054.x.View ArticlePubMedGoogle Scholar
- McCarthy ML, Aronsky D, Jones ID, Miner JR, Band RA, Baren JM, et al. The emergency department occupancy rate: a simple measure of emergency department crowding? Ann Emerg Med. 2008;51(1):15–24. doi:10.1016/j.annemergmed.2007.09.003. e1-2.View ArticlePubMedGoogle Scholar
- Pines JM. The left-without-being-seen rate: an imperfect measure of emergency department crowding. Acad Emerg Med. 2006;13(7):807. doi:10.1197/j.aem.2006.02.004. author reply −8.View ArticlePubMedGoogle Scholar
- Weiss SJ, Derlet R, Arndahl J, Ernst AA, Richards J, Fernandez-Frackelton M, et al. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med. 2004;11(1):38–50.View ArticlePubMedGoogle Scholar
- Sestoft B, Rohde CV. The complexity of the implementation of an observation and tracking chart for critical illness. Ugeskr, Laeger. 2011(1603–6824 (Electronic)).Google Scholar
- EDs. Danish Emergency Process Triage. Danish Emergency Departments. http://deptriage.dk/. Accessed 20 Apr 2015.
- Crane PW, Zhou Y, Sun Y, Lin L, Schneider SM. Entropy: A conceptual approach to measuring situation-level workload within emergency care and its relationship to emergency department crowding. J Emerg Med. 2014. doi:10.1016/j.jemermed.2013.08.113.
- Bernstein SL, Verghese V, Leung W, Lunney AT, Perez I. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med. 2003;10(9):938–42.View ArticlePubMedGoogle Scholar
- Epstein SK, Tian L. Development of an emergency department work score to predict ambulance diversion. Acad Emerg Med. 2006;13(4):421–6. doi:10.1197/j.aem.2005.11.081.View ArticlePubMedGoogle Scholar
- Beniuk K, Boyle AA, Clarkson PJ. Emergency department crowding: prioritising quantified crowding measures using a Delphi study. Emerg Med J. 2012;29(11):868–71. doi:10.1136/emermed-2011-200646.View ArticlePubMedGoogle Scholar
- Lin D, Patrick J, Labeau F. Estimating the waiting time of multi-priority emergency patients with downstream blocking. Health Care Manag Sci. 2014;17(1):88–99. doi:10.1007/s10729-013-9241-3.View ArticlePubMedGoogle Scholar
- Hurwitz JE, Lee JA, Lopiano KK, McKinley SA, Keesling J, Tyndall JA. A flexible simulation platform to quantify and manage emergency department crowding. BMC Med Inform Decis Mak. 2014;14:50. doi:10.1186/1472-6947-14-50.View ArticlePubMedPubMed CentralGoogle Scholar
- McCarthy ML, Ding R, Pines JM, Zeger SL. Comparison of methods for measuring crowding and its effects on length of stay in the emergency department. Acad Emerg Med. 2011;18(12):1269–77. doi:10.1111/j.1553-2712.2011.01232.x.View ArticlePubMedGoogle Scholar
- Wang H, Robinson RD, Bunch K, Huggins CA, Watson K, Jayswal RD, et al. The inaccuracy of determining overcrowding status by using the National ED overcrowding study tool. Am J Emerg Med. 2014;32(10):1230–6. doi:10.1016/j.ajem.2014.07.032.View ArticlePubMedGoogle Scholar
- Pines JM, Hilton JA, Weber EJ, Alkemade AJ, Al Shabanah H, Anderson PD, et al. International perspectives on emergency department crowding. Acad Emerg Med. 2011;18(12):1358–70. doi:10.1111/j.1553-2712.2011.01235.x.View ArticlePubMedGoogle Scholar
- Pines JM, Decker SL, Hu T. Exogenous predictors of national performance measures for emergency department crowding. Ann Emerg Med. 2012;60(3):293–8. doi:10.1016/j.annemergmed.2012.01.024.View ArticlePubMedGoogle Scholar
- Hoot NR, Zhou C, Jones I, Aronsky D. Measuring and forecasting emergency department crowding in real time. Ann Emerg Med. 2007;49(6):747–55. doi:10.1016/j.annemergmed.2007.01.017.View ArticlePubMedGoogle Scholar
- Jones SS, Allen TL, Flottemesch TJ, Welch SJ. An independent evaluation of four quantitative emergency department crowding scales. Acad Emerg Med. 2006;13(11):1204–11. doi:10.1197/j.aem.2006.05.021.View ArticlePubMedGoogle Scholar
- Flottemesch TJ, Gordon BD, Jones SS. Advanced statistics: developing a formal model of emergency department census and defining operational efficiency. Acad Emerg Med. 2007;14(9):799–809. doi:10.1197/j.aem.2007.05.011.View ArticlePubMedGoogle Scholar
- Madsen TL, Kofoed-Enevoldsen A. Five easy equations for patient flow through an emergency department. Dan Med Bull. 2011;58(10):A4318.PubMedGoogle Scholar
- McHugh M, van Dyke K, McClelland M, Moss D. Improving patient flow and reducing emergency department crowding: a guide for hospitals. Rockville: Agency for Healthcare Research and Quality; 2011.Google Scholar
- Bagust A, Place M, Posnett JW. Dynamics of bed use in accommodating emergency admissions: stochastic simulation model. BMJ. 1999;319(7203):155–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–42.View ArticlePubMedGoogle Scholar
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 370(9596):1453–7. http://dx.doi.org/10.1016/S0140-6736(07)61602-X.