Canadian in-hospital mortality for patients with emergency-sensitive conditions: a retrospective cohort study
BMC Emergency Medicine volume 19, Article number: 57 (2019)
The emergency department (ED) sensitive hospital standardized mortality ratio (ED-HSMR) measures risk-adjusted mortality for patients admitted to hospital with conditions for which ED care may improve health outcomes. This study aimed to describe in-hospital mortality across Canadian provinces using the ED-HSMR.
Hospital discharge data were analyzed from April 2009 to March 2012. The ED-HSMR was calculated as the ratio of observed deaths among patients with emergency-sensitive conditions in a hospital during a year (2010–11 or 2011–12) to the expected deaths for the same patients during the reference year (2009–10), multiplied by 100. The expected deaths were estimated using predictive models fitted from the reference year. Aggregated provincial ED-HSMR values were calculated. A HSMR value above or below 100 respectively means that more or fewer deaths than expected occurred within a province.
During the study period, 1,335,379 patients were admitted to hospital in Canada with an emergency-sensitive condition as the most responsible diagnosis. More in-hospital deaths (95% confidence interval) than expected were respectively observed for the years 2010–11 and 2011–12 in Newfoundland [124.3 (116.3–132.6); & 117.6 (110.1–125.5)] and Nova Scotia [116.4 (110.7–122.5) & 108.7 (103.0–114.5)], while mortality was as expected in Prince Edward Island [99.9 (86.5–114.8) & 100.7 (87.5–115.3)] and Manitoba [99.2 (94.5–104.1) & 98.3 (93.5–103.3)], and less than expected in all other provinces and territories.
Our study revealed important variation in risk-adjusted mortality for patients admitted to hospital with emergency-sensitive conditions among Canadian provinces. The ED-HSMR may be a useful outcome indicator to complement existing process indicators in measuring ED performance.
N/A – Retrospective cohort study.
On June 2016, Québec’s health and welfare commissioner published a report stating that the province had experienced years of the worst emergency department (ED) wait times in Canada and among high-income countries . Two years later, Québec data on ED length of stay (LOS) are still concerning and in fact all Canadian provinces lag behind many other jurisdictions in their attempts to grapple with this important public health issue [1, 2]. A compelling body of evidence associates overcrowding and prolonged ED LOS with adverse effects such as increased time to thrombolysis ; delays in antibiotics administration [3,4,5] and pain management [3, 6, 7]; patient dissatisfaction ; and an increased in-hospital and out of hospital mortality [3, 8,9,10]. However, metrics of ED operations only capture part of the quality picture as they focus only on time-based quality measures and do not include patient outcomes that remain the foremost validation of the performance of health systems and the ultimate targets for quality improvement initiatives .
Assessing health outcomes in the ED setting represent a significant challenge as most ED-relevant outcomes are usually observed in other care settings (e.g. ICU, hospital ward or home care) where patients have been transferred after initial ED management. Notwithstanding the complexity of linking outcomes to ED care, most expert guidelines still recommend the inclusion of outcome measures, such as mortality, to a comprehensive ED performance assessment framework [12,13,14].
The hospital standardized mortality ratio for emergency department sensitive conditions (ED-HSMR) represents such a metric [15,16,17]. Adapted from a methodology used by the Canadian Institute for Health Information (CIHI) for tracking overall in-hospital mortality , the ED-HSMR specifically measures risk-adjusted mortality for patients admitted to hospital with emergency sensitive conditions, which are conditions where ED management may potentially improve outcomes. Using a consensus methodology  and a national survey of ED care providers , 92 potential emergency-sensitive conditions (e.g. ectopic pregnancy) were identified and evaluated. Among these conditions, 37 mortality-related emergency sensitive diagnosis groups (e.g., A41 sepsis) from the 10th Canadian version of the International Classification of Disease (ICD-10-CA) were used to develop the emergency care sensitive HSMR variant. Easily retrieved from administrative databases, the ED-HSMR has been shown to be valid and reliable . It allows institutions or jurisdictions to follow their ED patient mortality over time and trigger internal performance reviews if trends are worrisome. It also provides opportunities for interprovincial comparisons of health outcomes observed among patients with emergency-sensitive conditions.
Consequently, to gain a broader understanding of ED performance and of the quality of care provided to emergency-sensitive conditions in Canada, this study aimed to describe in-hospital mortality across provinces using the ED-HSMR.
Study design and setting
We conducted a retrospective cohort study on national hospital discharge databases. We reviewed discharges from acute inpatient care institutions between April 1st 2009 and March 31st 2012 for nine provinces and three territories, and between April 1st 2009 and March 31st 2011 for the province of Québec. CIHI provided all administrative and patient-level data. Under the Canada Health Act, provinces and territories must provide universal health coverage to their citizens. All Canadian acute care facilities are publicly funded and owned.
We included all Canadian acute care facilities with an emergency department and hospital-based acute inpatient care. We excluded cancer centres, children’s hospitals and heart institutes because they treat specific populations with very different case-mixes. For risk-adjustment purpose, hospitals were classified into one of four peer-groups (teaching, large community, medium community and small community) based on academic designation, patient complexity and volume (see Additional file 1). This classification was adopted from the validated methodology used by the Canadian Institute for Health Information to calculate an all-cases HSMR .
We included all patients discharged dead or alive from hospital during the study period if they met the following criteria: 1) Admission to hospital through the ED; 2) Discharge from hospital with one of the 37 mortality-related emergency-sensitive diagnosis groups captured in the ED-HSMR (see Additional file 2); 3) Age between 29 days and 120 years at hospital admission; 4) Hospital length of stay equal to or less than 365 days; 5) Canadian resident. Patients were excluded if: 1) they were deceased at ED arrival; and 2) they were discharged against medical advice. Inclusion and exclusion criteria were directly derived from the methodology employed by CIHI to calculate the overall Canadian HSMR .
Characteristics of the institutions and patients
Where it appropriately applied, medians and interquartile ranges, means or proportions with 95% confidence intervals were used to describe the characteristics of the institutions and of the cases included. Unadjusted mortality rates for each one of the 37 emergency-sensitive diagnosis groups included in the ED-HSMR were calculated per hospital and per year of the study period.
ED-HSMRs were calculated for the fiscal years (April to March) 2010–2011 and 2011–2012 with the following equation:
A 95% confidence interval was calculated for each HSMR value using the Byar’s approximation . A HSMR value above 100 means that more deaths than expected occurred in an acute care facility. Conversely, a HSMR value below 100 means fewer deaths than expected occurred in an acute care facility. In other words, an ED-HSMR of 114 would indicate that each admission to a specific hospital for an emergency-sensitive condition has a probability of death 14% higher than what is expected from the Canadian average.
We estimated the expected number of deaths in 2010–11 or 2011–12 using fixed effects logistic regression models derived from the reference year (2009–10) for each hospital-peer group. After modelling mortality with different independent variables, the following covariates were retained in the final ED-HSMR predictive models: diagnosis groups, age (continuous), gender (dichotomous), in-hospital length of stay (6 groups: 1, 2, 3–9, 10–15, 16–21 and 22–365 days) and comorbidities (3 groups based on Charlson index score: Group 0 = score 0 (outside Québec) or scores 0 and 1 (Québec); Group 1 = scores 1 and 2 (outside Québec) or scores 2, 3 and 4 (Québec); Group 2 = scores 3 and more (outside Québec) or scores 5 and more (Québec); see Additional file 3). For managing missing data, we used a single imputation method, assigning most frequent values for categorical variables and medians, for continuous variables.
Probability of death at patient-level was calculated using the appropriate hospital-peer group specific model (teaching, large community, medium community and small community) from the reference year. After conversion from the log odds of death (pdeath = elog odds of death / [1 + (elog odds of death)]), all individual patient probabilities were summed to get the expected number of deaths in a specific hospital, in 2010–11 or 2011–12. An ED-HSMR for a specific acute care facility was only calculated if more than 20 deaths were expected within the study year at the institution, as fewer deaths yield unreliable and volatile HSMR measures .
The discriminatory power and calibration of the risk-adjustment models used to estimate the expected number of deaths for each hospital were reported in a previous publication . Areas under receiver operating characteristic curves of the predictive models used in the ED-HSMR were 0.80, 0.80, 0.80 and 0.81 for the teaching, large-community, medium-community and small-community peer-group hospitals, respectively.
Hospital-level and aggregated provincial ED-HSMRs
We report hospital-level ED-HSMRs through tables and caterpillar plots. Stratifications by peer-groups and provinces are graphically represented. Aggregated provincial ED-HSMR values were calculated by dividing the sum all observed (O) deaths with the sum of all expected (E) number of deaths of all institutions of a province or a territory (O/E × 100). All patients were included in aggregated measures, even those from hospitals with less than 20 expected deaths where no site-specific HSMR could be estimated. Analyses were performed using Stata version MP 11.2 (StataCorp, TX, USA).
Characteristics of cases and hospitals
During the 3-year study period, 1,335,379 patients were admitted to 629 hospitals across 11 provinces and territories from the ED with one of the 37 mortality-related emergency sensitive diagnosis groups captured in the ED-HSMR as the most responsible diagnosis. Table 1 describes their characteristics. Half of the cohort was composed of females older than 73 years and hospitalized for 5 days or less. Our study population presented a low comorbidity burden as more than 80% of patients had a Charlson index score of 0 or 1. Small hospitals represented 58% of all hospitals included in our study, but treated only 10.8% of all patients admitted. Hospital distribution was markedly different in the province of Québec where 42.5% of all hospitals were either teaching or large-community institutions. Provinces and Territories in the rest of Canada had a higher proportion of small-community hospitals (Table 2).
Overall and diagnosis group specific mortality rates
For the 3 years of the study period (including the reference year), overall mortality in our cohort was 8.9%, with chronic obstructive pulmonary disease, pneumonia, heart failure, acute myocardial infarction, stroke/cerebral infarction, and sepsis accounting for 55.5% of all deaths (Table 3). Diagnosis groups with the highest mortality rates were cardiac arrest (65.1%) and shock not elsewhere classified (50.0%).
The ED-HSM was estimated for 46.7 and 33.9% (Québec data not available) of all Canadian hospitals in 2010 and 2011, respectively (see Additional files 4 and 5). Among hospitals with sufficient number of expected deaths to calculate the ED-HSMR, 22.8% in 2010 and 31.5% in 2011 had a HSMR 95% confidence interval falling below the 100 threshold (fewer deaths than expected), and 10.5% in 2010 and 7.0% in 2011 had a HSMR 95% confidence interval falling above (more deaths than expected). Results are stratified by province and hospital-peer group in Table 4.
Provincial aggregated ED-HSMRs
Figure 1 illustrates the aggregated ED-HSMR estimates by province for 2010–11 and 2011–12. In both years, Nova Scotia and Newfoundland-Labrador had provincial ED-HSMRs higher than 100, while the 95% confidence intervals of the aggregated measures for Prince Edward Island and Manitoba crossed the 100 threshold. All other provinces and territories experienced fewer deaths than what was expected from mortality trends recorded in Canada during the reference year (2009–10).
We analyzed adjusted mortality rates in Canada using the ED-HSMR. Our results show that there is significant interhospital and interprovincial variation in mortality trends for patients admitted with emergency-sensitive conditions and that this variation appears to be consistent over the 2 years studied. The ED-HSMR could be used to measure the performance of Canadian hospital systems in emergency care and guide quality improvement initiatives.
To the best of our knowledge, we are the first to report on variations in mortality across Canadian provinces for a comprehensive list of emergency sensitive conditions. Two recent studies published in 2017 have reported mortality trends in Canada for two domains overlapping with emergency medicine: trauma and acute myocardial infarction (AMI) care. Both studies pointed out interprovincial disparities in the management of these acutely ill populations. Tran et al. examined Canadian and provincial 30-day in-hospital mortality rates after percutaneous coronary interventions (PCI) and coronary artery bypass grafting (CABG) between 2004 and 2013. Ten-year adjusted odds (2004–2013) of mortality were higher after PCI in Saskatchewan and after CABG in Newfoundland and Labrador . Similarly, Moore et al. reviewed data on 78,707 trauma patients in 7 provinces from 2006 to 2012. Authors reported that Nova Scotia, Newfoundland and Labrador, and Manitoba were the provinces with the highest risk-adjusted mortality rates in 2011 and 2012 ..
Our analyses yielded similar results with Newfoundland/Labrador, Nova Scotia, Manitoba and Prince Edward Island having the highest ED-HSMR estimates. Although our composite mortality indicator captures some trauma-related diagnoses (e.g. intracranial injury) and AMI, it is meant to employ a broad approach for identifying mortality trends involving 37 emergency sensitive conditions (e.g. sepsis). As a result, trauma and AMI care may explain a part of, but not all observed disparities reported in our own study. Moreover, mortality for emergency sensitive conditions appears to be higher than expected in provinces where the proportion of the population living in rural communities is highest . This finding echoes previous reports that highlighted significant disparities in access to high-quality emergency care in Canadian rural communities [23, 24]. In the same way it was done for trauma care across the country, Canadian decision- and policy-makers could use the ED-HSMR to trigger in-depth performance assessment to help identify opportunities for improving emergency care structures and processes to improve patient outcomes.
Since 2005, CIHI calculates annually for all Canadian hospitals a HSMR based on 72 diagnosis groups (e.g. Alzheimer’s disease) explaining 80% of all Canadian in-hospital deaths. Although using the same methodology, our HSMR variant differs significantly from CIHI’s by capturing only those conditions for which ED care may potentially influence mortality (e.g. pulmonary embolism). Many reports have advocated caution in using HSMRs or any other mortality indicators [19, 25]. Incomplete risk adjustment, low rate of preventable deaths and inconstant concordance between mortality and other quality indicators are some of the limitations impeding inference we could draw from mortality metrics to potential quality of care breaches. Moreover, in-hospital mortality is usually remote from the ED and occurs on hospital wards, making it virtually impossible to completely isolate ED influence on patient outcomes. However, as imperfect as it may be, other reports have shown that tracking mortality rates can drive system-level changes for improving patient care and healthcare organizations [26,27,28,29,30]..
Similar problems arise with ED wait time metrics. Like mortality rates, they are reflections of a care system rather than processes of care specific to the ED. Blocked access to in-hospital beds, limited primary care resources for management of patients with multimorbidities and a lack of patient education on appropriate use of EDs are some of the factors that can impact ED flow, but are not caused per se by poor ED care. As for ED wait time metrics, the ED-HSMR could serve as a strong incentive for hospital departments and community health care resources to move from siloed approached to quality improvement and work together to improve outcomes of patients with emergency sensitive conditions.
The Québec’s health and welfare commissioner report published in 2016 raises important questions. As worrisome as they are, wait times henceforth do not appear to be the only important risk factor impacting outcomes of ED patients with emergency sensitive conditions. Indeed, Québec ED-HSMR in 2010–11 reveals fewer deaths than expected. This complementary information highlights the importance of synergistically using structure, process and outcome metrics when evaluating health care quality. We believe that all jurisdictions should adopt a comprehensive quality framework that would include among other metrics the ED-HSMR.
This study has inherent limitations of all studies using administrative database sources. Previous reports have shown that coding of administrative data is usually accurate and inaccuracies had modest effect on an HSMR [31,32,33]. The ED-HSMR does not adjust for all potential confounders, such as severity of disease, smoking habits or socio-economic status. Although severity of disease and smoking habits are not easily extractable from existing hospital databases, socio-economic status can be assessed using deprivation index derived from patient postal codes. Future iteration of the ED-HSMR could test socio-economic status as a potential explanatory variable. Furthermore the ED-HSMR is not a performance measure suitable for small hospitals because of the low numbers of observed and expected in-hospital deaths. Small hospitals represent 58% of all Canadian hospitals. Further research is needed to identify outcome measures that could be used to assess the quality of care provided in low-volume institutions. Finally, we acknowledge that our data are several years old. However, we believe that our results demonstrate the feasibility of using an ED sensitive condition hospital standardized mortality ratio to measure ED performance as a way to supplement existing wait times and access-to-care indicators.
In conclusion, we analyzed Canadian in-hospital mortality rates using the ED-HSMR. Our study revealed important variation in risk-adjusted mortality for patients admitted to hospital with emergency-sensitive conditions among Canadian provinces. These results warrant in-depth evaluations to understand the root causes of the observed regional variation. The ED-HSMR may be a useful outcome indicator to complement existing process indicators in measuring ED performance.
Availability of data and materials
The datasets generated and analysed during the current study are not publicly available. Data were provided by the Canadian Institute for Health Information with an agreement to destroy in whole all datasets after study completion.
Canadian Institute for Health Information
Hospital standardized mortality ratio for emergency department sensitive conditions
Intensive care unit
Length of stay
Commissaire à la santé et au bien-être. Apprendre des meilleurs: Étude comparative des urgences du Québec 2016 [Available from: http://www.csbe.gouv.qc.ca/fileadmin/www/2016/Urgences/CSBE_Rapport_Urgences_2016.pdf.
Canadian Institute for Health Information. How Canada Compares: Results From The Commonwealth Fund’s 2016 International Health Policy Survey of Adults in 11 Countries 2017 [Available from: https://www.cihi.ca/sites/default/files/document/text-alternative-version-2016-cmwf-en-web.pdf.
Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10.
Kennebeck SS, Timm NL, Kurowski EM, Byczkowski TL, Reeves SD. The association of emergency department crowding and time to antibiotics in febrile neonates. Acad Emerg Med. 2011;18(12):1380–5.
Pines JM, Localio AR, Hollander JE, Baxt WG, Lee H, Phillips C, et al. The impact of emergency department crowding measures on time to antibiotics for patients with community-acquired pneumonia. Ann Emerg Med. 2007;50(5):510–6.
Barrett TW, Schriger DL. Annals of emergency medicine journal Club. Emergency department crowding is associated with poor care for patients with severe pain. Ann Emerg Med. 2008;51(1):6–7.
Sills MR, Fairclough DL, Ranade D, Mitchell MS, Kahn MG. Emergency department crowding is associated with decreased quality of analgesia delivery for children with pain related to acute, isolated, long-bone fractures. Acad Emerg Med. 2011;18(12):1330–8.
Singer AJ, Thode HC Jr, Viccellio P, Pines JM. The association between length of emergency department boarding and mortality. Acad Emerg Med. 2011;18(12):1324–9.
Guttmann A, Schull MJ, Vermeulen MJ, Stukel TA. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada. BMJ. 2011;342:d2983.
Carter EJ, Pouch SM, Larson EL. The relationship between emergency department crowding and patient outcomes: a systematic review. J Nurs Scholarsh. 2014;46(2):106–15.
Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):166–206.
Jones P, Harper A, Wells S, Curtis E, Carswell P, Reid P, et al. Selection and validation of quality indicators for the shorter stays in emergency departments National Research Project. Emerg Med Australasia : EMA. 2012;24(3):303–12.
Schull MJ, Guttmann A, Leaver CA, Vermeulen M, Hatcher CM, Rowe BH, et al. Prioritizing performance measurement for emergency department care: consensus on evidence-based quality of care indicators. CJEM. 2011;13(5):300–9 E28-43.
Madsen M, Kiuru S, Castren M, Kurland L. The level of evidence for emergency department performance indicators: systematic review. Eur J Emerg Med. 2015;22(5):298–305.
Berthelot S, Lang ES, Quan H, Stelfox HT. What are emergency-sensitive conditions? A survey of Canadian emergency physicians and nurses. CJEM. 2015;17(2):154–60.
Berthelot S, Lang ES, Quan H, Stelfox HT. Development of a hospital standardized mortality ratio for emergency department care. Ann Emerg Med. 2016;67(4):517–24 e26.
Berthelot S, Lang ES, Quan H, Stelfox HT. Panel on emergency-sensitive C. identifying emergency-sensitive conditions for the calculation of an emergency care inhospital standardized mortality ratio. Ann Emerg Med. 2014;63(4):418–24 e2.
Canadian Institute for Health Information. Technical Notes: Hospital Standardized Mortality Ratio (HSMR) Ottawa 2018 [Available from: https://www.cihi.ca/sites/default/files/document/hsmr-tech-notes_en_0.pdf.
Brien SE, Ghali WA. CIHI’s hospital standardized mortality ratio: friend or foe? HealthcarePapers. 2008;8(4):57–61 discussion 9-75.
Tran DT, Welsh RC, Ohinmaa A, Thanh NX, Bagai A, Kaul P. Quality of acute myocardial infarction Care in Canada: a 10-year review of 30-day in-hospital mortality and 30-day hospital readmission. Can J Cardiol. 2017;33(10):1319–26.
Moore L, Stelfox HT, Evans D, Hameed SM, Yanchar NL, Simons R, et al. Trends in injury outcomes across Canadian trauma systems. JAMA Surg. 2017;152(2):168–74.
Institut de la statistique du Québec. Tableau statistique canadien 2019 [Available from: http://www.stat.gouv.qc.ca/statistiques/economie/comparaisons-economiques/interprovinciales/portraits.pdf.
Fleet R, Bussieres S, Tounkara FK, Turcotte S, Legare F, Plant J, et al. Rural versus urban academic hospital mortality following stroke in Canada. PLoS One. 2018;13(1):e0191151.
Fleet R, Lauzier F, Tounkara FK, Turcotte S, Poitras J, Morris J, et al. Profile of trauma mortality and trauma care resources at rural emergency departments and urban trauma centres in Quebec: a population-based, retrospective cohort study. BMJ Open. 2019;9(6):e028512.
Goodacre S, Campbell M, Carter A. What do hospital mortality rates tell us about quality of care? Emerg Med J. 2015;32(3):244–7.
Shen YC, Wu VY. Reductions in Medicare payments and patient outcomes: an analysis of 5 leading Medicare conditions. Med Care. 2013;51(11):970–7.
Nguyen HB, Corbett SW, Steele R, Banta J, Clark RT, Hayes SR, et al. Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med. 2007;35(4):1105–12.
Jarman B. In defence of the hospital standardized mortality ratio. HealthcarePapers. 2008;8(4):37–42 discussion 69-75.
Joynt KE, Harris Y, Orav EJ, Jha AK. Quality of care and patient outcomes in critical access rural hospitals. JAMA. 2011;306(1):45–52.
Popowich J, Zaborowski J, Bellows M. Hospital standardized mortality ratios: a tale of two sites. Healthc Q. 2011;14(2):42–8.
Ben-Tovim DI, Pointer SC, Woodman R, Hakendorf PH, Harrison JE. Routine use of administrative data for safety and quality purposes--hospital mortality. Med J Aust. 2010;193(8 Suppl):S100–3.
Bottle A, Jarman B, Aylin P. Hospital standardized mortality ratios: sensitivity analyses on the impact of coding. Health Serv Res. 2011;46(6pt1):1741–61.
Safwenberg U, Terent A, Lind L. Differences in long-term mortality for different emergency department presenting complaints. Acad Emerg Med. 2008;15(1):9–16.
To conduct this study, the authors received unrestricted grants from the Département de médecine familiale et de médecine d’urgence de l’Université Laval and from the Department of Emergency Medicine, University of Calgary. These funding bodies had no role in the design of the study; collection, analysis, and interpretation of data; and in writing the manuscript.
Ethics approval and consent to participate
This study received ethics approval from the Conjoint Health Research Ethics Board at the University of Calgary (E-24580). Since it is a retrospective study on a large anonymized administrative dataset, patient consent was not required.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Hospital peer-groups definition
List of the Diagnosis Groups (n = 37) of the International Classification of Diseases (10th version) included in the ED-HSMR
Charlson index score groups
Distribution of ED-HSMRs of all eligible institutions (n = 294) in 2010–11
Caterpillar plots of ED-HSMRs by peer-group
About this article
Cite this article
Berthelot, S., Lang, E.S., Quan, H. et al. Canadian in-hospital mortality for patients with emergency-sensitive conditions: a retrospective cohort study. BMC Emerg Med 19, 57 (2019). https://doi.org/10.1186/s12873-019-0270-1
- Emergency department
- Standardized mortality ratio
- Emergency sensitive conditions