- Research article
- Open Access
- Open Peer Review
Prediction of 90-day mortality in older patients after discharge from an emergency department: a retrospective follow-up study
BMC Emergency Medicine volume 16, Article number: 26 (2016)
Older people frequently attend the emergency department (ED) and have a high risk of poor outcome as compared to their younger counterparts. Our aim was to study routinely collected clinical parameters as predictors of 90-day mortality in older patients attending our ED.
We conducted a retrospective follow-up study at the Leiden University Medical Center (The Netherlands) among patients aged 70 years or older attending the ED in 2012. Predictors were age, gender, time and way of arrival, presenting complaint, consulting medical specialty, vital signs, pain score and laboratory testing. Cox regression analyses were performed to analyse the association between these predictors and 90-day mortality.
Three thousand two hundred one unique patients were eligible for inclusion. Ninety-day mortality was 10.5 % for the total group. Independent predictors of mortality were age (hazard ratio [HR] 1.06, 95 % confidence interval [95 % CI] 1.04-1.08), referral from another hospital (HR 2.74, 95 % CI 1.22-6.11), allocation to a non-surgical specialty (HR: 1.55, 95 % CI 1.13-2.14), increased respiration rate (HR up to 2.21, 95 % CI 1.25-3.92), low oxygen saturation (HR up to 1.96, 95 % CI 1.19-3.23), hypothermia (HR 2.27, 95 % CI 1.28-4.01), fever (HR 0.43, 95 % CI 0.24-0.75), high pain score (HR 1.55, 95 % CI 1.03-2.32) and the indication to perform laboratory testing (HR 3.44, 95 % CI 2.13-5.56).
Routinely collected parameters at the ED can predict 90-day mortality in older patients presenting to the ED. This study forms the first step towards creating a new and simple screening tool to predict and improve health outcome in acutely presenting older patients.
Older patients frequently attend emergency departments (EDs) in comparison with younger adults [1, 2]. Admittance to the ED is associated with risk of negative health outcomes such as functional decline  and mortality . However, little is known about predictors of mortality in the period after presentation to the ED in older patients.
Predictors of poor outcome in older patients can be divided into two categories. On one hand, there is the level of vulnerability of the older patient, which is reflected in for instance multi-morbidity, poly-pharmacy, functional capacity and cognitive and social functioning . Frequently studied prediction tools such as the Identification of Seniors At Risk  and the Triage Risk Screening Tool  are based on these parameters. On the other hand, parameters reflecting severity of disease at presentation may also determine poor outcome . Specific diagnoses are well known predictors of mortality but are very numerous and hard to categorise, partly due to the large heterogeneity of older patients, especially in the presence of multi-morbidity . Other, more generic data on severity of disease are routinely recorded as part of medical practice, e.g., time of arrival , vital signs  and laboratory parameters , and may also predict poor outcome. However, little is known about their association with mortality in older patients in the period after discharge from the ED. Identifying such predictors may enable us to design an adequate screening tool in order to target older patients at high risk of negative health outcome early during ED admittance. A screening tool may enable fast-tracking patients that are likely to be admitted to an inpatient ward and shorten their stay at the ED. In case of high risk of mortality, advanced care planning may be initiated at the ED or shortly after admission, or rehabilitation in case of high risk of functional decline.
Our aim was to study whether routinely recorded parameters in the ED, such as way and time of arrival, vital signs and laboratory results independently predict 90-day mortality. We performed a retrospective follow-up study among patients aged 70 years or older visiting our ED.
Our study was conducted at the ED of the Leiden University Medical Center, a tertiary university teaching and level 1 trauma hospital in the Netherlands. Patients aged 70 years and older that had attended the ED between 1 January 2012 and 31 December 2012 were included retrospectively. The Medical Ethics Committee of the Leiden University Medical Center waived the obligation of approval as data were collected in the past as part of routine clinical care.
Health care in the Netherlands
The Netherlands is a small and highly populated country in Europe measuring 41.5 thousand square kilometres  and counting 16.7 million people in 2012 . Standard medical care is equally accessible for every Dutch citizen through legally mandatory health insurance. Primary care is provided by general practitioners (GPs). Specialist care can only be accessed after referral by a GP. One of the exceptions are EDs of hospitals, where a substantial proportion of patients are self-referred . The Leiden University Medical Center is a tertiary referral centre in Leiden. The ED is one of two level 1 trauma EDs that together serve a catchment area of 400.000 inhabitants, both urban and rural. The population is predominantly Caucasian and includes all social classes. Our ED is equipped with 15 rooms of which three are specially designed to accommodate trauma victims. Patients are triaged by an ED nurse. Within hours self-referred patients are evaluated by an ED physician or ED resident. Out of hours self-referred patients are primarily evaluated by a GP and if indicated subsequently referred to an ED physician or ED resident. Referred patients are directly allocated to a resident of the appropriate medical specialty present at the ED. After evaluation patients are either treated at the ED and discharged home or admitted to an inpatient ward. Patients with an electrocardiogram indicative for myocardial infarction bypass our ED and are immediately referred to the catheterisation laboratory . As a consequence, they are not included in the present study.
Selection of study population
Patients were identified in our computerised patient record system (ChipSoft-EZIS®, version 5.2, 2006–2014, Amsterdam, The Netherlands, www.chipsoft.nl). Several steps of exclusion criteria were applied. Our study was aimed at a selection of older patients that may benefit from additional interventions during or following an ED visit. First, medical records based upon unjustified ED use were excluded. Unjustified ED use was defined as ED use for any other reason than acute medical care, such as outpatient check-ups on weekends, plaster cast readjustments, performed blood tests for other medical departments and patients who decided to leave the ED before medical attention was bestowed. We believe these are not representative for the acutely presenting older patient visiting the ED and may disturb associations between predictors and outcome results. Second, patients who deceased in the ED and patients receiving cardiopulmonary resuscitation therapy upon arrival were excluded from analysis since prognosis of these patients is known to be poor and these patients fall outside the scope for identifying new predictors . As we used retrospective data, we were unable to assess whether an ED visit was the first or one of many visits. Patients may have visited other hospitals as well as ours or made visits outside our selected timeframe. Therefore, we included only the first ED visit of each patient in 2012.
Apart from demographic characteristics (age and gender), we selected routinely collected parameters that may reflect severity of disease as presented in the acute situation. We investigated time and way of arrival, presenting complaint, consulting medical specialty, vital signs, pain score and laboratory parameters. These data were automatically generated from the digital patient records and outliers were manually checked for validity by a researcher. Triage category was not included since we were interested in universal predictors and hospitals differ in the triage systems they use.
Time of ED visit was determined from ED registration time and subdivided in three categories, day (08.00 h–15.59 h), evening (16.00 h–23.59 h) and night time (00.00 h–07.59 h). Way of arrival at the ED was mutually exclusively noted as self-referral, brought in by ambulance, referral by a GP, internal referral from another department or referral by another hospital. Patients categorised as self-referral or referral by a GP visited the ED with private transportation. By contrast, patients who arrived by ambulance were categorised as brought in by ambulance regardless of whether the ambulance was ordered by a referring GP or because of an emergency call. Dutch ambulance staff is trained to judge the accuracy of emergency calls at the scene. Ambulance staff will only transport such patients to the hospital if they consider the referral justified. At our hospital, triage is based on the Manchester Triage System (MTS) . This system uses flow charts for 55 disease presentations to determine the level of urgency and associated target time a patient should receive care from a physician. The presenting complaints of our study population were categorised according to these MTS disease presentations . Disease presentations occurring in less than 3 % of patients were merged as ‘other’. The medical specialty a patient was assigned to was categorised as surgical or non-surgical . Finally, we listed clinical measurements that were recorded in the ED: vital signs, pain score and laboratory results. At triage, an ED nurse determined which clinical measurements were medically indicated according to protocols. They were measured at triage or soon after a patient was placed into a treatment room. Laboratory testing is performed on indication and either ordered by an ED nurse or consulting physician. The first set of vital signs assessed in the ED was recorded. Vital signs were categorised according to the Modified Early Warning Score and included systolic blood pressure, heart rate, respiratory rate and body temperature . Oxygen saturation was recorded as well . Categories containing less than 1 % of patients were combined with adjacent categories, but not with the reference category, in order to minimise the number of categories. Pain was evaluated using the Numeric Rating Scale (NRS) rating from 0 to 10 and categorised as no or light (NRS 0–3), mild (NRS 4–6) and serious (7–10) pain according to the Dutch guidelines for pain classification in emergency settings . Blood pressure, heart rate, respiratory rate and oxygen saturation were measured using a medical monitor (IntelliVue MP50®, Eindhoven, The Netherlands, www.philips.nl/healthcare). Body temperature was determined by a tympanic thermometer (Genius 2®, Mansfield, USA, www.covidien.com). Registered laboratory results were haemoglobin, thrombocytes, leukocytes, C-reactive protein, sodium, potassium, creatinine, urea, troponin T and non-fasted glucose. Vital signs and laboratory parameters will only be assessed if there is a medical indication to do so. If data on vital signs were missing, they were either not measured or they were measured but not recorded in the medical chart correctly. It is impossible to categorise this in a retrospective manner. Therefore, we assumed that missing vital signs meant that there was no indication to perform these measurements.
Primary end point
Our primary outcome measure was mortality in the first three months after ED admittance. Beyond this time period, the association of predictors measured at baseline and mortality is likely to be obscured by the occurrence of new medical events. Mortality data were acquired from the municipal personal records database on 1 May 2014.
Data are displayed as mean and standard deviation if normally distributed and median and interquartile range if not normally distributed. To investigate the association between predictors and mortality we used Cox proportional hazards models. We performed uni- and multivariate Cox regression analysis. In the univariate models only one parameter was entered as independent variable. In the multivariate analyses, multiple parameters were entered as independent variables simultaneously to assess which were independent predictors of mortality. Our study was aimed at potential predictors assessed upon or soon after arrival at the ED. Results of laboratory testing became available at least one hour after withdrawal, but laboratory testing is usually ordered in the first few minutes after a patient is placed into a treatment room. Therefore, we added merely the medical indication to perform laboratory testing to the set of predictors in the multivariate model. As an in-depth analysis we have additionally analysed the univariate association of individual laboratory results with mortality using univariate Cox regression. The level of significance was set at P < 0.05. All statistical analyses were performed using IBM SPSS Statistics package (version 20).
During 2012, there were 27.862 Emergency Department (ED) visits of which 4458 (16 %) visits were by patients aged 70 years or older. Visits were excluded because of inappropriate ED use (n = 136), receiving cardiopulmonary resuscitation upon arrival (n = 67) and patients who deceased in the ED (n = 5). This left 4250 suitable ED presentations of which 959 were repeat visits, leaving 3291 unique patients eligible for the analyses (Fig. 1).
Baseline characteristics of the study population are described in Table 1. Median age was 78.3 years (interquartile range 74.0–83.6 years) and 53.1 % was female. Most patients arrived by ambulance (35.2 %) or with private transportation after referral by their GP (33.7 %). Patients were assigned to a non-surgical specialty in 58.3 % of cases. Mortality rate at 30 days after ED presentation was 7.0 % and increased to 10.5 % at 90 days after an ED visit (Fig. 2).
Regression analyses were performed to investigate the association between routinely assessed predictors in the ED and mortality in the first 90 days of follow-up (Table 2). A substantial portion of the univariate associations remained significant in the multivariate model i.e., age (hazard ratio [HR] 1.06, 95 % confidence interval [CI] 1.04-1.08), referral by another hospital (HR 2.74, 95 % CI 1.22-6.11), presenting complaint classified as ‘unwell’ (HR 1.99, 95 % CI 1.23-3.20), allocation to a non-surgical specialty (HR 1.55, 95 % CI 1.13-2.14), increased respiration rate (21–29 breaths per minute [bpm]: HR 1.63, 95 % CI 1.06-2.52; ≥30 bpm: HR 2.21, 95 % CI 1.25-3.92), decreased oxygen saturation (91–94 %: HR 1.63, 95 % CI 1.16-2.31; ≤90 %: HR 1.96, 95 % CI 1.19-3.23), hypothermia (HR 2.27, 95 % CI 1.28-4.01), fever (HR 0.43, 95 % CI 0.24-0.75), high pain score (HR 1.55, 95 % CI 1.03-2.32) and the indication to perform blood tests (HR 3.44, 95 % CI 2.13-5.56).
Table 3 demonstrates how abnormal versus normal laboratory results relate to mortality risk among patients who had an indication for performing blood tests. The majority of abnormal laboratory results show an increased hazard as compared to measurements within normal range. Strongest associations were a high level of troponin T (HR 3.26, 95 % CI 2.47-4.30), thrombocytes (HR 3.18, 95 % CI 2.11-4.80) and leukocytes (HR 2.50, 95 % CI 1.99-3.14). Patients for whom no laboratory tests were performed had a significantly decreased mortality risk in comparison with patients whose laboratory results were within reference range. For instance, hazard ratio for patients without a sodium measurement was 0.36 (95 % CI 0.26-0.52) as compared to patients with a sodium measurement within reference range.
The main finding of the present study is that routinely, at entrance assessed, clinical parameters can be used to predict 90-day mortality in older persons admitted to the emergency department (ED). Independent predictors of 90-day mortality risk included: increasing age, referral by another hospital, disease presentation categorised as ‘unwell’, allocation to a non-surgical specialty, low respiration rate, low oxygen saturation, body temperature and the performance of blood tests. In addition, abnormal laboratory results, which become known at a later stage during an ED visit, are univariately associated with increased mortality risk. Patients for whom no laboratory tests were performed showed a decreased mortality risk.
Potential predictors of poor outcome in acutely presenting older adults have been studied before. Like in our study, increasing age was shown to associate with in-hospital mortality , as well as mortality risk 1 year after presentation . Our research aimed at predictors known upon or soon after arrival of a patient at the ED in order to investigate their potential for new screening instruments. Other researchers also included predictors into their models that become available at a later stage during an ED visit, such as length of stay at the ED [21, 22]. Kennelly et al. found an association between arrival by ambulance and mortality, whereas our study did not . Van Walraven et al. developed the hospital-patient one-year mortality risk (HOMR) model . The HOMR model assesses 1 year mortality risk for adults ≥18 years who are acutely hospitalised, but it was not validated for ED visitors who were directly discharged without admittance to an inpatient ward. In addition, previous research shows that abnormal vital signs at triage associate with intensive care unit admission and in-hospital mortality in patients from the age of 16  as well as in older patients from the age of 75 . Furthermore, a high Modified Early Warning Score can be used to predict a worse in-hospital stay (e.g., mortality and hospitalisation) in older adults . Our study demonstrates that respiration rate, oxygen saturation, body temperature and pain score associate with 90-day mortality independent of other risk factors. Systolic blood pressure and heart rate did not remain significantly associated with mortality in the multivariate model. However, anatomical and physiological changes that occur with ageing may limit older people to generate an adequate response to injury . As a consequence, some vital signs may not be reliable in reflecting the actual condition of an older patient .
Managing older people in the ED can be complex because of atypical disease presentation, poly-pharmacy and multiple co-morbidities. Risk factors for adverse health outcomes include functional dependence, lack of social support and cognitive impairment . Many risk factors and frailty screening tools such as the ‘Identification of Seniors at Risk’ have been evaluated in their ability to predict health outcome in older adults. Individually, they all lack sufficient prognostic accuracy to identify patients at high risk for poor outcome . We found that routinely collected clinical parameters associate with mortality in older patients admitted to the ED. Although this is not unexpected, it implies that early assessed characteristics of an ED visit are not only of value with respect to short term outcomes, but may be useful when considering the period after discharge as well. Models including both disease specific parameters (for example respiration rate) and parameters reflecting functional and cognitive status may give rise to a more complete assessment of the older individual. Our findings lay ground for creating new prediction models using routinely collected parameters alongside frailty characteristics in order to adequately predict outcome in acutely presenting older patients. We are currently performing prospective studies to develop and validate such predictive models with respect to multiple negative endpoints such as mortality, admission rate, quality of life and functional status (www.apop.eu ). These prediction models should be able to detect patients at high risk for poor outcome and enable the development of appropriate interventions to improve acute medical care for older patients.
The present study was limited by its retrospective nature and could not provide reliable information on frailty characteristics such as multi-morbidity, poly-pharmacy and functional and cognitive impairment and these characteristics could not be studied in our model. However, it is unlikely that the investigated predictors in our study would change when collected in a prospective matter. Our study was set at a single centre tertiary referral hospital which may make our results less generalisable. Strong points of our study were the large sample size of over three thousand ED visits, the use of universal predictors that were likely to be free of bias and the fact that mortality is a very robust end point of which data were available for all patients through municipality records. Our study is unique in the fact that we investigated predictors early known during an ED visit which may be suitable for a screening instrument.
A proper screening instrument that identifies older patients at risk of poor outcome is the first step towards changing outcome. We aim that a screening instrument will enable us to set up special care trajectories in order to improve recovery after acute presentation at the ED. These tailored trajectories could include extra attention on rehabilitation, prevention of delirium and advanced care planning and are currently investigated in a prospective study concerning the acutely presenting older patient (‘APOP study’ ).
Routinely collected parameters of older persons attending the ED can be used to predict 90-day mortality. This survey constitutes preparatory work towards creating a proper screening instrument for predicting and improving health outcome in acutely presenting older patients.
°C, degrees Celcius; APOP, acutely presenting older patient; BP, blood pressure; bpm, breaths per minute; BPM, beats per minute; CI, confidence interval; CPR, cardiopulmonary resuscitation; ED, emergency department; h, hours; HR, hazard ratio; IQR, interquartile range; L, liter; mg, milligram; mmol, millimol; N, number; NRS, numeric rating scale; sd, standard deviation; μmol, micromol
Samaras N, Chevalley T, Samaras D, Gold G. Older patients in the emergency department: a review. Ann Emerg Med. 2010;56(3):261–9.
Aminzadeh F, Dalziel WB. Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med. 2002;39(3):238–47.
Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K. Frailty in elderly people. Lancet. 2013;381(9868):752–62.
Buurman BM, Hoogerduijn JG, de Haan RJ, Abu-Hanna A, Lagaay AM, Verhaar HJ, et al. Geriatric conditions in acutely hospitalized older patients: prevalence and one-year survival and functional decline. PLoS One. 2011;6(11):e26951.
McCusker J, Bellavance F, Cardin S, Trepanier S, Verdon J, Ardman O. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc. 1999;47(10):1229–37.
Hustey FM, Mion LC, Connor JT, Emerman CL, Campbell J, Palmer RM. A brief risk stratification tool to predict functional decline in older adults discharged from emergency departments. J Am Geriatr Soc. 2007;55(8):1269–74.
Dundar ZD, Ergin M, Karamercan MA, Ayranci K, Colak T, Tuncar A, et al. Modified Early Warning Score and VitalPac Early Warning Score in geriatric patients admitted to emergency department. Eur J Emerg Med. 2015. doi:10.1097/MEJ.0000000000000274.
Chotirmall SH, Picardo S, Lyons J, D’Alton M, O’Riordan D, Silke B. Disabling disease codes predict worse outcomes for acute medical admissions. Intern Med J. 2014;44(6):546–53.
Maggs F, Mallet M. Mortality in out-of-hours emergency medical admissions—more than just a weekend effect. J R Coll Physicians Edinb. 2010;40(2):115–8.
van Houwelingen AH, den Elzen WP, Mooijaart SP, Heijmans M, Blom JW, de Craen AJ, et al. Predictive value of a profile of routine blood measurements on mortality in older persons in the general population: the Leiden 85-plus Study. PLoS One. 2013;8(3):e58050.
Surface area. The World Bank Group. http://data.worldbank.org/indicator/AG.SRF.TOTL.K2. Accessed 28 June 2016.
Population; key figures. Statistics Netherlands, Den Haag/Heerlen. 2015. http://statline.cbs.nl/Statweb/publication/?DM=SLEN&PA=37296eng&D1=a&D2=0,10,20,30,40,50,62&LA=EN&HDR=G1&STB=T&VW=T. Accessed 28 June 2016.
Schafer W, Kroneman M, Boerma W, van den Berg M, Westert G, Deville W, et al. The Netherlands: health system review. Health Syst Transit. 2010;12(1):v–xxvii. 1–228.
Atary JZ, de Visser M, van den Dijk R, Bosch J, Liem SS, Antoni ML, et al. Standardised pre-hospital care of acute myocardial infarction patients: MISSION! guidelines applied in practice. Neth Heart J. 2010;18(9):408–15.
Herlitz J, Eek M, Engdahl J, Holmberg M, Holmberg S. Factors at resuscitation and outcome among patients suffering from out of hospital cardiac arrest in relation to age. Resuscitation. 2003;58(3):309–17.
Mackway-Jones K, Marsen J, Windle J, editors. Emergency Triage: Manchester Triage Group. Third ed. Chichester (West Sussex, UK): John Wiley & Sons, Ltd.; 2014.
American College of Surgeons. What are the surgical specialties? c1996-2014 https://www.facs.org/education/resources/medical-students/faq/specialties. Accessed 29 October 2014.
Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521–6.
Mower WR, Myers G, Nicklin EL, Kearin KT, Baraff LJ, Sachs C. Pulse oximetry as a fifth vital sign in emergency geriatric assessment. Acad Emerg Med. 1998;5(9):858–65.
Richtlijn ‘Pijnbehandeling bij traumapatiënten in de spoedzorgketen’ Utrecht: Het Kwaliteitsinstituut voor de Gezondheidszorg CBO. 2011. http://www.diliguide.nl/document/900/traumapatienten-pijnbehandeling-in-de-spoedzorgketen.html. Accessed 19 June 2015.
Fayyaz J, Khursheed M, Umer Mir M, Khan U. Pattern of emergency department visits by elderly patients: study from a tertiary care hospital, Karachi. BMC Geriatr. 2013;13:83.
Kennelly SP, Drumm B, Coughlan T, Collins R, O’Neill D, Romero-Ortuno R. Characteristics and outcomes of older persons attending the emergency department: a retrospective cohort study. Qjm. 2014;107(12):977-87.
van Walraven C, McAlister FA, Bakal JA, Hawken S, Donze J. External validation of the Hospital-patient One-year Mortality Risk (HOMR) model for predicting death within 1 year after hospital admission. Cmaj. 2015;187(10):725-33.
Barfod C, Lauritzen MM, Danker JK, Soletormos G, Forberg JL, Berlac PA, et al. Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department—a prospective cohort study. Scand J Trauma Resusc Emerg Med. 2012;20:28.
Lamantia MA, Stewart PW, Platts-Mills TF, Biese KJ, Forbach C, Zamora E, et al. Predictive value of initial triage vital signs for critically ill older adults. West J Emerg Med. 2013;14(5):453–60.
Scheetz LJ. Life-threatening injuries in older adults. AACN Adv Crit Care. 2011;22(2):128–39. quiz 40-1.
Carpenter CR, Shelton E, Fowler S, Suffoletto B, Platts-Mills TF, Rothman RE, et al. Risk factors and screening instruments to predict adverse outcomes for undifferentiated older emergency department patients: a systematic review and meta-analysis. Acad Emerg Med. 2015;22(1):1–21.
Acutely Presenting Older Patient: Leiden University Medical Center; http://www.apop.eu/index.cfm?setlang=en. Accessed on 28 June 2016.
We would like to thank Mary Ann Etty (Department of Information Technology – Medical Intelligence) for her help in extracting the data from the medical records.
The Institute for Evidence-Based Medicine in Old Age (IEMO) is funded by the Dutch Ministry of Health and Welfare and supported by ZonMW (project number 62700.3002).
Availability of data and materials
Necessary data were obtained from the computerised patient record system of our hospital and collected in a database for analysis. Data were processed anonymously. To ensure the privacy of our patients, data will not be made available for the public. However, we are happy to answer any questions that you may have concerning the data or other aspects of our study. Please contact the corresponding author J.A. Lucke for more information (Full postal address: PO Box 9600, 2300 RC Leiden, The Netherlands; Telephone: +31 (0)71 526 9111; Email address: email@example.com).
According to the definition given by the International Committee of Medical Journal Editors (ICMJE), the following people all qualify for authorship: SEH, JAL, NH, JdeG, AJF, CH, BdeG, AJMdeC, GJB, SPM. SEH and JAL contributed equally to this work and are considered the primary authors of this article. SEH conducted the statistical analyses, interpretation of data and was the primary writer of the article. JAL collected the data and assisted with the statistical analyses and interpretation of data. NH and JdeG were the primary reviewers of the statistical analyses and interpretation of data. AJF, CH, BdeG, AJMdeC and GJB contributed to the development of the study design and implementation. SPM conceived and coordinated the study and supervised the drafting process. All authors have participated in critically revising the manuscript for important intellectual content and approved the final manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. AJMdeC deceased on 17 January 2016.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
This study had a retrospective and observational design. The Medical Ethics Committee of the Leiden University Medical Center waived an obligation of approval as data were collected in the past as part of routine clinical care and additional actions from patients were not required (reference number P13.250).
Anton J. M. de Craen deceased.