Healthcare risk stratification model for emergency departments based on drugs, income and comorbidities: the DICER-score

Abstract

Background

During the last decade, the progressive increase in age and associated chronic comorbidities and polypharmacy. However, assessments of the risk of emergency department (ED) revisiting published to date often neglect patients’ pharmacotherapy plans, thus overseeing the Drug-related problems (DRP) risks associated with the therapy burden. The aim of this study is to develop a predictive model for ED revisit, hospital admission, and mortality based on patient’s characteristics and pharmacotherapy.

Methods

Retrospective cohort study including adult patients visited in the ED (triage 1, 2, or 3) of multiple hospitals in Catalonia (Spain) during 2019. The primary endpoint was a composite of ED visits, hospital admission, or mortality 30 days after ED discharge. The study population was randomly split into a model development (60%) and validation (40%) datasets. The model included age, sex, income level, comorbidity burden, measured with the Adjusted Morbidity Groups (GMA), and number of medications. Forty-four medication groups, associated with medication-related health problems, were assessed using ATC codes. To assess the performance of the different variables, logistic regression was used to build multivariate models for ED revisits. The models were created using a “stepwise-forward” approach based on the Bayesian Information Criterion (BIC). Area under the curve of the receiving operating characteristics (AUCROC) curve for the primary endpoint was calculated.

Results

851.649 patients were included; 134.560 (15.8%) revisited the ED within 30 days from discharge, 15.2% were hospitalized and 9.1% died within 30 days from discharge. Four factors (sex, age, GMA, and income level) and 30 ATC groups were identified as risk factors and combined into a final score. The model showed an AUCROC values of 0.720 (95%CI:0.718–0.721) in the development cohort and 0.719 (95%CI.0.717–0.721) in the validation cohort. Three risk categories were generated, with the following scores and estimated risks: low risk: 18.3%; intermediate risk: 40.0%; and high risk: 62.6%.

Conclusion

The DICER score allows identifying patients at high risk for ED revisit within 30 days based on sociodemographic, clinical, and pharmacotherapeutic characteristics, being a valuable tool to prioritize interventions on discharge.

Highlights

Risk scores are often used to predict the clinical outcomes of patients in many healthcare settings.

To the date, no prediction model of emergency department (ED) visits based on patients’ pharmacotherapy, income level, and comorbidities have been developed.

We have designed an ED risk score combined four risk factors (sex, age, comorbidity score and income level) and 30 drug categories to identify those patients at high risk of health-care visit.

Introduction

During the last decades, the progressive increase in age and associated chronic comorbidities and polypharmacy in the population has led to a growing demand for healthcare resources, particularly emergency services. Secondary effects of population ageing include the onset of drug-related problems (DRP), which can be due to failure of pharmacotherapy because of ineffectiveness, safety issues, or the need for additional medicines. Approximately 5–10% of the hospitalisations and 10–20% of Emergency Department (ED) visits are due to DRPs, most of them considered avoidable [1,2,3].

The rise in emergency room visits and unplanned hospital admissions represents one of the primary challenges for healthcare systems globally, particularly in the elderly population. Due to the elevated occurrence of emergencies among older individuals [4], there is a likelihood that older adults will contribute to a growing proportion of visits to ED in the future. Several studies have already highlighted a rising demand for ED services. For instance, in England, the number of ED visits by individuals aged 65 years or older surged by 46% between 2001 and 2012 [5]. In the USA, the annual visit rate among those aged 65 years or older was 511/1000 persons and increased with age [6]. Elderly patients exhibit a higher hospitalization rate, necessitate more resources, and face an elevated risk of adverse outcomes [5]. This increase aligns with the mounting healthcare expenses associated with patient´s progressive and irreversible decline following hospital admissions [4]. Hence, prioritizing the prevention of hospital admissions potentially linked to polypharmacy should be considered a priority for health administrations. Over the past decade, numerous research studies have shown that multidisciplinary initiatives targeting primary prevention of DRP effectively lower the likelihood of ED visits and hospitalizations [7, 8]. However, assessments of the risk of ED revisiting published to date often neglect patients’ pharmacotherapy plans, thus overseeing the DRP risks associated with the therapy burden. Hence, although some algorithms have been developed to predicting (or stratify the risk of) readmissions and health outcomes in patients visited at the ED, none of them combines clinical characteristics (e.g., summary measures of the comorbidity burden), social characteristics, and pharmacotherapy. Considering the high social demand for emergency care, prioritization strategies must be developed for multidisciplinary teams to prevent EDs revisits and hospital readmissions. Several index of frailty status and comorbidity are becoming essential tools in electronic medical records systems, continually expanding in comprehensiveness each year. The availability of tools for risk stratification based on chronic pharmacotherapy will enable the identification of high-risk patients as priorities for optimizing their treatment, offering significant potential to decrease potentially avoidable hospital admissions.

This study aims to create a comprehensive scoring system based on comorbidity burden, social vulnerability, and pharmacotherapy tailored for automated calculation by electronic healthcare systems, facilitating the assessment of short-term risks associated with ED visits and hospital admissions.

Methods

Study design, population, and data sources

We designed an observational retrospective population-based study in Catalonia, a North-East region in Spain, with a population of 7.7 million people. The Catalan population receives comprehensive healthcare services from the regional Catalan Health Service, utilizing a network comprising 64 general hospitals, 27 psychiatry hospitals, 375 primary care centers, 91 skilled nursing facilities for intermediate care, and 130 outpatient mental health facilities. Detailed sociodemographic and clinical information, including diagnoses, annual income, and healthcare utilization, has been gathered from the Catalan Health Surveillance System (CHSS) since 2011, encompassing the entire population of Catalonia. This record, used in previous publications in other areas [9,10,11], gathers data recorded in multiple settings, linked through a unique identification number used for public insurance purposes. These environments encompass primary care, acute care hospitals, intermediate care hospitals, mental health centers, outpatient clinics, and emergency services. Additionally, this documentation compiles data pertaining to prescriptions, pharmacy costs, and invoices, encompassing non-urgent medical transportation, outpatient rehabilitation, home oxygen therapy, and dialysis. No data about private healthcare could be collected because these centers use different codes for patient identification. However, owing to the co-payment system of medicines established in Spain, chronic prescriptions outside the public health system are unusual. Medications are introduced in the registry using the codes of the Anatomical Therapeutic Chemical (ATC) Classification System.

The aforementioned information originates from the interactions between patients and any public healthcare entity or service and is regularly transferred from the electronic health records of healthcare providers to the Catalan Health Service (the public insurer in Catalonia), which uses it for billing purposes, among others.

The study included data from all visits to life-threatening hospital emergencies (triage 1, 2, or 3) between January 1 and December 31, 2019. The following cases were excluded from the analysis: emergencies in children under 18 years of age; emergencies with triage 4 or 5; emergencies without completion of care: transfer or referral to another health center, and evasion or voluntary or administrative discharge; deaths in the emergency department; v) emergencies without patient identifier; emergencies with the main diagnosis of complications of pregnancy, childbirth and postpartum (system 11); and emergencies with the main diagnosis of injuries (CCS 00225, 00240 and 00244) [12].

Once selected the study cohort, the full dataset was randomly split into a model development cohort (60%) and validation cohort (40%).

All data were handled according to the General Data Protection Regulation 2016/679 on data protection and privacy for all individuals within the European Union and the local regulatory framework regarding data protection [13]. Data from different health administrative databases were linked and de-identified by a team not involved in the study analysis; study investigators only had access to a fully anonymized database. The retrospective use of healthcare data was approved by the Hospital Santa Creu i Sant Pau Ethics Committee (Nº: (Nº: IIBSP-COD-2022-40), which waived the need for obtaining informed consent for data utilization. Results are presented according to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. STROBE and RECORD guidelines for observational studies and studies using routinely collected health data were also considered.

Variables

The sociodemographic variables considered in this study were age, sex, and income level, classified as high (annual income > 100,000 €), intermediate (18,000–100,000 €), low (< 18,000 €), and very low (receiving welfare support from the government) [14, 15]. Comorbidity burden was calculated based on the Adjusted Morbidity Groups (GMA) [16,17,18]. The GMA tool considers all chronic diagnoses present at a given time and acute diagnoses reported during the study period. The GMA index score is computed by adding the weights of each diagnosis group, being used to generate mutually-exclusive risk groups based on the index distribution in the general population as follows: baseline risk (healthy stage; GMA index up to the 50th percentile of the total population), low risk (GMA index between the 50th and 80th percentiles), moderate risk (GMA risk between the 80th and 95th percentiles), high risk (GMA index between the 95th and 98th percentiles), and very-high risk (GMA index above the 98th percentile). Forty-four medication groups associated with medication-related health problems were assessed using ATC codes [19] dispensed in the two months preceding the index visit to the emergency room. The number of different drugs (5-digit ATC) dispensed during the previous two months is also calculated. This last measure was analyzed as a continuous variable.

Endpoints

Study outcomes included all-cause revisiting to the ED, hospital admission, and death within the 30 days following discharge of the index episode. The primary outcome, used as a response variable for model development and validation, was a composite of achieving any of the three outcomes.

Statistics

Categorical variables were described as frequencies and percentages, and quantitative variables as the mean and standard deviation (SD). Categorical variables were compared using Pearson’s Chi-squared test with Yates’ continuity correction. The statistical significance threshold was set at a bilateral alpha value of 0.05.

The dataset’s composition was driven by events; all included factors were either clinical conditions or variables essential for healthcare system registration. Consequently, the variables analyzed contained no missing data, and no efforts were made to fill in any gaps through data imputation.

To assess the performance of the different variables, we used generalized linear models (logistic regression) to build multivariate models for ED revisits, with the contribution of each factor expressed as an odds ratio (OR) and its 95% confidence interval (CI). The models were created using a “stepwise-forward” approach based on the Bayesian Information Criterion (BIC), in which a naïve model is sequentially complemented with the most relevant variables, eventually leading to the main effects model [20].

In both the development and validation datasets, we evaluated the performance of each model using various statistical measures. For our primary analysis, we opted for the area under the curve of the receiving operating characteristics (AUCROC) curve. This curve assesses the model’s discriminatory ability as the threshold varies, ranging from 0.5 (indicating low discrimination capacity) to 1 (indicating high discrimination capacity). AUC-ROCs of < 0.70, 0.70–0.89, and ≥ 0.90 were considered poor, adequate, and excellent, respectively. Furthermore, we conducted secondary analyses employing the Bayesian Information Criterion (BIC) and the area under the precision-recall (AUC-PR) curve. The BIC quantifies in-sample prediction error, considering the trade-off between achieving a good fit (avoiding overfitting) and maintaining model simplicity (preventing underfitting). The range of values for the BIC is contingent on the study sample, with lower values denoting superior performance and higher values indicating poorer performance. On the other hand, the AUC-PR curve delineates the balance between precision (minimizing false-positive rates) and recall (minimizing false-negative rates). Unlike the ROC curve, the AUC-PR curve provides a less biased assessment, especially for outcomes with low frequency [21]. All analyses were conducted using the R statistical package, version 4.0.3 [22].

Ethics approval

The study protocol was approved by the Independent Institutional Review Board of the Hospital Santa Creu I Sant Pau Ethics Committee, which waived the need for written informed consent (protocol code IIBSP-PRM-2021-39 (PI21/01818).

The results obtained are presented in accordance with the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. This study adhered to the Declaration of Helsinki guidelines.

Results

Study cohort

The study group included 834,679 cases with a mean age of 49.1 (SD: 18.2 years). The main characteristics of the study cohort are shown in Table 1. Overall, 271,270 (32.5%) individuals met any of the primary endpoint criteria: 131,879 (15.8%) had ED revisits, 126,871 (15.2%) hospital readmissions, and 75,956 (9.1%) died within the 30 days following discharge from ED. The frequency of the composite endpoint was higher among men, increased with the number of drugs prescribed, and decreased with the income level.

Table 1 30-day composite endpoint of the six outcome variables considered in the analysis

Model development and validation

Table 2 summarizes the contribution of each parameter (sex, age, income level, GMA index, and drug dispensation, classified by ATC groups) to explaining the composite endpoint. The DICER-Score value is obtained after incorporating each of the beta coefficients into the logistic regression model [Risk for ED visit: 1/(1-exp(Constant+$\sum xbeta$)].The assessment of the comorbidity burden using the GMA as a summary measure accounted for the highest effect size on the composite endpoint. The final model (DICER-Score) reached an AUROC of 0.720 (95%CI 0.718–0.721) in the development sample and 0.719 (95%CI 0.718–0.721) in the validation sample (Fig. 1), consistently with an adequate convergent validity. Four risk categories were established: low risk (Risk for ED visit cut off-value: 16.4%), moderate risk (Risk for ED visit: 33.1%), high risk (Risk for ED visit: 54.4%) and very high risk (Risk for ED visit: 69.1%). Figure 2 shows the agreement between the development and validation cohort regarding the frequency of the composite endpoint of ED revisit, hospital re-admission or death within the 30 days following ED discharge.

Table 2 Relative Risk of 30-day Emergency Department visits for the variables included in the multivariable analysis

Fig. 1

Performance of the DICER score for predicting the composite endpoint of ED revisit, hospital admission or death within the 30 days following discharge from the index episode. (A): development dataset. (B): validation dataset. AUCROC: area under the curve of the receiving operating characteristics. AUCPR: area under the precision-recall curve

Fig. 2

Risk of 30-day composite variable (ED revisit, hospital admission or death) for the predictive and validation population according to the four established risk groups

Discussion

Our results show that a comprehensive view of the patient context, including demographic characteristics, comorbidity burden, socioeconomic status, and drug dispensation has robust predictive capacity of the risk for ED revisit and hospital admission at ED discharge. Although the comorbidity burden has a strong effect in our model, the number and type of drugs dispensed significantly contributed to explaining this risk. This result indicates that drug consumption, which is typically disregarded in risk models at the ED, shall be considered when appraising the likelihood of a patient to revisit or be admitted to a hospital.

Risk scores are often used to predict the clinical outcomes of patients in many healthcare settings [23, 24]. In the context of pharmacotherapy, several scales have been developed for predicting adverse effects in hospitalized patients [25, 26]. In many of them, the risk factors identified are similar to those in our score, including the patient´s age, comorbidity scales, or anticoagulation. In the field of ED, Hao et al. [8]. developed a model for predicting ED revisits based on multiple variables, obtaining a prospective prediction value of 0.704, similar than the obtained in our model. Other studies, which have formulated risk scales to forecast emergency visits but concentrated on specific pathologies such as cancer [27] or decompensated heart failure [28], have yielded predictive values comparable to those observed in our investigation, falling within the range of 0.70 to 0.80. However, to date, no specific revisit prediction scales based on patients’ pharmacotherapy, income level, and comorbidities have been developed, being this point the main strength of this score. As previous risk scores published [16,17,18], the model’s incorporation of a multitude of variables renders this score valuable for its automated calculation by electronic medical information systems. Our score highlights that individuals with high chronic disease (and consequent therapeutic) burden are particularly prone to revisiting the ED, in line with previous studies [4, 29]. The results of our score are intended to be helpful for prioritizing patients who are candidates for interventions to improve their pharmacotherapy plan in a highly frequented area with a large number of annual visits, such as the ED. Our analysis serves as an illustration of how proficient information systems can facilitate the creation of “learning healthcare systems.” These systems strive to enhance the quality of care by iteratively embracing and evaluating evidence-based solutions.

Osmotic laxatives, β-lactamic antibiotics, and folic acid analogs were the three therapeutic groups associated with a higher risk of ED revisits or hospitalizations. Constipation is a well-known cause of ED visits, particularly in older patients, and anticholinergic burden associated with chronic pharmacotherapy has been associated with ED visits [30]. Although previous studies have found an association between anticholinergic burden and hospital care in geriatric patients, results of early treatment optimization after hospital discharge have not yet been reported. Our data indicate the need for the optimization of long-term treatments in this group of patients. Additionally, ED visits associated with antibiotic failure have also been identified as an important issue in the ED [31]. Finally, it is well known that patients with folic acid analogs (mainly methotrexate) have a high risk of hospitalization due to infections or other drug adverse events [32, 33]. Our results indicate that this group of patients must be under special surveillance and followed up after discharge from these units. Other factors such as advanced age, polypharmacy, or low socioeconomic status are common factors associated with the risk of ED visits, adverse effects, and hospital admissions [34, 35].

The development of this score represents a noteworthy advancement in identifying and optimizing patients with polypharmacy and a high risk of re-consultation, with a distinct applicability in both primary care and among patients discharged from emergency services, particularly those consulting for drug-related issues. The complexity of pharmacotherapy and its adverse effects in frail patients is widely recognized as being associated with a higher risk of emergency room visits and hospital admissions. The implementation of this risk scale will enable the creation of specific programs aimed at optimizing pharmacotherapy and providing close monitoring based on the risk of hospital consultation. Multiple interventions have been performed and analyzed to reduce hospital admissions and ED visits secondary to DRPs. Various studies have observed that interventions such as patient education, medication review on discharge, or telephone consultations can reduce admissions to the ED [29, 36]. Ravn-Nielsen et al. and Juanes A et al. demonstrated that interventions based on treatment revision at discharge and over a telephone call could reduce the risk of a revisit [4, 37]. In this regard, the DICER-score provides clinicians with a helpful tool to increase the efficiency of these interventions and optimize resources in the ED to reduce revisit rates.

Limitations

This study has some limitations that should be considered. First, it was conducted in a specific population within a specific healthcare system. External validation may be needed to extrapolate the results obtained in areas with a different heathcare system. The results of the DICER score include different variables and multiple ATC groups. That is why direct calculation by electronic medical records is desirable for its efficient application. Second, despite the novelty of considering the socioeconomic status in our model, this classification is based on annual income solely and lacks information regarding social support and other vulnerability sources. Nevertheless, the characteristics of the source dataset in terms of the quality and extension should be considered as a strength that provides robustness to the predictive model. It is worth mentioning that, while the DICER score can predict patients at a high risk of ED revisiting or hospitalization, further studies must be conducted to establish the most appropriate strategies to reduce revisits in this population. The AUROC obtained in our model is lower than 0.8, that has been extensively considered as a very good predictor cut-off value [38]. The individual risk of ED visit is highly variable, in which culture and sociodemographic variables not included in medical records are involved. However, the AUCROC value obtained is higher than other commonly used in clinical practice [23, 24]. Finally, the DICER score, as well as other multimorbidity indices, may have limited capacity to predict outcomes in specific populations like children or patients with mental disorders.

Future investigations should undertake the validation of this scale across diverse populations, considering the characteristics of each population, including variations in comorbidities and the pharmacological treatment approaches for chronic conditions. The utilization and adjustment of this scale in conditions characterized by a high incidence of emergency room visits, and the integration of novel drugs into their management (e.g., chronic heart failure), warrant careful evaluation.

Additionally, it is crucial to compare the outcomes derived from this scale with established clinical practice metrics such as the anticholinergic load or fall risk screening scores. Furthermore, there is a need for fresh research to explore whether the modification and simplification of chronic treatment, as reflected in the risk scale reduction, translate into a tangible decrease in emergency room visits. This would contribute valuable insights into the practical implications of using the scale in optimizing patient outcomes and healthcare resource utilization.

Conclusion

This study underscores the need for using comprehensive approaches to the assessment of risk of undesired outcomes in individuals visited at the ED. The proposed score shows that not only demographic and clinical characteristics but also socioeconomic status and pharmacotherapy account for the risk of ED revisit or hospital admission within the 30 days following discharge. Importantly, the contribution of pharmacotherapy varies across ATC groups. The DICER score may help clinicians and hospital managers identifying patients at higher risk of ED revisiting and hospital admissions, with potential implications not only in the quality of care but also in resource allocation and planning.