Skip to main content

Derivation and validation of the ED-SAS score for very early prediction of mortality and morbidity with acute pancreatitis: a retrospective observational study

Abstract

Background

Existing scoring systems to predict mortality in acute pancreatitis may not be directly applicable to the emergency department (ED). The objective of this study was to derive and validate the ED-SAS, a simple scoring score using variables readily available in the ED to predict mortality in patients with acute pancreatitis.

Methods

This retrospective observational study was performed based on patient data collected from electronic health records across 2 independent health systems; 1 was used for the derivation cohort and the other for the validation cohort. Adult patients who were eligible presented to the ED, required hospital admission, and had a confirmed diagnosis of acute pancreatitis. Patients with chronic or recurrent episodes of pancreatitis were excluded. The primary outcome was 30-day mortality. Analyses tested and derived candidate variables to establish a prediction score, which was subsequently applied to the validation cohort to assess odds ratios for the primary and secondary outcomes.

Results

The derivation cohort included 599 patients, and the validation cohort 2011 patients. Thirty-day mortality was 4.2 and 3.9%, respectively. From the derivation cohort, 3 variables were established for use in the predictive scoring score: ≥2 systemic inflammatory response syndrome (SIRS) criteria, age > 60 years, and SpO2 < 96%. Summing the presence or absence of each variable yielded an ED-SAS score ranging from 0 to 3. In the validation cohort, the odds of 30-day mortality increased with each subsequent ED-SAS point: 4.4 (95% CI 1.8–10.8) for 1 point, 12.0 (95% CI 4.9–29.4) for 2 points, and 41.7 (95% CI 15.8–110.1) for 3 points (c-statistic = 0.77).

Conclusion

An ED-SAS score that incorporates SpO2, age, and SIRS measurements, all of which are available in the ED, provides a rapid method for predicting 30-day mortality in acute pancreatitis.

Peer Review reports

Background

Recently, the incidence of acute pancreatitis has risen worldwide, with results from a retrospective analysis in the US showing a 13% increase in hospital admissions over 10 years [1, 2]. The annual incidence of acute pancreatitis in the US ranged from 13 to 45 per 100,000 people [3]. While most patients who present with acute pancreatitis in the emergency department (ED) achieve positive outcomes, significant morbidity and mortality still exists [4]. On average, up to 25% will progress to severe acute pancreatitis, and pancreatic necrosis will develop in 20 to 30% of patients, both of which are associated with significant mortality [5, 6].

Multiple tools exist to aid clinicians in predicting morbidity and mortality risk associated with acute pancreatitis. These include well-known scoring systems such as Ranson’s criteria, the Acute Physiology and Chronic Health Evaluation (APACHE)-II score, and the Bedside Index for Severity in Acute Pancreatitis (BISAP) [7,8,9]. However, these scoring systems largely incorporate variables collected over 24 h or more of hospital admission and it may not be possible to obtain a final score until 48 h post-admission. Very early prognostic scores that apply to the ED triage and management of patients with acute pancreatitis are lacking.

The primary objective of this study was to establish a simple prognostic assessment of acute pancreatitis that clinicians can apply in the ED for early disease identification. In developing this objective, two key factors were observed that were repeatedly present early in severe acute pancreatitis. Systemic inflammatory response syndrome (SIRS) was a prominent indicator of disease severity [10,11,12]. Mild to moderate hypoxia was common and likely associated with early acute lung injury [13]. Hence, these findings were incorporated with other variables often available in the ED to derive and validate the ED-SAS, a simple scoring system that provides a prognostic assessment of mortality in patients admitted to the ED with acute pancreatitis.

Methods

Study design and setting

A retrospective analysis was performed at Vanderbilt University Medical Center (VUMC) in Nashville, TN, USA and Henry Ford Health System (HFHS) in Detroit, MI, USA. VUMC is a level 1 trauma center with approximately 125,000 annual ED visits. HFHS includes 9 EDs, 4 of which are free-standing, and 5 hospitals that experience approximately 460,000 annual ED encounters. An institutional review board at each site approved the study protocol with waiver of informed consent.

Selection of participants

Data from VUMC were used for the derivation cohort (January 2013 to September 2017). Data from HFHS were used for the validation cohort (January 2014 to January 2018). Eligible patients were selected in similar manner to VUMC’s Synthetic Derivative, a de-identified mirror-image of the system’s electronic health records, by Nashville Biosciences, a subsidiary of VUMC established to support translational research, and from HFHS’ electronic health records (EPIC, Verona, WI). Eligible patients were 18 years of age or older, had a diagnosis of acute pancreatitis (International Classification of Diseases [ICD], 9th Revision, code 577.0 or ICD, 10th Revision, code K85), and required hospital admission within 24 h of presentation in the ED. Only the first encounter with a patient over the study assessment period was included to ensure that no patient was represented more than once in the data set. Acute pancreatitis was confirmed through elevated serum lipase levels. We excluded patients with chronic pancreatitis, hepatic failure, or with a serum lipase <3x upper limit of the normal reference range. While prognostic assessment of patients discharged from the ED may be useful, we did not include these patients, given high rates of concomitant chronic pancreatitis and reduced opportunity to obtain outcomes data given the retrospective nature of the study.

Data collection and processing

Data collected for the derivation and validation cohorts included subject-level information on demographics; relevant comorbidities based on ICD codes; diagnoses of respiratory failure, sepsis, acute kidney injury (AKI), and other relevant sequelae as per ICD codes; mechanical ventilation determined by procedure codes for invasive mechanical ventilation; hospital procedure and visit records; mortality; and the first ED-recorded lab values and vital signs. Mechanical ventilation did not include non-invasive modes of ventilation.

Data analysis pulled electronic health data using standardized methods within each health system. For the validation cohort only, additional chart abstraction was performed to assess the accuracy of electronic data collection for the diagnosis of acute pancreatitis. Physician chart abstractors were blinded to the study hypothesis. All abstractors were trained before initiating chart abstraction and were provided with a data collection manual that included variable definitions and details. Within the validation cohort, physicians abstracted 10% of study charts to authenticate electronic data capture and to test agreement with electronic chart abstraction for the diagnosis of acute pancreatitis. We tested agreement by calculating the K coefficient for the diagnosis of acute pancreatitis. Complete data related to whether patients were alive or dead at the time of hospital discharge were available. For time points following hospital discharge, health system data in the validation cohort included statewide hospital mortality data.

Outcome measures

The primary outcome was 30-day mortality. Secondary outcomes included 180-day mortality; intensive care unit (ICU) admission; length of hospital stay; presence of sepsis, respiratory failure, or AKI; and need for dialysis or mechanical ventilation. To determine these latter outcomes, we used ICD codes documented during the patient’s index hospitalization for acute pancreatitis.

Primary data analysis

We selected predictor variables known to be associated with mortality from acute pancreatitis based on existing disease models that were present early in a patient’s ED evaluation [8, 10, 14, 15]. These predictor variables include age > 60 years, gender, self-reported race/ethnicity, the Charlson comorbidity index [16], the presence of ≥2 SIRS criteria (heart rate > 90/min, respiratory rate > 20/min, temperature > 38.0 °C or < 36.0 °C, and white blood cells > 12 × 109/L or < 4 × 109/L), the presence of SpO2 < 96%, and hematocrit > 44%. The SpO2 cut-off of < 96% was used as it includes the descending portion of the oxygen saturation curve. The selection of variables to include in our study’s scoring system was refined to ensure that only those that are readily available during routine ED evaluation were included.

For the derivation cohort, these variables were analyzed for their association with 30-day mortality via univariate analysis. We retained variables for the final model with p-values < 0.10. A complete case analysis was also performed. To reduce bias in parameter estimates, given the low event rate, multivariable logistic regression with Firth’s penalized likelihood estimate was utilized [17]. To determine the most parsimonious predictor model, the Akaike Information Criterion was assessed. In addition, model calibration using graphical assessment by loess smoothers [18] and model discrimination with area under the receiver operating characteristic curve (c statistic) were evaluated.

We performed logistic regression for the ED-SAS score categories with 0 as the reference to assess the odds of death and all secondary outcomes. We first performed this analysis within the derivation data set and applied this derived predictor score to the validation sample to assess its association with primary and secondary outcomes. No model recalibration was performed within the validation data. All analyses were performed with SAS 9.4 (Cary, NC). Analyses were performed between pairs of physician raters, and kappa coefficients (K) were calculated to determine interrater reliability.

Results

The derivation cohort included 599 patients, and the validation cohort included 2011 patients. Interrater reliability was excellent for chart review for the diagnosis of acute pancreatitis with a K coefficient of 0.94. In general, mean age, SpO2, and respiratory rate were similar between the two cohorts (Table 1). The 30-day mortality rate was similar across the two cohorts with 25 (4.2%) deaths in the derivation cohort and 78 (3.9%) in the validation cohort (Table 2). Respiratory failure rates were also similar between the two cohorts (4.2% vs 5.4%, respectively), but rates of sepsis were higher in the derivation cohort than in the validation cohort (13.4% vs. 6.6%, respectively; Table 2). In addition, length of hospital stay and ICU admissions were higher in the derivation cohort (Table 2).

Table 1 Baseline characteristics of patients with acute pancreatitis admitted to the ED a
Table 2 Primary and secondary outcomes upon review of electronic health record data

While four variables were associated with 30-day mortality via univariate analysis in the derivation cohort, only the presence of ≥2 SIRS criteria, an ED SpO2 saturation < 96%, and age > 60 years were significant upon multivariate analysis and subsequently retained in the final ED-SAS Score (Table 3). Based on this, the final three variables included in the model were SpO2 < 96%, age > 60 years, and ≥ 2 SIRS criteria (Table 4). Each variable was equally weighed and contributed 0 or 1 points to an overall score based on the presence or absence of these three variables, resulting in an ED-SAS range of 0 to 3 points. The probability of 30-day mortality increased incrementally with higher ED-SAS scores for both cohorts (Fig. 1). In addition, increases in the odds ratio (OR) for 30-day mortality, 180-day mortality, AKI, respiratory failure, sepsis, and ICU admission were directly associated with increased ED-SAS score in the validation cohort (Table 5).

Table 3 Univariate analysis of acute pancreatitis predictor variables
Table 4 ED-SAS Score. A scoring system that utilizes variables readily available in the ED
Fig. 1
figure1

Probability of 30-day mortality based on ED-SAS scores. After application of the ED-SAS score, the probability of 30-day mortality increased incrementally with higher ED-SAS scores for both cohorts

Table 5 Odds of primary and secondary outcomes in validation cohort based on ED-SAS score

Discussion

The incidence of acute pancreatitis has been increasing worldwide. While multiple risk stratification tools exist, they come with limitations [1, 2, 7,8,9]. The 3 most commonly used risk stratification tools in the ED include the BISAP score, the APACHE II Scoring System, and Ranson’s criteria [1]. Imaging-based scores also exist, which grade computed tomography findings of pancreatitis, but these may not be performed routinely and may underestimate severity in patients who present early in the disease course [14, 19,20,21]. While these scoring systems are regularly used to predict mortality, they were not designed specifically for utility in the ED. [1, 7,8,9]

The ability to identify patients with acute pancreatitis who are at high-risk for mortality using a stratification tool focused on variables that are collected early in the ED could potentially lead to rapid identification of patients with poor prognoses while at the same time, identifying those patients who do not require agreement management and for whom home-based care may be more appropriate. In this study across two health systems and multiple EDs, we derived and validated the ED-SAS, a simple prognostic tool that focuses on 3 variables, all of which can be collected in the ED and are readily available in the early evaluation of a patient with acute pancreatitis: age, pulse oximetry reading on room air (SpO2 saturation < 96%), and the presence of SIRS criteria. The score was named ED-SAS as SpO2, Age, and SIRS are all variables assessed in the ED. Results from this analysis suggest increased risk of 30-day mortality with increasing ED-SAS score; ED-SAS scoring could aid in ED prognostication and influence triaging decisions.

The inclusion of advanced age, SIRS criteria, and low pulse oximetry are consistent with the existing body of literature. Age has been validated in existing inpatient prediction scores. SIRS is known to impact the clinical course and outcome in acute pancreatitis [10, 22, 23]. SIRS is also an early predictor of severe acute pancreatitis, with studies demonstrating that the severity of acute pancreatitis is greater among patients with SIRS on day 1 [24]. While less recognized than SIRS, moderate to severe arterial hypoxia is common in patients with acute pancreatitis and is related to the respiratory injury often associated with the inflammatory response seen with acute pancreatitis [10, 13]. Among patients that die early from acute pancreatitis, respiratory injury is often a prominent finding and is present in approximately 60% of all elderly patients who pass from acute pancreatitis within the first week [15, 25]. Although prior studies used arterial blood gas measurements to assess hypoxia, we relied on pulse oximetry in this study, given the infrequent use of arterial blood gas in the ED for these patients [13].

This retrospective, observational, chart review study, which relied on commonly documented predictor variables, comes with some limitations. While standardized methods for data collection were used, there remains a potential for bias, and there may be important predictors of morbidity and mortality that were not considered. Further, selection bias may exist, as patients who were discharged directly from the ED were excluded from the analysis; this limits the analysis to those with more severe cases of acute pancreatitis. We did not perform a direct comparison between the ED-SAS score and other common scoring systems or ED predictors of mortality such as the quick Sepsis Related Organ Failure Assessment score, which may have utility in evaluating acute pancreatitis, as it does in sepsis [26]. In addition, the design for assessing mortality may underestimate the true rates, given that subjects were not directly contacted, as electronic health record data were used to determine 30- and 180-day mortality. While it is possible that some deaths were not accounted for, the potential of systematic bias related to these missing data is limited by inclusion of statewide hospital death data within the validation cohort. Future research in this area should also account for additional biomarkers and clinical findings, including lactate dehydrogenase, aspartate aminotransferase, pleural effusion, and mental status. Due to the retrospective nature of data collection, we were surprised to find significant missing data for some of these variables. The ED-SAS model provides a simple assessment of high-risk patients with acute pancreatitis early in their ED course, but it was not designed to assess outcomes, the intensity of inpatient services that patients require to adequately determine eligibility for ED discharge, or outpatient management. However, as this information would be of great interest to emergency physicians, additional studies to assess this would be of value. Finally, we did not compare the ED-SAS score to clinical gestalt and are therefore unable to assess whether it provides added value to the clinical judgement of an experienced physician.

Future studies of the ED-SAS should include analysis of all patients admitted to the ED for acute pancreatitis, regardless of severity and admission to hospital. In doing so, it may be possible to utilize this predictive scoring system in a broader range of patients with acute pancreatitis, thereby mitigating selection bias.

Conclusions

In this retrospective, observational analysis of patients hospitalized for acute pancreatitis, the ED-SAS was derived and validated as a simple predictive tool that could be used to estimate the 30-day risk of mortality using patient characteristics commonly available in the ED. This tool has the potential to aid clinicians in prognosticating patients with acute pancreatitis.

Availability of data and materials

Data are available on reasonable request from the corresponding author.

Abbreviations

AKI:

Acute kidney injury

APACHE-II:

Acute Physiology and Chronic Health Evaluation II score

BISAP:

Bedside Index for Severity in Acute Pancreatitis

ED:

Emergency department

ED-SAS:

Emergency department SpO2, age, and SIRS

HFHS:

Henry Ford Hospital System

ICD:

International Classification of Diseases

ICU:

Intensive care unit

SIRS:

Systemic inflammatory response syndrome

VUMC:

Vanderbilt University Medical Center

References

  1. 1.

    Waller A, Long B, Koyfman A, Gottlieb M. Acute pancreatitis: updates for emergency clinicians. J Emerg Med. 2018;55:769–79.

    Article  Google Scholar 

  2. 2.

    Krishna SG, Kamboj AK, Hart PA, Hinton A, Conwell DL. The changing epidemiology of acute pancreatitis hospitalizations. A decade of trends and the impact of chronic pancreatitis. Pancreas. 2017;46(4):482–8.

    Article  Google Scholar 

  3. 3.

    Yadav D, Lowenfels AB. The epidemiology of pancreatitis and pancreatic cancer. Gastroenterol. 2013;144:1252–61.

    Article  Google Scholar 

  4. 4.

    Eachempati SR, Hydo LJ, Barie PS. Severity scoring for prognostication in patients with severe acute pancreatitis: comparative analysis of the Ranson score and the APACHE III score. Arch Surg. 2002;137:730–6.

    Article  Google Scholar 

  5. 5.

    Barauskas G, Ignatavičius P, Vitkauskienė A, Pundzius J, Dambrauskas Ž. Impact of etiology on course and outcomes of severe acute pancreatitis. Medicina (Kaunas). 2015;51(3):167–72.

    Article  Google Scholar 

  6. 6.

    Singh VK, Moran RA, Afghani E. de-Madaria E. Treating acute pancreatitis: what’s new? Exp Rev Gastroenterol Hepatol. 2015;9(7):901–11.

  7. 7.

    Bradley EL. A clinically based classification system for acute pancreatitis. Summary of the international symposium on acute pancreatitis, Atlanta, Ga, September 11 through 13, 1992. Arch Surg. 1993;128:586–90.

    Article  Google Scholar 

  8. 8.

    Larvin M, McMahon MJ. APACHE-II score for assessment and monitoring of acute pancreatitis. Lancet. 1989;2:201–5.

    CAS  Article  Google Scholar 

  9. 9.

    Wu BU, Johannes RS, Sun X, Tabak Y, Conwell DL, Banks PA. The early prediction of mortality in acute pancreatitis: a large population-based study. Gut. 2008;57:1698–703.

    CAS  Article  Google Scholar 

  10. 10.

    Shields CJ, Winter DC, Redmond HP. Lung injury in acute pancreatitis: mechanisms, prevention, and therapy. Curr Opin Crit Care. 2002;8:158–63.

    Article  Google Scholar 

  11. 11.

    Mofidi R, Duff MD, Wigmore SJ, Madhavan KK, Garden OJ, Parks RW. Association between early systemic inflammatory response, severity of multiorgan dysfunction and death in acute pancreatitis. The Br J Surg. 2006;93:738–44.

    CAS  Article  Google Scholar 

  12. 12.

    Mofidi R, Lee AC, Madhavan KK, Garden OJ, Parks RW. Prognostic factors in patients undergoing surgery for severe necrotizing pancreatitis. World J Surg. 2007;31:2002–7.

    CAS  Article  Google Scholar 

  13. 13.

    Imrie CW, Ferguson JC, Murphy D, Blumgart LH. Arterial hypoxia in acute pancreatitis. The Br J Surg. 1977;64:185–8.

    CAS  Article  Google Scholar 

  14. 14.

    Papachristou GI, Muddana V, Yadav D, et al. Comparison of BISAP, Ranson's, APACHE-II, and CTSI scores in predicting organ failure, complications, and mortality in acute pancreatitis. Am J Gastrolenterol. 2010;105:435–41.

    Article  Google Scholar 

  15. 15.

    Andersson R, Andren-Sandberg A. Fatal acute pancreatitis. Characteristics of patients never reaching hospital. Pancreatol. 2003;3:64–6.

    Article  Google Scholar 

  16. 16.

    Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83.

    CAS  Article  Google Scholar 

  17. 17.

    Heinze G, Puhr R. Bias-reduced and separation-proof conditional logistic regression with small or sparse data sets. Stat Med. 2010;29:770–7.

    Article  Google Scholar 

  18. 18.

    Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33:517–35.

    Article  Google Scholar 

  19. 19.

    Bollen TL, Singh VK, Maurer R, et al. A comparative evaluation of radiologic and clinical scoring systems in the early prediction of severity in acute pancreatitis. Am J Gastrolenterol. 2012;107:612–9.

    Article  Google Scholar 

  20. 20.

    Kuo DC, Rider AC, Estrada P, Kim D, Pillow MT. Acute pancreatitis: what’s the score? J Emerg Med. 2015;48:762–70.

    Article  Google Scholar 

  21. 21.

    Sahu B, Abbey P, Anand R, Kumar A, Tomer S, Malik E. Severity assessment of acute pancreatitis using CT severity index and modified CT severity index: correlation with clinical outcomes and severity grading as per the revised Atlanta classification. Indian J Radio Imaging. 2017;27:152–60.

    Article  Google Scholar 

  22. 22.

    McKay CJ, Gallagher G, Brooks B, Imrie CW, Baxter JN. Increased monocyte cytokine production in association with systemic complications in acute pancreatitis. Br J Surg. 1996;83:919–23.

    CAS  Article  Google Scholar 

  23. 23.

    Lundberg AH, Granger DN, Russell J, et al. Quantitative measurement of P- and E-selectin adhesion molecules in acute pancreatitis: correlation with distant organ injury. Ann Surg. 2000;231:213–22.

    CAS  Article  Google Scholar 

  24. 24.

    Singh VK, Wu BU, Bollen TL, et al. Early systemic inflammatory response syndrome is associated with severe acute pancreatitis. Clin Gastroenterol Hepatol. 2009;7:1247–51.

    Article  Google Scholar 

  25. 25.

    Jacobs ML, Daggett WM, Civette JM, et al. Acute pancreatitis: analysis of factors influencing survival. Ann Surg. 1977;185:43–51.

    CAS  Article  Google Scholar 

  26. 26.

    Hallac A, Puri N, Applebury D, et al. The value of quick sepsis-related organ failure assessment scores in patients with acute pancreatitis who present to emergency departments: a three-year cohort study. Gastroenterol Res. 2019;12:67–71.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

No funding was provided for this study. All authors had full access to all data in the study and provided input on the analyses before and during writing of the report. The authors take responsibility for the data integrity and the accuracy of the data analysis. JM and RS primarily composed the manuscript; CalciMedica (La Jolla, CA, USA) commissioned medical writing assistance from Sarah Odeh (San Francisco, CA, USA) to support subsequent drafts under the direction of all authors. All authors reviewed and revised each draft and approved the final submitted version. The corresponding author had final responsibility for the decision to submit for publication.

Author information

Affiliations

Authors

Contributions

JM, YW, and SH conceived and designed the study. JM, YW, GA, HGA, NF, AS, VN, AA, and JD worked on data acquisition. RS and JM assisted with the statistical analysis and interpretation. JM, YW, RS, GM, SB, GA, JD, HGA, NF, AS, VN, AA, JD, SH, and LE participated in synthesis of the data and manuscript review. JM, YW, RS, GM, SB, GA, JD, HGA, NF, AS, VN, AA, JD, SH, and LE approved the final manuscript. JM and RS primarily composed the manuscript and take responsibility for it as a whole.

Corresponding author

Correspondence to Joseph Miller.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

JM, YW, RS, GM, SB, GA, JD, HGA, NF, AS, VN, AA, JD, and LE report no conflicts of interest related to this work. SH serves as chief medical officer for CalciMedica (La Jolla, CA, USA).

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Miller, J., Wu, Y., Safa, R. et al. Derivation and validation of the ED-SAS score for very early prediction of mortality and morbidity with acute pancreatitis: a retrospective observational study. BMC Emerg Med 21, 16 (2021). https://doi.org/10.1186/s12873-021-00410-w

Download citation

Keywords

  • Acute pancreatitis
  • Predictive score
  • Validation study
  • Emergency department
\