Skip to main content

Diagnostic agreement between emergency medical service and emergency department physicians, a prospective multicentre study

Abstract

Introduction

Early and adequate preliminary diagnosis reduce emergency department (ED) and hospital stay and may reduce mortality. Several studies demonstrated adequate preliminary diagnosis as stated by emergency medical services (EMS) ranging between 61 and 77%. Dutch EMS are highly trained, but performance of stating adequate preliminary diagnosis remains unknown.

Methods

This prospective observational study included 781 patients (> 18years), who arrived in the emergency department (ED) by ambulance in two academic hospitals. For each patient, the diagnosis as stated by EMS and the ED physician was obtained and compared. Diagnosis was categorized based on the International Classification of Diseases, 11th Revision.

Results

The overall diagnostic agreement was 79% [95%-CI: 76–82%]. Agreement was high for traumatic injuries (94%), neurological emergencies (90%), infectious diseases (84%), cardiovascular (78%), moderate for mental and drug related (71%), gastrointestinal (70%), and low for endocrine and metabolic (50%), and acute internal emergencies (41%). There is no correlation between 28-day mortality, the need for ICU admission or the need for hospital admission with an adequate preliminary diagnosis.

Conclusion

In the Netherlands, the extent of agreement between EMS diagnosis and ED discharge diagnosis varies between categories. Accuracy is high in diseases with specific observations, e.g., neurological failure, detectable injuries, and electrocardiographic abnormalities. Further studies should use these findings to improve patient outcome.

Peer Review reports

Introduction

The emergency medical services (EMS) and emergency department (ED) staff encounter diverse disease presentations. Admissions to hospital are primarily through the ED and therefore reflects an important entry point into the acute care chain with the hospital environment. It is known that the correct (suspected) diagnosis by EMS personnel can speed up the diagnostic and therapeutic processes at the ED. [1] On the other hand, an incorrect preliminary diagnosis from EMS can have negative effects on patient outcomes, leading to increased frequency of intensive care unit (ICU) admissions, prolonged ED and hospital stays, as well as higher mortality rates. [2,3,4] Therefore, the preliminary diagnosis by EMS should be correct.

Previous studies examined the accuracy of preliminary diagnoses made by EMS compared to the discharge diagnoses in the ED. These studies have reported overall accuracies ranging from 61 to 77%. [5, 6] However, the accuracy varied depending on the type of disease. [5,6,7,8] For instance, Koivulahti et al. reported rates of agreement in categories such as mental health and intoxication (86%), cerebral strokes (81%), respiratory emergencies (58%), and infectious diseases (31%). [5] Similar levels of agreement have been documented for respiratory emergencies in other studies. [7] However, some categories, such as cerebral strokes and mental health problems, have variable outcomes in terms of diagnostic accuracy in comparable research. [6]

Focused training has been shown to improve the accuracy of EMS in stating adequate diagnosis. [5] In the Netherlands, EMS nurses receive extensive training and are probably among the most highly trained in the world. Given the pivotal role of preliminary diagnosis in patient outcomes, it is important to ensure its validity within the Dutch context. Therefore, our study aims to investigate the accuracy of EMS (preliminary) compared to the ED diagnosis with two large Dutch university hospitals. The outcomes were also correlated to relevant clinical outcomes.

Methods

Study design and setting

This prospective observational study was performed at both locations of the Amsterdam University Medical Centre. From March 11th until October 28th, 2021, all adult patients (> 18years) presented to the ED by ambulance were included. Excluded patients were those with ongoing cardiopulmonary resuscitation and inter-hospital transfers.

Dutch EMS system

In the Netherlands, each ambulance team consist of an EMS nurse and a driver. EMS nurses hold licenses to administer Advanced Life Support and have completed a foundational nursing program, complemented by specialized training in areas such as emergency medicine, intensive care medicine, or cardiac care medicine, prior to embarking on their EMS traineeship. EMS drivers, on the other hand, undergo training to provide medical support to EMS nurses and to ensure the safe transportation of patients via ambulance.

Ethical concerns

Ethical approval was received by the Medical Ethical Committee of the AmsterdamUMC (Waiver: W-19_480 # 19.554). Informed consent of participants was waived by the Medical Ethical Committee.

Data collection

Patients were included on workdays between 10 a.m. and 6 p.m. as at this time a researcher was available, and most patients were presented by ambulance. EMS diagnosis and patient characteristics such as age and sex were included. Also, years of working experience of the EMS nurses was collected. ED discharge diagnosis, the need for hospital admission and ICU admission was obtained from the electronic patient record.

Data availability

Data is not publicly available. However, data is available upon reasonable request.

Diagnostic agreement

Based on the International Classification of Diseases, 11th Revision, the preliminary and ED discharge diagnosis was grouped. Two researchers (PG and LV) grouped the diagnosis based on similarities in terms of symptoms and treatment with subsequent consensuses. In case more than one diagnosis was reported, the principal diagnosis was used. If a symptom was reported instead of a preliminary diagnosis, we classified this symptom into the most appropriate disease category, see Table 1.

After classification of the EMS and ED diagnosis, they were compared and listed as either concordant or discordant. Similar diagnosis included diagnosis that were either identical diagnosis or more precise diagnosis, i.e., bacterial pneumonia instead of pneumonia. Dissimilar diagnosis was those diagnosis that did not fulfil correct diagnosis criteria. Cohen’s kappa (κ) was used to measure this interrater reliability, interpreting 0.81-1.00 as almost perfect [9].

Table 1 Classification of diseases based on the ICD-11 codes

Outcomes

The primary outcome of this study was the rate of agreement between preliminary diagnoses made by ambulance staff and physician’s discharge diagnosis made in the ED, expressed as a percentage.

Secondary outcome included the association with concordant or discordant diagnosis with the need for hospital admission, ICU admission within 72 h and 28-day mortality.

Statistical analysis

All analyses were performed using SPSS Statistics, version 27.0.0 (IBM Corporation, Armonk, NY, USA). Descriptive data were generated for all variables with frequencies, percentages, and mean [Standard deviation (SD)] or median [Interquartile range (IQR)] depending on the distribution of the data. Cases with a missing preliminary or discharge diagnosis were excluded. A p-value < 0.05 was considered statistically significant.

To compare accuracies between disease categories, a forest plot was constructed via Excel (Microsoft Excel, 2021), using the observed percentages and its 95%-confidence intervals for the different disease categories.

Results

Characteristics of study population

During inclusion period, a total of 800 patients were screened for eligibility criteria. Of these patients 65 were excluded due to missing diagnosis. This resulted in a study population of 735 patients. The median age was 66 years and 404 (55.0%) were male. Most of the preliminary diagnoses were neurological emergencies (24.4%) and cardiovascular emergencies (17.8%), see Table 2.

Primary outcome

Out of the 735 patients included in the study, the preliminary diagnosis was accurate in 584 cases (79.5%). The interrater reliability agreement was almost perfect with a Cohen’s kappa value of 0.95. There was a difference in frequency between disease categories when comparing the population with a concordant preliminary diagnosis and the group with a discordant preliminary diagnosis, see Table 2. For instance, acute internal emergencies were one of the less frequent disease categories (4.6%) in patients with a concordant preliminary diagnosis but represented the largest percentage (23.8%) in the population with a discordant preliminary diagnosis.

Table 2 Characteristics total study population (n = 735), population with a concordant preliminary diagnosis (n = 584), and population with an discordant preliminary diagnosis (n = 151)

Differences in accuracy between disease categories

As shown in Fig. 1, the highest accuracy of 94% was observed in traumatic injuries, and the lowest in acute internal emergencies (43%). The total accuracy of 79% is also represented with its 95%-confidence interval [77–82%]. For example, infectious diseases show a significant higher accuracy (85% [95%-CI: 78–92%]) compared to endocrine and metabolic emergencies (48% [95%-CI: 27–68%]).

Fig. 1
figure 1

Forest plot. The vertical axis shows the disease categories with below the total accuracy. The horizontal axis represents the observed accuracy (box) with its 95%-confidence interval (horizontal line), both expressed in percentages. CI = confidence interval

Secondary outcome

For 706 patients, 28-day mortality data was available. The 28-day mortality was not significantly different between patients with concordant diagnosis (6.6%) versus discordant diagnosis (8.5%), p = 0.429.

For 732 patients the need for ICU admission within 72 h after ED presentation was available. The need for ICU admission was not significantly different between the groups, 7.9% for concordant diagnosis versus 4.6% for discordant diagnosis, p = 0.171.

For all 735 patients the need for hospital admission after ED presentation was available. For patients with concordant preliminary diagnosis, 46.9% were admitted and for those with a discordant diagnosis this was 40.4%, p = 0.152.

Discussion

This is the first Dutch study that examined to what extent the preliminary diagnosis made by the EMS is in agreement with ED discharge diagnosis. Based on these findings, we conclude that the accuracy of EMS nurses is high for patients with neurological emergencies, traumatic injuries, and those with infectious diseases. As hypothesized, the extent of agreement was higher compared to prior studies, however, the difference was only slightly increased. [5, 6] A possible explanation of this level of agreement, is the high level of education and traineeship for the Dutch EMS, however, another explanation is that many of the EMS nurses were previously working at the ED/ICU and therefore have a significant clinical experience.

On the contrary, a German study, examining diagnostic agreement for EMS emergency physicians, showed a slightly lower level of agreement (64%,) despite their high level of education [10]. This difference could be attributed to the use of a more specific disease classification (e.g., hypoglycaemia, hypertensive crisis), resulting in a stricter assessment of whether the EMS diagnosis was correct or not. In our perspective, a less detailed classification of the EMS diagnosis is more clinically relevant as this will engage the correct medical expertise (i.e., internal medicine, trauma, cardiac etc.) upon entry into the ED.

Furthermore, the variation in accuracies between the different disease categories are in concordance with the study of Koivulahti et al. [5] They stated that high accuracies are found in categories with specific observations, such as neurological failure, detectable injuries, and electrocardiographic abnormalities. The lower accuracies for endocrine and metabolic and acute internal emergencies might be attributed to the lack of readily available diagnostics and specific symptoms and the requirement for more profound knowledge of diseases including more extensive anamnesis and physical examination.

Particularly interesting is the high level of diagnostic agreement for patients with infectious diseases. A previous study in the Netherlands showed that in only 18% of (severe) sepsis, the disease severity was accurately diagnosed despite correct identification of an infectious disease state by the EMS. [11] This maybe be an interesting field of not only science to enhance the care for this specific sick group of patients.

Strengths and limitations

Despite the prospectively collected data in two academic teaching hospitals, this study has several limitations. A limitation pertains to the relatively small sample size compared to the number of disease categories. Therefore, it is expected that the accuracy in these categories may be overoptimistic. [12] Considering the setting of this study (two university hospitals) may also influence external validity of this study as more peripherally located hospital may see different groups of patients with different diagnosis.

Also, patients were only included during working days between 10 a.m. and 6 p.m., while previous studies showed that the accuracy in overall clinical judgement fluctuates during the day and especially in the night. [13, 14] However, we can’t be certain at this point to what degree the overall (24/7) accuracy of diagnosis of EMS versus ED in the Netherlands may be. In addition, the ED physician’s discharge diagnosis was considered the gold standard for determining whether the preliminary diagnosis was correct or not. However, previous studies state that physicians can also make wrong conclusions. [15]

Conclusion

In the Netherlands, the extent of agreement between EMS diagnosis and ED discharge diagnosis varies between categories. Accuracy is high in diseases with specific observations, e.g., neurological failure, detectable injuries, and electrocardiographic abnormalities. Further studies should investigate how we can use this to reduce time spend at the ED or to improve patient outcome.

Data availability

Data is not publicly available. However, data is available upon reasonable request. Please contact L.I.Veldhuis@amsterdamumc.nl for data requests.

Abbreviations

ED:

Emergency department

EMS:

Emergency medical services

ICU:

Intensive care unit

References

  1. Alam N, et al. Epidemiology, recognition and documentation of sepsis in the pre-hospital setting and associated clinical outcomes: a prospective multicenter study. Acute Med. 2016;15(4):168–75.

    Article  PubMed  Google Scholar 

  2. Hautz WE, et al. Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scand J Trauma Resusc Emerg Med. 2019;27(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bastakoti M, et al. Discrepancy between emergency department admission diagnosis and hospital discharge diagnosis and its impact on length of stay, up-triage to the intensive care unit, and mortality. Diagnosis (Berl). 2021;9(1):107–14.

    Article  PubMed  Google Scholar 

  4. Eames J, Eisenman A, Schuster RJ. Disagreement between emergency department admission diagnosis and hospital discharge diagnosis: mortality and morbidity. Diagnosis (Berl). 2016;3(1):23–30.

    Article  PubMed  Google Scholar 

  5. Koivulahti O, Tommila M, Haavisto E. The accuracy of preliminary diagnoses made by paramedics - a cross-sectional comparative study. Scand J Trauma Resusc Emerg Med. 2020;28(1):70.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Purabdollah M, et al. Comparison of the Agreement and Accuracy between paramedic and hospital diagnosis. Air Med J. 2022;41(2):228–32.

    Article  PubMed  Google Scholar 

  7. Fuller GW, et al. The diagnostic accuracy of pre-hospital assessment of acute respiratory failure. Br Paramed J. 2020;5(3):15–22.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Green RS, et al. Paramedic Recognition of Sepsis in the Prehospital setting: a prospective observational study. Emerg Med Int. 2016;2016:p6717261.

    Article  Google Scholar 

  9. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–82.

    Article  PubMed  Google Scholar 

  10. Ramadanov N, et al. Diagnostic agreement between Prehospital Emergency and In-Hospital Physicians. Emerg Med Int. 2019;2019:3769826.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Veldhuis LI, et al. Recognition of sepsis in the pre-hospital environment one year after intensive traineeship. Eur J Intern Med. 2021;92:115–6.

    Article  CAS  PubMed  Google Scholar 

  12. Peduzzi P, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9.

    Article  CAS  PubMed  Google Scholar 

  13. Ramadanov N, et al. Influence of Time of Mission on correct diagnosis by the Prehospital Emergency Physician: a retrospective study. Emerg Med Int. 2019;2019:p3727081.

    Article  Google Scholar 

  14. Akerstedt T, Wright KP Jr. Sleep loss and fatigue in Shift Work and Shift Work Disorder. Sleep Med Clin. 2009;4(2):257–71.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hussain F, et al. Diagnostic error in the emergency department: learning from national patient safety incident report analysis. BMC Emerg Med. 2019;19(1):77.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study is not funded.

Author information

Authors and Affiliations

Authors

Contributions

LV: conceptualization, methodology, investigation, data curation, writing original draft. PG: investigation, writing original draft. PN: supervision and reviewing. JL: supervision and reviewing. All authors reviewed the manuscript.

Corresponding author

Correspondence to Lars I. Veldhuis.

Ethics declarations

Ethics approval and consent to participate

This study involves human participants and was approved by the Medical Ethics Committee of AmsterdamUMC. The Medical Ethics Committee of AmsterdamUMC waived the need for patient consent.

Consent for publication

not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Veldhuis, L.I., Gouma, P., Nanayakkara, P.W.B. et al. Diagnostic agreement between emergency medical service and emergency department physicians, a prospective multicentre study. BMC Emerg Med 24, 120 (2024). https://doi.org/10.1186/s12873-024-01041-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12873-024-01041-7

Keywords