Patient safety incident capture resulting from incident reports: a comparative observational analysis
© Reznek et al.; licensee BioMed Central. 2015
Received: 19 March 2014
Accepted: 24 March 2015
Published: 11 April 2015
Patient safety incident (PSI) discovery is an essential component of quality improvement. When submitted, incident reports may provide valuable opportunities for PSI discovery. However, little objective information is available to date to quantify or demonstrate this value. The objective of this investigation was to assess how often Emergency Department (ED) incident reports submitted by different sources led to the discovery of PSIs.
A standardized peer review process was implemented to evaluate all incident reports submitted to the ED. Findings of the peer review analysis were recorded prospectively in a quality improvement database. A retrospective analysis of the quality improvement database was performed to calculate the PSI capture rates for incident reports submitted by different source groups.
363 incident reports were analyzed over a period of 18 months; 211 were submitted by healthcare providers (HCPs) and 126 by non-HCPs. PSIs were identified in 108 resulting in an overall capture rate of 31%. HCP-generated reports resulted in a 44% capture rate compared to 10% for non-HCPs (p < 0.001). There was no difference in PSI capture between sub-groups of HCPs and non-HCPs.
HCP-generated ED incident reports were much more likely to capture PSIs than reports submitted by non-HCPs. However, HCP reports still led to PSI discovery less than half the time. Further research is warranted to develop effective strategies to improve the utility of incident reports from both HCPs and non-HCPs.
KeywordsIncident reporting Quality improvement Patient safety Emergency medicine
In 1999, the Institute of Medicine presented startling findings that 44,000 to 98,000 patients die annually due to medical errors . Since that report, patient safety has become a major priority in healthcare.
Incident reporting is important for improving patient safety because it can lead to the identification of patient safety incidents (PSIs), defined by the World Health Organization as “event[s] or circumstance[s] that could have resulted, or did result, in unnecessary harm to a patient” . PSI capture is a crucial step in patient safety improvement because it enables subsequent development and implementation of preventative measures . In Emergency Medicine (EM), potential incidents are reported to Emergency Department (ED) leadership from a wide variety of sources including ED healthcare providers (HCPs), HCPs in other departments, hospital quality officers, hospital committees, as well as patients and their families. These reports may represent valuable opportunities for discovery of true PSIs, however little objective information is available to quantify or demonstrate this value. In EM, a limited number of studies suggest that patient complaints may be useful in improving quality [4,5], and one study has implied that physician-generated concerns may be more valuable for quality improvement . To our knowledge however, there are no studies in any field of healthcare that have specifically evaluated the efficacy of incident reports from different sources in identifying PSIs.
The objective of this investigation was to assess how often ED incident reports from different sources led to the identification of PSIs.
Study design and setting
Data for this observational study were collected between October 2010 and April 2012 at an urban, tertiary care ED with an approximate annual census of 66,000 adult patients. The ED is the primary teaching site for an accredited 3-year EM residency. Board-certified EM physicians care directly for approximately 95% of the patients either as the only provider or through direct supervision of EM and rotating residents. Mid-level practitioners care for approximately 5% of the patients as the primary provider. The University of Massachusetts Medical School Institutional Review Board reviewed and approved the study.
Methods and measurements
To analyze incident reports submitted to the ED, a standardized peer review process was implemented in October 2010. The process was based on one previously described at Detroit Receiving Hospital (DRH) , however we modified the DRH process also to include systems failure identification and analysis. As a matter of policy, every incident report submitted to the ED was analyzed via this process which is described below.
As a first step, each incident report underwent a preliminary screening review. Due to practical considerations, we opted for a single-reviewer format for the preliminary screening. To minimize potential accuracy and bias limitations inherent to a single-reviewer screening, the chosen preliminary reviewer had greater than ten years of experience in healthcare quality and the incident reports could only be closed if it could be concluded with absolute certainty that no PSI had occurred. If a PSI could not be excluded with absolute certainty, the incident report was forwarded to the ED Peer Review Committee (PRC) for analysis.
Convening monthly, the PRC was open to all ED physicians, residents and mid-level practitioners, however eight board-certified, attending physicians attended regularly to provide consistency in the evaluation process. Prior to each PRC meeting, ED practitioners involved with a case to be reviewed were required to provide a written account of the care and any other circumstances related to the specific quality issue(s) in question. This allowed for first-hand accounts of the care of the patient that might not be readily available in the medical record to be available to the PRC. In addition, the involved providers were encouraged to attend the PRC meeting. The medical records (and the practitioners’ written feedback if they were not present) were de-identified to foster a non-punitive and objective environment and to maintain patient confidentiality.
Patient safety incident types identified by the peer review committee
A failure in assessment of potential disease severity during triage
Abnormal vital signs not recognized as a potential sign of shock
A failure due to an issue with ED staff communication or a shared responsibility across multiple ED staff
Change in vital signs not communicated to the attending physician
A failure due to an issue with communication between ED and hospital staff or a shared responsibility between the ED and hospital staff
Pertinent information not communicated to the admitting team
ED work environment
A failure resulting from the lack, malfunction, or mal-design of resources, equipment or physical space within the ED or a failure resulting from following an established ED policy or clinical practice guideline
Missing equipment needed for care of a patient
Hospital work environment
A failure resulting from the lack, malfunction, or mal-design of resources equipment or physical plant outside the ED but still within the hospital or a failure resulting from following an established hospital policy or guideline
Specialty testing areas remotely located from the ED causing prolonged transport time
A failure occurred after the patient was admitted to an in-patient service but was still physically in the ED
Major cognitive error
An error in cognition which represents serious mismanagement in a knowledge area basic to emergency
Failure to diagnose and treat ST-elevation myocardial infarction
An error in cognition which represents mismanagement which is either less serious than a major cognitive error or in an area less basic to emergency medicine
Failure to consider the institutional antibiogram during antibiotic selection for treatment
Missed radiographic finding
An error in interpretation of a radiographic study that did not reach the level of a cognitive or major cognitive error
Missed fracture on radiographic interpretation that was still splinted based on clinical suspicion
An error in following a clinical or administrative policy, guideline or standard practice that does not reach the level of cognitive or major cognitive error
Failure to alert the transplant service when a transplant patient is in the ED
A technical error during performance of a procedure that does not reach the level of a cognitive or major cognitive error
Insufficient sterile technique
All incident reports received by the ED during the study period were recorded prospectively in a secure, quality improvement database. The data included the respective source of each incident report and the results of the peer review analysis. The entire peer review process, including the database and practitioner written responses, was structured so that it complied with state peer review legal discoverability protection requirements.
Sources of incident reporting
An ED practitioner who was directly involved in the care of the patient when the perceived patient safety incident occurred
An ED practitioner not directly involved in the care of the patient at the time that the perceived patient safety incident occurred
A practitioner within the hospital that is not an ED practitioner or a clinical committee within the hospital (for example - the Stroke Care Committee)
A health care practitioner from outside the hospital
A patient or family member
Central hospital management forwarding a concern from an outside agency (for example - an insurance company generated concern)
Risk management; a hospital administrative unit responsible for risk assessment and quality management.
For the present study, a retrospective analysis of the existing peer review database was performed to determine the PSI capture rate of incident reports from each of the groups and sub-groups. Specifically, each incident report in the database was reviewed to determine the source group and sub-group as well as if any failures or errors were identified for that case indicating that a PSI had occurred.
Outcomes and analysis
The primary outcome of this study was the frequency of PSI capture based on the source of the incident report. Frequency of PSI capture was expressed as a percent. A two-tailed Fisher’s exact test was used to determine if the capture rates differed between the HCP and non-HCP groups. We similarly compared the frequency of PSI capture within the sub-groups.
Incident reports and patient safety incident capture by source group
Number of reports
Percent of total reports
Number resulting in PSI identification
PSI capture rate
Healthcare Providers (HCP) n = 211
p = 0.99
Non-Healthcare Providers (Non-HCP) n = 140
p = 0.14
Among the subcategories of HCPs and non-HCPs, PSI capture ranged from 41.7-45.0% and 0–20.0% respectively. Reports submitted by the four different HCP subcategories and by the three different non-HCP subcategories did not demonstrate within-group differences with respect to the frequency of PSI capture (p = 0.99 and p = 0.14 respectively) (Table 3).
The results of this investigation of ED incident reports revealed two key findings. First, incident reports submitted by HCPs led to the capture of PSIs nearly 4.5 times more frequently than those submitted by non-HCPs. And second, while HCP-generated incident reports were more likely to result in PSI capture, they resulted in capture only less than half the time.
Given that HCPs are likely to have more insight into healthcare delivery systems than non-HCPs, the finding that HCP-generated reports led more frequently to PSI discovery may not be altogether surprising. More remarkable however was the finding of over a four-fold difference in resultant PSI discovery between these groups. The non-HCP category in our sample was dominated by reports submitted by patients and families. Data from previous investigations have suggested that patient and family concerns often are related to incomplete or delayed relief in symptoms, suboptimal practitioner communication or billing related to their medical services [4,5,9]. The design of this study did not address this topic directly as the specific concerns prompting the incident reports were not systematically recorded in the database, however our experience was consistent with these previous investigations. Of note, while concerns generated by patients and families most often did not lead to PSI identification, we feel strongly that their feedback is still highly valuable for understanding and improving the patient experience.
With regards to this investigation’s second key finding of HCP-generated incident reports capturing PSIs 44.5% of the time, the reasons for this seemingly low capture rate are unclear. Reports from each HCP sub-group resulted in PSI identification at a similar frequency regardless of the HCP’s role or if they were directly involved in the care in question. For providers not directly involved in the care in question (ED, HP and OP), the relatively low percentage of capture could be due to the fact that they were not present during the care in question and therefore were biased toward presuming that an adverse outcome was due to a preventable failure or error. However, this does not explain the capture rate for self-reporting. Perhaps self-reporters were biased by emotion or a sense of personal responsibility when adverse outcomes occurred. In any case, the observed over-reporting by all types of HCPs can be interpreted as a promising sign of their engagement in reducing PSIs. The observed over-reporting by HCPs in this study may appear to contradict previous studies suggesting that HCP under-reporting is a common barrier to improving patient safety [3,9], however the two phenomena are not necessarily mutually exclusive. Based on the prior literature and the results of this study, it is likely that HCP under-reporting and over-reporting are occurring simultaneously. The implication is that HCPs are engaged in patient safety improvement, but there is opportunity to better educate them about patient safety. Optimizing practitioner education in this area may improve over and under-reporting simultaneously. Ultimately reaching a capture rate of 100% may be unrealistic. Furthermore, if a perfect capture rate is attained, it is likely that incidents are going under-reported. To our knowledge, this is the first study reporting capture rates from incident reports, so there are no other data to assist with benchmarking. While we cannot definitively conclude that the rate observed in this study is low, we believe that a capture rate of 44.5% from HCP incident reports can be improved. It stands to reason that the ideal rate may be between the observed rate and 100%. In our institution, we are considering an initial goal of 75% as we continue to improve our incident reporting processes.
While education related to patient safety may be a viable mechanism to improve the PSI capture rate for HCP incident reports, the mechanism to improve them for non-HCP reports is less clear. A prior study reported utilizing structured patient interviews to elicit ED patient feedback rather than self-generated concerns, however the capture rate was still low (9%) . Despite the low rate similar to that observed in our study, we believe that structured patient interviews may still be a promising option. To improve the capture rate of patient and family incident reporting, perhaps the structured interview tool previously reported requires redesign or perhaps follow-up interviews should be directed only toward patients that are high risk for having been subject to a PSI.
Incident reporting by HCPs as well as patients and family members traditionally has been cited as an important tool for PSI identification [9,11-14]. Recently however, some research has shown that while incident reporting can detect latent errors, it may miss a large fraction of PSIs . More proactive and systematic surveillance techniques have been suggested as superior for improving patient safety [15-18] Systematic surveillance techniques however may have potential shortcomings including the fact that they may miss PSIs that do not fit strict systematic surveillance criteria [15,16]. For this reason, we believe that incident reporting and systematic surveillance complement each other, and there is in fact a role for both in healthcare quality improvement . Therefore, gaining a better understanding of incident reporting is important so that we may continue to improve the effectiveness of this incident identification modality.
Previously we have published results of an investigation of the type, frequency, and resultant harm related to PSIs that were discovered via the ED-based incident reporting system . The purpose of the present study was not to review these details again but to investigate the utility of incident reports from different sources in leading to PSI discovery in general. Nevertheless, it is worth noting that in the prior publication, we sought to broadly classify PSIs into those related to either practitioner-based error or systems failures. We are aware that this approach differs from the more detailed categorization scheme used by the National Health Service in the United Kingdom well as that which has been proposed by the World Health Organization [21,22]. Therefore, directly comparing our findings of PSI types with either of those systems would be difficult.
This study of incident reports has potential limitations that warrant consideration when interpreting the results. First, the peer review process was designed to be systematic and as objective as possible; however any review process of incident reports necessarily must involve human interpretation. Barriers for impartial decision making among peers are known to exist, and hindsight bias is likely unavoidable in determining medical errors . A range of measures were designed into the review process to mitigate the influence of biases (discussed in the Methods section), but having completely removed all bias was unlikely. A second potential limitation of the study was the fact that the peer review database only included the original source when more than one report for a case was received. In our anecdotal experience however, very few cases involved reports from both HCPs and non-HCPs, and there was no apparent trend in whether HCPs or non-HCPs were the first to report. The reason for this may have been that non-HCPs also had easily accessible mechanisms to submit incident reports in our institution. In addition to other mechanisms, non-HCPs could submit concerns verbally to the “Patient Care Services” department at any time including at the time of service. Another potential limitation of this study was that it was performed at a single center and focused on EM as opposed to multiple medical specialties. With regard to the generalizability of our study within EM, the frequency of incident reports we observed (0.36% of ED patient encounters) is consistent with other published ED studies which cite a range of 0.012% to 0.61% in EDs with 13,000 to 94,000 patient encounters per year [24,25]. While it cannot be determined if the distribution of incident reporter type is consistent with other EDs, the overall frequency of incident reporting is similar to other reports in the literature. The applicability of this study beyond EM is less clear. It may be the case that incident reporting is more likely to occur in EDs because nearly all care is handed off to other HCPs who naturally review the care that has been delivered prior to their involvement. However the fact that incident reports from non-ED HCPs resulted in similar PSI capture rates as those from ED HCPs in the study provides some promising evidence that the results may be generalizable to other specialties.
determine the most effective strategies to further increase and improve HCP incident reporting
determine if soliciting directed patient and family feedback may improve PSI capture and
determine if patient feedback systems targeted at subgroups of ED patients that are at higher risk of safety incidents may improve PSI capture rates from this group.
- Kohn LT, Corrigan JM, Donaldson MS, editors. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press, Institute of Medicine; 1999.Google Scholar
- WHO. More than words: Conceptual framework for the international classification for patient safety; final technical report; 2009. Available at: http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf.
- Leape LL. Errors in medicine. Clin Chim Acta. 2009;404:2–5.View ArticlePubMedGoogle Scholar
- Chande VT, Bhende MS, Davis HW. Pediatric emergency department complaints: a three-year analysis of sources and trends. Ann Emerg Med. 1991;20:1014–6.View ArticlePubMedGoogle Scholar
- Schwartz LR, Overton DT. Emergency department complaints: a one-year analysis. Ann Emerg Med. 1987;16:857–61.View ArticlePubMedGoogle Scholar
- Griffey RT, Bohan JS. Healthcare provider complaints to the emergency department: a preliminary report on a new quality improvement instrument. Qual Saf Health Care. 2006;15:344–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Berk WA, Welch RD, Levy PD, Jones JT, Arthur C, Kuhn GJ, et al. The effect of clinical experience on the error rate of emergency physicians. Ann Emerg Med. 2008;52:497–501.View ArticlePubMedGoogle Scholar
- Cosby KS. A framework for classifying factors that contribute to error in the emergency department. Ann Emerg Med. 2003;42:815–23.View ArticlePubMedGoogle Scholar
- Schenkel S. Promoting patient safety and preventing medical error in emergency departments. Acad Emerg Med. 2000;7:1204–22.View ArticlePubMedGoogle Scholar
- Friedman SM, Provan D, Moore S, Hanneman K. Errors, near misses and adverse events in the emergency department: what can patients tell us? CJEM. 2008;10:421–7.View ArticlePubMedGoogle Scholar
- Department of Health. An Organisation with a Memory: Report of an Expert Group on Learning from Adverse Events in the NHS. London: The Stationery Office; 2000.Google Scholar
- Petersen LA, Orav EJ, Teich JM, O’Neil AC, Brennan TA. Using a computerized sign-out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv. 1998;24:77–87.PubMedGoogle Scholar
- Weingart SN, Pagovich O, Sands DZ, Li JM, Aronson MD, Davis RB, et al. What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. J Gen Intern Med. 2005;20:830–6.View ArticlePubMedPubMed CentralGoogle Scholar
- Weingart SN, Zhu J, Chiappetta L, Stuver S, Schneider E, Epstein A, et al. Hospitalized patients’ participation and its impact on quality of care and patient safety. Int J Qual Health Care. 2011;23:269–77.View ArticlePubMedPubMed CentralGoogle Scholar
- Vincent C. Incident reporting and patient safety. BMJ. 2007;334:51.View ArticlePubMedPubMed CentralGoogle Scholar
- Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med. 2003;18:61–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Sari AB, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ. 2007;334:79.View ArticlePubMedGoogle Scholar
- Hospital incident reporting systems do not capture most patient harm: OIG, Department of Health and Human Services. 2012. Available at: http://oig.hhs.gov/oei/reports/oei-06-09-00091.asp.
- Olsen S, Neale G, Schwab K, Psaila B, Patel T, Chapman EJ, et al. Hospital staff should use more than one method to detect adverse events and potential adverse events: incident reporting, pharmacist surveillance and local real-time record review may all have a place. Qual Saf Health Care. 2007;16:40–4.View ArticlePubMedPubMed CentralGoogle Scholar
- Jepson ZK, Darling CE, Kotkowski KA, Bird SB, Arce MW, Volturo GA, et al. Emergency department patient safety incident characterization: an observational analysis of the findings of a standardized peer review process. BMC Emerg Med. 2014;14:20.View ArticlePubMedPubMed CentralGoogle Scholar
- World Health Organization. Conceptual framework for the international classification of patient safety. Version 1.1. 2009. Retrieved from http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf.
- NHS. National Reporting and Learning System (NRLS). http://www.nrls.npsa.nhs.uk/. Accessed January 7, 2015.
- Wears RL, Nemeth CP. Replacing hindsight with insight: toward better understanding of diagnostic failures. Ann Emerg Med. 2007;49:206–9.View ArticlePubMedGoogle Scholar
- Chamberlain JM, Shaw KN, Lillis KA, Mahajan PV, Ruddy RM, Lichenstein R, et al. Creating an infrastructure for safety event reporting and analysis in a multicenter pediatric emergency department network. Pediatr Emerg Care. 2013;29:125–30.View ArticlePubMedGoogle Scholar
- Considine J, Mitchell B, Stergiou HE. Frequency and nature of reported incidents during Emergency Department care. Emerg Med J. 2011;28:416–21.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.