Evaluation of comprehensiveness and reliability of electronic health records concerning resuscitation efforts within academic intensive care units: a retrospective chart analysis

Background According to the literature, the validity and reliability of medical documentation concerning episodes of cardiopulmonary resuscitation (CPR) is suboptimal. However, little is known about documentation quality of CPR efforts during intensive care unit (ICU) stays in electronic patient data management systems (PDMS). This study analyses the reliability of CPR-related medical documentation within the ICU PDMS. Methods In a retrospective chart analysis, PDMS records of three ICUs of a single university hospital were searched over 5 y for CPR check marks. Respective datasets were analyzed concerning data completeness and data consistency by comparing the content of three documentation forms (physicians’ log, nurses’ log, and CPR incident form), as well as physiological and therapeutic information of individual cases, for missing data and plausibility of CPR starting time and duration. To compare data reliability and completeness, a quantitative measure, the Consentaneity Index (CI), is proposed. Results One hundred sixty-five datasets were included into the study. In 9% (n = 15) of cases, there was neither information on the time points of CPR initiation nor on CPR duration available in any data source. Data on CPR starting time and duration were available from at least two data sources in individual cases in 54% (n = 90) and 45% (n = 74), respectively. In these cases, the specifications of CPR starting time did differ by a median ± interquartile range of 10.0 ± 18.5 min, CPR duration by 5.0 ± 17.3 min. The CI as a marker of data reliability revealed a low consistency of CPR documentation in most cases, with more favorable results, if the time interval between the CPR episode and the time of documentation was short. Conclusions This study reveals relevant proportions of missing and inconsistent data in electronic CPR documentation in the ICU setting. The CI is suggested as a tool for documentation quality analysis and monitoring of improvements.


Introduction
Recording reliable and valid data on resuscitation efforts during in-hospital cardiac arrest serves the interests of patients as well as legal, quality assurance, and scientific purposes. In practice, discrepant and missing data in resuscitation records are a frequent phenomenon [1][2][3][4][5][6][7][8][9]. Since in operating rooms as well as in intensive care settings, automated collection of patient vital parameters, supplemented by semi-automated or manual documentation of additional details, in electronic patient data management systems (PDMS) is on the rise, the electronic record may be the only place where data on episodes of cardiopulmonary resuscitation (CPR) are kept. Experimental findings suggest, that electronic documentation might have some benefits compared to standard paper files during CPR [10][11][12][13]. However, the quality of electronic patient records concerning the documentation of CPR in intensive care units (ICU) is largely unknown.
In this retrospective study, we assessed the comprehensiveness and reliability of CPR documentation in the PDMS of three ICUs of a single academic center. For this purpose, we analyzed whether the time point and duration of resuscitation measures, the cardiac rhythm, and cardiac defibrillations were represented plausibly and consistently within the PDMS. Further, a metric for systematically comparing the consistency of time interval related CPR documentation is suggested.

Ethics approval and consent to participate
According to the local clinical ethics committee, no ethics approval or patient consent was necessary (inquiry no. 17-468-104, University of Regensburg Ethics Committee). The data originated from three distinct ICUs of a university hospital. No experiments on humans or animals were conducted for this study.

Data collection
The participating ICUs use the PDMS MetaVision Suite (Fa. iMDSoft Ltd., Tel Aviv, Israel). Vital signs are recorded at an interval of 1 min. Part of the PDMS documentation is the daily measurement of the condensed, 10-item Therapeutic Intervention Scoring System (TISS), the Core-TISS-10. The Core-TISS-10 encompasses the 10 most laborious procedures from the TISS-28, with the intention to reflect the workload of critical care for the individual patient and is a component of the German reimbursement system [14,15]. In all electronic patient records from 2012 until 2016, the electronic documentation of the Core-TISS-10 was searched for the keyword "resuscitation".
Whenever the TISS search revealed the keyword "resuscitation", the different documentary forms were manually searched for hints concerning details on the appointed incident. These three forms comprised of the physicians' free text log, the nurses' free text log, and the CPR incident documentation form. From each, the documented time point of CPR initiation and its duration were recorded for analysis. In addition, a fourth data source for determining CPR initiation and duration was analyzed, namely the recorded vital parameters and therapy details. These were manually reviewed for predefined indicators of CPR initiation and cessation (Table 1), respectively. The searched interval thereby reached from 15 min before the earliest mentioning of CPR initiation in the three mentioned documentation forms until 15 min after the latest mentioning of CPR termination. The interval between the earliest indicator of CPR initiation according to Table 1 until the earliest indicator of CPR cessation was considered the estimated CPR episode according to the vital signs and therapy documentation.
From these data, up to four separate CPR intervals could be found in the previously mentioned four data sources documentation (physicians' log, nurses' log, the CPR incident form, the vital signs and therapy documentation), which were then compared for consistency. In addition, the main cardiac rhythm during CPR as well as applied electrical defibrillations were assessed.

Exclusion criteria
Cases were excluded, if CPR was performed fully or in part outside the ICU, e.g., when patients were admitted during CPR or if CPR was necessary during patient transport outside the ward. Further, cases without manual chest compression were not included into the analysis. This was the case, whenever resuscitation efforts were described as "pharmacologic resuscitation". Finally, all cases with a documented CPR duration greater than 120 min were removed from data consistency evaluation, because the plausibility of such events was deemed questionable.

Consentaneity index for assessing CPR documentation quality
To assess the plausibility and consistency of the documented CPR time courses between the three forms and the vital / therapy data, we compared the entries of the various documentation forms of individual patients. Concerning the timely parallelism of the documentation, a newly defined quality indicatorthe Consentaneity Index (CI)was calculated as follows: The period of time between the earliest mentioned CPR beginning until the latest mentioned CPR ending was divided into one-minute intervals. For each interval, we assessed, in how many of the four data sources the ongoing CPR was indicated to be ongoing. The Gaussian partial sum of the resulting count for each interval was summed up. I.e., if CPR was mentioned in one form, one point was assigned. For the second mentioning within the same one-minute interval, two additional points were counted; for the third mentioning three additional points etc. Subsequently, the points of all intervals were added up, and the summed score was divided by the number of intervals and the partial sum of the number of all forms analyzed (n all = 4, see Table 2a). Thus, the CI takes values between 0 (no overlap of documented CPR intervals at all) and 1 (perfect fit of CPR documentation in all data sources).
The CI as described above is influenced by both the consistency and the completeness of CPR documentation. Thus, data sources lacking information on a CPR event reduce the value of the CI. To allow a focus on the consistency of CPR documentation alone, the standard CI was further differentiated into a so-called soft CI, which is not influenced by empty data sources. The soft CI is calculated by substituting the number of all forms by the number of forms presenting data on the actual CPR incident (n data , Table 2b).

Calculation example
Case Vignette.
A 60-year-old intensive care unit patient falls into ventricular fibrillation at 12:01 o'clock. Manual chest compressions are initiated immediately, inspiratory oxygen is increased to 100%, and the defibrillator is brought into action. After a single shock, sinus rhythm is restored at 12:02, and ROSC with sufficient circulation parameters is obtained.
In the physician's log, resuscitation efforts starting at 12:01 and lasting for two minutes are documented. The nurses' log speaks of a 3-min resuscitation interval with a starting time at 12:00. An entry is created within the CPR incident form with a single time point at 12:05. Finally, from the vital signs and therapy documentation, the first visible indicator of CPR activity is the defibrillation, which is documented at 12:02. At 12:03, inspiratory oxygen fraction is reduced to baseline settings, indicating the successful termination of CPR.

Calculating the standard consentaneity index
According to the clinical documentation, the CI is calculated as presented in Fig. 1. The evaluated CPR interval ranges from 12:00 to 12:05 and thus comprises 6 one-minute intervals (i = 6). For each minute with a single mentioning of CPR activity in the data sources (n = 1), one point is assigned. In case of two overlapping CPR entries (n = 2), three points are added (Δ2 = 3). At 12:02, three data sources report the ongoing CPR (n = 3), thus Δ3 = 6 points are added. Since for 12:04 there is no documentation of resuscitation efforts, for this minute no points are assigned. The sum ∑Δn = 12 is subsequently divided by the number of intervals i = 6 and the Gaussian partial sum of the number of data sources evaluated (Δn max = 10). The calculation results in a standard CI of 0.2 ( Fig. 1).  Table 2 Calculation of the Consentaneity Index (CI) (i = number of CI intervals examined; n = number of simultaneously appearing CPR documentation within the individual CI interval; n all = number of forms examined within the study (i.e., 4); n data = number of forms with CPR data; Δ = operator for the Gaussian partial sum with Δx = x(x + 1)/2) a) Standard CI:

Interpretation of the CI
To assist in the interpretation of standard / soft CI findings, typical documentation aberrations (missing data, differences in CPR starting time, and / or duration) and the resulting CI values are displayed in Table 3 and Fig. 2).

Daytime effect
To assess whether documentation quality would vary between day and night shifts, both the standard and soft CI were compared between CPR incidents during daytime (6:00 AM until 09:59 PM) to those during nighttime (10:00 PM until 5:59 AM) in cases with known CPR starting times.

Latency of documentation
The PDMS allowed further to trace the latency between the initiation of CPR and the time point of entering the incident into the PDMS. Thus, whenever a time point of CPR initiation could be found in a given data source, the time interval until its documentation was noted. In a second step, this time latency was correlated to data quality, namely the soft CI.

Case numbers
The search for CPR patients revealed 245 hits. Twentytwo cases were removed from the study since the CPR mentioning had been deleted after false documentation (which was still visible due to the traceability of all changes within the electronic record). In addition, 42 cases have been omitted because the resuscitation was partly performed outside the ICU. Sixteen cases were removed due to the resuscitation being solely pharmacologic without obvious need for chest compressions. Thus, 165 cases were included into the study.
For analysis involving the CI, CPR procedures with a documented duration of more than 120 min were disregarded, what was the case in 11 incidents. Therefore, 154 cases could be included into the evaluation of the CI (Fig. 3).

Data completeness Frequency of documentation in data sources
Out of the 165 CPR episodes included into the study, 33 (20%) were represented in all four examined data sources. Seventy-four (45%) cases were reported in three sources, 27 (16%) in two, and 16 (10%) in only one data source. There were 15 CPR incidents (9%) for which no data in any of the analyzed documentation locations could be found.
Incident documentation was most likely to be found in the physician's log (76%), followed by the nurse's log (69%) and the CPR incident form (60%). From vital signs and therapeutic measures, 55% of CPR incidents could be read out.

Completeness of time related documentation
In most cases, time related data on CPR incidents were found within at least one of the evaluated data sources. However, no information could be found in 23% of cases concerning CPR duration and 13% concerning CPR starting time. In 9%, neither data on duration nor starting time could be retrieved (Table 4).
To conclude on time-related CPR documentation reliability, data need to be available from at least two data sources within individual incidents. This was applicable

Consentaneity index
Standard consentaneity index The standard CI could be evaluated in 154 cases. The actual CI ranged from 0 to 0.431. The median CI was 0.100 (IQR 0.140, 10th percentile 0.000, 90th percentile 0.298) (Fig. 4a and c).
Soft CI The soft CI focusses on the consistency of available data and is not influenced by empty data sources for a given CPR episode. Therefore, the results for the soft CI are higher: The median soft CI was found to be 0.244 (IQR 0.298, 10th percentile 0.000, 90th percentile 0.859), with a data range of 0 to 1 (which was reached in 10 cases) (Fig. 4b and c).

Consistency of cardiac rhythm and defibrillation data
The initial cardiac rhythm was available from the physicians' log in 47.9%, from the nurses' log in 29.7%, and from the incident reporting form in 9.7%. In 68 cases (41.2%), the cardiac rhythm was absent in all data sources, and in eight cases (4.8%) the different data sources for individual cases contained conflicting information. Defibrillation was performed in 33 cases (20.0%). A shockable rhythm was mentioned in 26 cases (15.7%), which was treated by defibrillation in 21 cases.   Of the remaining 139 cases without explicit mentioning of a shockable rhythm, in twelve cases defibrillation was documented (four of those in cases with documentation of a non-shockable rhythm and eight with no rhythm available). Thus, according to the PDMS record, the rate of correct defibrillation or correct refraining from defibrillation was 89.7% (148 cases). In 17 cases (10.3%), according to the documentation, shockable cardiac rhythms were not associated with defibrillator use and vice versa (Table 5).

Correlation between documentation latency and data quality
A short time interval between CPR initiation and documenting the incident was positively correlated to a higher soft CI. The Pearson analysis resulted in r = − 0.16 (P = 0.67), the Spearman analysis showed r = − 0.268 (P = 0.002) (Fig. 5).

Importance of reliable CPR documentation
Much of our current knowledge on CPR originates from large scientific data registries. To date, the timeconsuming manual data entry into these repositories is a drawback in many of these systems. In operating rooms, intensive care as well as out-of-hospital emergency settings, automated collection of patient vital parameters, combined with semi-automated or manual documentation of therapeutic measures and further details, in electronic PDMS is on the rise. If such a PDMS is in place, the relevant data that are needed for research purposes are readily available in electronic form. Automatic data transfer from PDMS to scientific registries, however, lacks the plausibility check that is associated with manual data handling, thus bearing the risk of poor data quality. In a nutshell, data quality of electronic routine medical data collection is paramount for its use for scientific purposes. In this study, the authors evaluated the reliability of CPR related documentation within a PDMS in three independent ICUs of a single university hospital. Special interest was put on the time of CPR onset and its duration, two relevant parameters for quality assurance and research. Both data comprehensiveness and data consistency showed shortcomings. For the quantification concerning the reliability of PDMS entries, a numeric measure, the Consentaneity Index (CI), was introduced.
As demonstrated in this work, in many cases the medical record provided conflicting information on the starting time of CPR and its duration. Thus, the exact CPR interval could not be reliably determined. This renders it difficult or even impossible, to correlate other clinical findings or therapeutic interventions to the CPR intervals for scientific purposes. Further, it seems advisable not to relay on a single data source when the CPR interval is intended to be read from the PDMS documentation (e.g., when exporting PDMS data to scientific registries). Whenever intervals of running CPR are to be identified from routine medical records, the authors propose to choose time intervals with a soft CI of 1, i.e., all CPR mentioning data sources are in positive accordance.
Another purpose for data use relying on accurate time-related CPR documentation is process quality management. For example, in-hospital VF should be defibrillated within less than 3 min [16]. Therefore, the latency between circulatory arrest and the first defibrillation is part of the Utstein template for CPR documentation. The Utstein authors rate the importance of this parameter as follows: "In the hospital, the time interval from collapse/arrest to first defibrillation attempt may be the most important process indicator of effective response when VF is the initial cardiac arrest rhythm" [17]. If, as found in this study, individual data on CPR starting time differs in every second case by more than 10 min, its use for benchmarking the three-minute-defibrillation requirement seems questionable.

CPR documentation quality in the literature
In the literature, there are numerous studies dealing with the quality of CPR documentation in different settings. Most of these studies focus on the completeness of CPR data in the medical record [3,4,[6][7][8][9]. A few studies address data consistency: Frisch et al. evaluated differences in time discrepancies between the medical record and time-stamped audio recordings in out-of-hospital cardiac arrest situations [1]. A combined registry analysis and single center chart review revealed implausible (negative or zero) resuscitation time intervals in inhospital cardiac arrest [2]. Two studies suggest, the CPR documentation quality can be ameliorated: an intervention study involving a CPR training session showed, that both CPR documentation as well as compliance with CPR guidelines improved [8]. In a pediatric setting, the introduction of weekly resuscitation case discussions did improve both CPR guideline coherence and CPR documentation [9]. However, none of these studies is dealing explicitly with data quality of CPR recordings in the ICU setting. The PDMS used in this study allows the entry of CPR related data in several data forms. While these options are facilitating daily recording, it allows the medical record to be fragmentary and even contradictory.

Data completeness
Despite the fact that an up-to-date PDMS was in use on the three ICUs in this study, there were lacks in data completeness. In 18% of reviewed cases, either CPR starting time or duration were not available from the electronic record (Table 4). Only in a minority of cases (20%), information on the individual CPR could be found in all four examined data sources.

Data consistency
The analysis of data consistency showed contradictions within individual cases concerning the starting point and the CPR duration. In addition, there were some cases with inconsistencies between the indication for electrical defibrillation, as read from the heart rhythm documentation, and the documented use of the defibrillator.
Data consistency was more favorable if the CPR documentation was completed shortly after the event. Longer latencies between resuscitation and data entry were associated with lower values of the soft CI.

Improving CPR documentation
The results of this work suggest that electronic CPR documentation in the ICU setting needs improvement. This purpose could be addressed by technical and nontechnical measures. From a technical point of view, the user friendliness of CPR documentation can be improved by designing smart CPR event forms for instant (ideally real-time) use. An integration of those event forms in the physicians' and nurses' log could avoid data gaps and contradictions. Non-technical approaches consist particularly of the attempt to strengthen the importance of documentation in CPR training and team organization. Anecdotal experience suggest that CPR documentation works best, if one team member is taking over this task without other responsibilities during the CPR process. However, fulfilling medical necessities remains the number one priority during resuscitation efforts, and documentation must be subordinated, if human resources are limited. The CI could be a valuable tool to monitor the effect of any improvement efforts concerning CPR documentation in the ICU.

Limitations
There are some limitations to our study. It was not possible to correlate any documentation to the factual proceedings of the CPR situations. Thus, no conclusion of the validity of the PDMS content can be made.
Some CPR events will have been missed by the study due to the search strategy applied. If the respective TISS documentation of the code was absent, it was not included into the evaluation. Since there is no reason to assume that those missing cases should differ from the ones included into the study, a selection bias seems unlikely.
Under certain constellations with implausibly long CPR episodes (e.g., according to the CPR incident form), the soft CI can take false high values. Therefore, CPR documentations lasting longer than two hours have been excluded from the analysis.