Skip to main content

Usability of novel major TraumaApp for digital data collection

Abstract

Background

Delivery of major trauma care is complex and often fast paced. Clear and comprehensive documentation is paramount to support effective communication during complex clinical care episodes, and to allow collection of data for audit, research and continuous improvement. Clinical events are typically recorded on paper-based records that are developed for individual centres or systems. As one of the priorities laid out by the Scottish Trauma Network project was to develop an electronic data collection system, the TraumaApp was created as a data collection tool for major trauma that could be adopted worldwide.

Methods

The study was performed as a service evaluation based at the Queen Elizabeth University Hospital Emergency Department. Fifty staff members were recruited in pairs and listened to five paired major trauma standby and handover recordings. Participants were randomised to input data to the TraumaApp and one into the existing paper proforma. The time taken to input data add into was measured, along with time for clarifications and any errors made. Those using the app completed a System Usability Score.

Results

No statistically significant difference was demonstrated between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05). Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma. The mean System Usability score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer).

Conclusion

Digital real-time recording of clinical events using a tool such as the TraumaApp is comparable to completion of paper proforma. The System Usability Score for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessments and the refined user interface. This would benefit from further research, exploring data completeness and a potential mixed methods approach to explore training requirements for use of the TraumaApp.

Peer Review reports

Background

Trauma contributes to 8% of all deaths worldwide [1]. It is the leading cause of death in Western countries for those under 40 years old [2]. Inclusive trauma systems have been shown to improve outcomes in major trauma [3]. The Scottish Trauma Network (STN) was established to coordinate care between the Scottish Ambulance Service, local trauma units with the creation four newly created of four Major Trauma Centres (MTCs) [2]. Development of the network has provided the opportunity to standardise processes with the need for granular data to monitor impact of this investment on patient outcomes. Clear and comprehensive documentation is paramount to enable effective communication between teams [4], and to allow collection of data for audit, enabling systems to systematically improve clinical care [5].

The National Institute for Health and Care Excellence guidelines lay out consensus criteria for complete trauma documentation, including Standby and Handover information, A to E assessment (complete primary assessment of a trauma patient in critical order as per Advanced Trauma Life Support guidance) [6] and interventions recorded contemporaneously by a designated member of the trauma team [7].

Similar to the majority of centres worldwide, the Queen Elizabeth University Hospital (QEUH) in Glasgow trauma case management is currently documented on a paper proforma. While documents across trauma systems capture similar key data there is considerable variation and compliance with completion is limited [8]. One of the priorities laid out by the Scottish Trauma Network (STN0) in 2019/20 was to develop an electronic data collection system, allowing easier and more accurate data collection [5].

Electronic documentation has consistently been shown to improve the quality and utility of documentation by reducing errors, omissions [9, 10] and length of stay in the Emergency Department (ED) [11, 12]. Baumann et al. performed a systematic review which showed that, initially, documentation time was slower following implementation of an electronic documentation system. However, as familiarity increased the documentation time decreased again [13]. The TraumaApp was developed as a collaboration between the EmQuire research group, Scottish Trauma Network and Daysix, a digital transformation company. The work was funded by InnovateUK (Project 104,540) to create a data collection tool for major trauma that could be adopted worldwide. During the two-year project, extensive co-design involving a range of trauma clinicians enabled rapid wireframing and prototyping to refine the tool. The data dictionary was based on current Scottish Trauma Audit Group and Trauma Audit and Research Network (TARN) data reporting and was aligned to international reporting standards [5, 14, 15].

Methods

The aim was to compare the feasibility of using a digital data collection tool (TraumaApp) against the paper trauma proforma used in the ED at the QEUH specifically,

  • The time taken to input information and the number of mistakes or clarifications required.

  • Assess usability of the TraumaApp

Secondary aims included assessment of improvement in digital tool use over time, suggesting familiarity, and gaining qualitative feedback on app use.

The study was performed as a service evaluation based at the Queen Elizabeth University Hospital ED. This tertiary ED sees approximately 110,000 > 16-year-old presentations per year and will be officially designated a MTC in August, 2021. The department is currently staffed by 30 consultants, 55 training grade doctors and 122 nurses. All staff types may be expected to use the TraumaApp including new roles such as trauma co-ordinators.

Fifty ED Staff members were recruited in pairs (Table 1). They were consented and, in each pair, one participant entered data into the TraumaApp and one into the existing paper proforma. Participants listened to five paired Standby and Handover recordings (Additional file 1) in sequence, with the cases in random order (Table 2) using a random number generator. Staff were familiar with the paper documentation from previous clinical exposure and departmental educational sessions. Participants were timed to assess how long it took to input the data after each recording was completed. Additionally, the time taken for clarifications was also recorded. Finally, the proforma and the app were assessed against a “template” and any mistakes and omissions were counted.

Table 1 Participant Demographics
Table 2 Order in which cases were played

Prior to assessment, the participants who used the TraumaApp were given five minutes to familiarise themselves with the ‘Standby’ and ‘Handover’ screens (Fig. 1) on departmental iPads®. Following completion of the five cases they were asked to complete a System Usability Scale (SUS) [16]. This 10-question scale was selected as the most used and reliable measure of usability [12]. Free text feedback was also collected for both the app and the paper proforma.

Fig. 1
figure 1

Image 1a and 1b—Standby and Handover screens from TraumaApp

Data were collected on an Apple iPad® using development version 1.0.6 of the TraumaApp based on iOS (DaySix Edinburgh, UK). Data were analysed by un-paired t-test using Microsoft Excel (Version 2103 Office 365).

Results

Demographics

Of 50 participants, two were called away during the initial case recording and were excluded from the results. Of the 48 included participants, there were 30 Junior Doctors (62.5%), seven Consultants (14.6%), seven Nurses (14.6%), two Major Trauma Coordinators, one Acute Care Nurse Practitioner and one Student Nurse. Table 1 describes the characteristics of the participants including their familiarity with Apple operating system and personal use of digital device.

Primary analysis

Time taken to input data for each recording is displayed in Fig. 2 for digital versus paper documentation. The length of the recordings is also shown.

Fig. 2
figure 2

Mean time taken to input data (seconds)

There was no statistically significant difference between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05)(Table 3).

Table 3 Mean time taken to input data and p-values (rounded to 2dp)

Figure 3 shows data for mean time needed for clarifications after listening to the recordings, together with the mean number of errors noted.

Fig. 3
figure 3

Time to clarifications (seconds) and number of errors

Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma.

SUS Score

All participants using the TraumaApp (n = 24) completed a System Usability Score (SUS) questionnaire after completing the data entry. The mean SUS score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer), as shown in Fig. 4.

Fig. 4
figure 4

SUS scores

Improvement in app use/ familiarity over time

The recorded data input times were standardised as percentages of recording duration and then re-ordered in the sequence that they were played to the participants. The mean values were compared in groups Digital and Paper, as shown in Fig. 5. There was no significant decrease in time taken over repeated use of the app or the paper proforma.

Fig. 5
figure 5

Percentage time taken to input data, clarification time and number of errors from first to final recording played.

Discussion

Early identification and management of injuries following major trauma is critical in reducing morbidity and mortality [7]. This relies on clear communication, synchronous assessment and point of care investigations. Accurate documentation allows the clinical team to have a shared understanding of the case and enhances team situational awareness, to provide effective, safe care [17]. Contemporaneous documentation is often complicated by the rapid and high volume of information-rich data in often busy chaotic environment. This is most notable during standby calls, where documented information allows for appropriate trauma call preparation, and handover at arrival to hospital, where patient condition and pre-hospital treatment are relayed to direct ongoing management [18].

This study shows equivalence between digital and paper documentation time of standby and handover. The results show that both digital data collection and use of paper proforma result in some loss of information and errors during Standby and Handover, which has been identified in previous studies [19]. The significantly higher clarification time and volume of errors in the digital cohort seen in Case 1 was deemed to be due to a high volume of information to be entered. Further work is required around training to enhance familiarity and to optimise the user interface to facilitate data entry. The System Usability Scale was developed in the 1980s to allow objective assessment of perceived usability [20]. A mean score of 68 has been internationally validated as the standard of acceptable usability [12], and the 25 participants who completed the questionnaire gave a mean score of 75.

A secondary objective to examine familiarity to data recording over time was more difficult to assess, due to the differing lengths and complexities of the recorded cases. The case orders were randomised to combat this, and the time taken to record data was calculated as a percentage of each case’s recording length, as shown in Fig. 5a and b. A general downward trend over time, seen in Fig. 5a and b, suggests development of familiarity in both digital and paper documentation. No true assessment could be made about improvement in use of the Trauma App over time, as each participant only used it for 20–25 min. Previous studies of digital system use suggests that development of familiarity with the icons and images used in an application is required to improve operating efficiency [21]. This indicates that users may benefit from instruction on the TraumaApp but the addition of iconography would potentially increase the training demand. Twenty of the 24 app users reported that they became more familiar with the app as they used it, although this is not supported by the data in Fig. 5.

Limitations

This was a single-centre study involving a small cohort of participants. An increased number of participants may have given more power to allow identification of significant differences between digital and paper documentation. The TraumaApp continues to be refined prior to live release, so data may not be truly representative of the version used in practice. Demographic of participants is not representative of those who are likely to be scribing for trauma cases.

Conclusion

No significant difference was identified overall in the utility of digital documentation using the TraumaApp compared to the existing paper proforma in this study. The System Usability Scale for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessment. This would benefit from further research, exploring data completeness and a potential mixed methods approach to gain further user feedback and assessment of performance and utility.

Availability of data and materials

The datasets generated and/ or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

STN:

Scottish Trauma Network

MTCs:

Major Trauma Centres

QEUH:

Queen Elizabeth University Hospital

STAG:

Scottish Trauma Audit Group

ED:

Emergency Department

SUS:

System Usability Scale

References

  1. World Health Organisation. Injuries and Violence. [Internet] 2021 [cited 19/05/2021] Available from https://www.who.int/news-room/fact-sheets/detail/injuries-and-violence

  2. Scottish Trauma Network. Introduction and background. [Internet] 2021 [cited 19/05/21] Available from https://www.scottishtraumanetwork.com/about/

  3. Utter GH, Maier RV. Inclusive trauma systems: do they improve triage or outcomes of the severely injured? J Trauma. 2006;60(3):529–37.

    Article  Google Scholar 

  4. Collins SA, Cato K, Albers D, et al. Relationship between nursing documentation and patients’ mortality. Am J Crit Care. 2013;22(4):306–13.

    Article  Google Scholar 

  5. Scottish Trauma Audit Group. STAG Priorities [Internet] 2020 [Cited 31/3/2021.] Available from https://www.stag.scot.nhs.uk/new.htm.

  6. van Olden GDJ, Meeuwis JD, Bolhuis HW, Boxma H, Goris RJA. Clinical impact of advanced trauma life support. Am J Emerg Med. 2004;22:522–5.

    Article  Google Scholar 

  7. National Clinical Guideline Centre. Major trauma: assessment and initial management. 2016. https://www.nice.org.uk/guidance/ng39/evidence/full-guideline-2308122833 Accessed 19/05/21

  8. O’Connor TL, Raposo AE, Heller-Wescott T. Improving trauma documentation in the emergency department. J Trauma Nurs. 2014;21(5):238–43.

    Article  Google Scholar 

  9. Coffey CA, Wurster LA, Groner J, Hoffman J, Hendren V, Nuss K, Haley K, Gerberick J, Malehorn B, Covert J. A comparison of paper documentation to electronic documentation for trauma resuscitations at a level I paediatric trauma center. J Emerg Nurs. 2015;41(1):52–6.

    Article  Google Scholar 

  10. Bilyeu P, Eastes L. Use of the electronic medical record for trauma resuscitations: how does this impact documentation completeness? J Trauma Nurs. 2013;20(3):166–8.

    Article  Google Scholar 

  11. Zikos D, Diomidous M, Mpletsa V. The effect of an electronic documentation system on the trauma patient;s length of stay in an emergency department. J Emerg Nurs. 2014;40(5):469–75.

    Article  Google Scholar 

  12. Bloom BM, Pott J, Thomas S, Gaunt DR, Hughes TC. Usability of electronic health record systems in UK EDs. Emerg Med J Epub. doi:10.1136/ emermed-2020–210401 Available from https://emj.bmj.com/content/38/6/410

  13. Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation time: a systematic review. Health Pol. 2018;122(8):827–36.

    Article  Google Scholar 

  14. The Trauma Audit and Research Network. Resources. [Internet] 2021[cited 19/05/21] Available from https://www.tarn.ac.uk/Content.aspx?ca=4

  15. Palmer CS, Davey TM, et al. Standardised Trauma Monitoring: The development of a minimum dataset for trauma registries in Australia and New Zealand. Injury Int J Care Injured. 2013;44:834–41.

    Article  Google Scholar 

  16. J, Brooke. SUS: A "quick and dirty" usability scale. Usability evaluation in industry. s.l. : CRC Press, 1996.

  17. Lowe DJ, Ireland AJ, Ross A, et al. Exploring situational awareness in emergency medicine: developing a shared mental model to enhance training and assessment. Postgraduate Medical Journal. 2016;92:653–8.

    Article  Google Scholar 

  18. Evans SM, Murray A, Patrick I, Fitzgerald M, Smith S, Cameron P. Clinical handover in the trauma setting: a qualitative study of paramedics and trauma team members. Qual Saf Health Care. 2010;19: e57. https://doi.org/10.1136/qshc.2009.039073.

    Article  PubMed  Google Scholar 

  19. Carter AJE, Davis KA, Evans LE, Cone DC. Information loss in emergency medical services handover of trauma patients. Prehosp Emerg Care. 2009;13(3):280–5.

    Article  Google Scholar 

  20. Lewis JR. The system usability scale: past, present, and future. International journal of human-computer interaction. 2018;34(7):577–90.

    Article  Google Scholar 

  21. Shen Z, Zhang L, Xiao X, et al. Icon familiarity affects the performance of complex cognitive tasks. I-Perception. 2020;11(2):1–18.

    Article  Google Scholar 

Download references

Acknowledgements

Dr Alexandra Mather and Dr Joe Campbell for support in the early development of the App. The WoS Innovation hub for supporting the project team developing the app. The Scottish Trauma Network and Digital Health Scotland for clinical and technical expertise from inception to delivery. InnovateUK for providing grant funding Project 104540.

Funding

Funded by InnovateUK Project 104540.

Author information

Authors and Affiliations

Authors

Contributions

DJL and JB conceived and designed the study. JB, EW and LL carried out participant recruitment and conducted the study. JB and EW performed data analysis and interpretation. JB drafted the manuscript and all authors provided editorial comments. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to David J. Lowe.

Ethics declarations

Ethics approval and consent to participate

There was no requirement for ethics approval and each participant was given a copy of the study protocol and verbally consented.

Consent for publication

Not applicable.

Competing Interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Transcripts of five cases.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Butler, J., Wright, E., Longbottom, L. et al. Usability of novel major TraumaApp for digital data collection. BMC Emerg Med 22, 39 (2022). https://doi.org/10.1186/s12873-022-00578-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12873-022-00578-9

Keywords

  • Trauma
  • Digital
  • Data collection
  • System usability score
  • TraumaApp
  • Clinical decision support