Non-technical skills (NTS) are important for the proper functioning of emergency medical ambulance crews but have hardly been researched in the conditions of clinical pre-hospital care.
The primary objective of this study, therefore, is to describe the use of NTS in practice. The secondary objective is to compare if the performance of NTS varies according to the type of case.
In this multicentric observational study the modified Team Emergency Assessment Measure (TEAM) score was used to assess the performed NTS of two or more crews on site. The evaluation consisted of leadership, teamwork and task management, rated by a field supervisor.
The study observations took place in real clinical pre-hospital emergency medical cases when two or more crews were dispatched between October 2019 and August 2020. The sample size was determined by researchers prior to the study to at least 100 evaluated events per each of the three participating emergency medical services.
The results are presented as median and interquartile range. The internal reliability, consistency and validity of test items and results were evaluated. The Kruskal–Wallis test and the post hoc Mann-Whitney U test with Bonferroni correction were used for multiple comparisons of three groups.
A total of 359 events were evaluated. Surprisingly, the median value for all eight items was as high as 3.0 with a similar interquartile range of 1.0. There were no differences observed by case type (CPR vs. TRAUMA vs. MEDICAL) except from item 1. A post hoc analysis revealed that this difference is in favour of a higher rated performance of non-technical skills in CPR.
The overall result of the performance of non-technical skills can be regarded as very good and can serve for further evaluations. The crews achieved better parameters of NTS in leadership in resuscitation situations in comparison with general medical events.
The study is registered at Clinical Trials under the ID: NCT04503369.
Training of non-technical skills (NTS), including leadership and team training, is used to improve outcomes of cardiopulmonary resuscitation (CPR) . NTS consists of several aspects: (i) teamwork – the work of the team consists of information exchange and coordination of all activities performed, in-team communication is always calm and assertive, and team members support each other; (ii) task management – with the right planning and preparation of individual activities, the likelihood that all standards of care and best practices which will follow increase; (iii) situational awareness – refers primarily to the collection of information, its understanding, and on that basis, prediction of the direction in which the situation will develop; maintaining situational awareness is crucial in order to move towards the right goal; (iv) decision-making – with good situational awareness, one is able to keep in mind all information needed to make the right decision at the right time; in some cases, it is advisable to consult with other team members or to verify their consent to the proposed procedure; (v) communication – a vital tool when working in a team . There is a lack of published studies about the use of NTS in a real medical environment and no articles describing the use of NTS in a real, non-simulated emergency medical services environment .
The primary objective of this study is to determine whether Emergency Medical Services (EMS) crews in the Czech Republic are using NTS in practice and, if they do, to describe how. The secondary objective is to compare whether NTS and team performance varies according to the type of case, i.e. resuscitation (CPR), trauma or general medical events.
A prospective, observational multicentric study of NTS in clinical pre-hospital emergencies was conducted. No randomisation was applied.
The study took place in three independent organisations in three Czech regions: EMS of Prague, EMS of the Karlovy Vary Region and EMS of the Pilsen Region. The total population of these three regions is approx. 2.2 million  with approx. 250,000 emergency cases per year . The data from real clinical events was collected between October 2019 and August 2020.
All EMS cases in the Czech Republic in which pre-hospital care was provided by two or more crews (at least four crew members) and the field supervisor were available for observation. The EMS is organised by the regional government as a rendezvous system – advanced care ambulance with paramedic and emergency medical technician and rapid response vehicle with emergency medical technician (or paramedic) and physician on board. Those units can be supported by a field supervisor in all three regions.
Only those cases where two or more ambulance units/crews met with the presence of the field supervisor were observed and included in the study.
In all eligible events, the date and time, the number of crews working in the field (two, three or more), the type of event (CPR, Trauma and Medical, i.e. general medicine emergency events), the name of the field supervisor evaluating the event and the location of the EMS were recorded (Fig. 1).
All emergency cases with only one crew or events without the presence of the field supervisor were excluded, as well as all cases without complete forms.
Adaptation of the team emergency assessment measure
The modified and simplified Team Emergency Assessment Measure questionnaire (TEAM) was used in this study (Fig. 1). The creation and validation of this tool is described elsewhere and its validation was not part of this study [6,7,8,9,10,11]. The TEAM was modified for this study as follows: items one through six, i.e. (1) the team leader let the team know what was expected of them through direction and command, (2) the team leader maintained a global perspective, (3) the team communicated effectively, (4) the team worked together to complete tasks in a timely manner, (5) the team acted with composure and control and (6) team morale was positive, all remained unchanged. Item 7 (the team adapted to changing situations) and item 8 (the team monitored and reassessed the situation) were merged into one item, as were items 9 (the team anticipated potential actions) and 10 (the team prioritised tasks). Item 11 (the team followed approved standards/guidelines) was not used, nor was item 12 (the global score) to simplify the field evaluation in the pre-hospital setting. Moreover this modification has been used in the past to evaluate NTS in simulated scenarios , and was therefore well known to researchers and field supervisors.
Each of eight TEAM items were rated using a five-point scale (range 0–4; 0 never / hardly ever, 1 seldom, 2 about as often as not, 3 often, 4 always / nearly always) and covered three categories – leadership, teamwork and task management – the same way as the original TEAM . The total score was calculated as the sum of the values of the eight items and used for further statistical interpretations.
Twenty field supervisors (EMS of Prague, n = 5; EMS of the Karlovy Vary Region, n = 6; EMS of the Pilsen Region, n = 9) underwent a standardised e-learning course on the use of the modified TEAM score before the study began. At the end of the e-learning course, each participating field supervisor had to evaluate video recordings of two simulated clinical scenarios (numbered one and two) with actors for further evaluation of inter-rater variability of field supervisors assessments.
This study compared the results of observations between subgroups of cardiopulmonary resuscitation (CPR – defined by the occurrence of cardiac arrest with ongoing CPR on scene), traumatic (TRAUMA – defined by the occurrence of any injury) and general medical events (MEDICAL – defined by any other non-traumatic, non-CPR but general medical situations, including paediatric cases).
Baseline characteristics are reported as numbers and percentages. The results of individual items and the total score of modified TEAM questionnaires are presented as median and interquartile range. Internal reliability, consistency and validity were evaluated through inter-item correlation and Cronbach’s alpha coefficient and item to total correlation . Inter-rater reliability was assessed with intraclass correlation coefficient (ICC) of evaluation of the two different simulated scenarios (video recordings) .
The nonparametric Kruskal–Wallis test was used to compare modified TEAM scores among three presented subgroups, with p < 0.05 considered as significant. The post hoc Mann-Whitney U test with Bonferroni correction was used for multiple comparisons.
The sample size was not calculated but was determined by researchers prior to the study to at least 100 evaluated events per each participating emergency medical services.
The data were collected and basic calculations performed in Excel (Microsoft, USA). Statistical software STATISTICA 7.0 (StatSoft, USA) was used for statistical analyses and calculation. The ICC calculation software Mangold, Pascal (2018), based on Wirtz & Caspar 2002, (Germany) was used to calculate the adjusted average scores, assuming no interaction effect was present.
Reliability, consistency and validity of testing of modified TEAM score
The inter-rater variability assessed by ICC was 0.958 for e-learning scenario number 1 and 0.701 for e-learning scenario number 2. Inter-item correlation for items 1–8 varied from 0.53 to 0.78, with average inter-item correlation 0.63. Cronbach’s alpha coefficient of the final dataset was 0.93 and item-total correlations varied from 0.79–0.87.
A total of 359 events were evaluated in the study after three events were excluded due to incomplete data (Fig. 2). The case types were represented differently between the three different EMS: medical events were the most common category in the EMS of the Karlovy Vary Region, but hardly seen in the Prague EMS, and in the Pilsen Region the categories were more evenly distributed. However, these differences were no longer evident in the total number of events monitored (CPR, n = 110 vs. TRAUMA, n = 122 vs. MEDICAL, n = 127). The most frequently evaluated events were when two ambulances intervened, i.e. with four crew members on the spot (n = 317; 88%) (Table 1).
Results of the overall “modified TEAM SCORE”
Monitoring using a modified TEAM score for the whole set of events showed their high use of non-technical skills. Surprisingly, the median value for all eight items was 3.0 with a similar interquartile range. No major shortcomings or reductions in the values of any of the skills expressed in the individual items were observed. The overall modified TEAM ratings including total score are presented in detail in Table 2.
Results of the “modified TEAM score” by type of case
The presented data were very similar when comparing the modified TEAM score by case type (CPR vs. TRAUMA vs. MEDICAL). The Only significant difference was observed in item 1 (The team leader let the team know what was expected of them through direction and command) H (2; 359) = 7.64, p = 0.02. A post hoc analysis revealed that this difference is in favour of a higher rated performance of non-technical skills in CPR than in MEDICAL events: U (NCPR = 110, NMEDICAL = 127) = 5627, Z = − 257,990, p = 0.0099, and none of the other pairwise comparisons were significant after Bonferroni adjustment (all p > 0.17). Similarly, despite tendency to better results in item 4 (The team worked together to complete tasks in a timely manner; p = 0.06) and item 5 (The team acted with composure and control; p = 0.09) this result did not reach statistical significance and therefore no post hoc multiple comparisons analysis was performed. There were no differences observed among other items and total score (Table 3).
The results showed high use of the NTS on an average level and there was no difference observed among the subgroups of different medical conditions.
So far, similar results were published only from the hospital environment or simulations or clinical situations with a focus on one part of the non-technical skills or during events of out-of-hospital cardiac arrest only [6, 15,16,17]. The presented data are comparable in terms of consistency and reliability, but also in terms of the values of the results with the original work, and sufficient data consistency and interrater variability was observed, calculated and published [7,8,9,10,11, 16, 18]. The research team used the complete TEAM form for pilot field observations . However, based on the pilot findings, the researchers used a simplified form for field supervisors to work in pre-hospital care. In this study we evaluated the situation on the spot, not later, for example from a video recording .
This baseline characteristic of NTS obtained from three independent emergency medical services organisations from different regions in the Czech Republic (from the urban region of the capital, rural and mountainous areas, and from a region where urban and rural characteristics of pre-hospital care are combined) provides awareness of the starting level of staff who have not yet been trained in non-technical skills. The specific reason for the study was not to monitor the clinical performance of the provided care, but to focus on NTS. We did not focus on the correlation between non-technical skills performance and clinical outcome.
From the outset, there was no intention to compare individual EMS but to create a suitable case mix, which could correspond with the global spectrum in pre-hospital emergency care in the Czech Republic. The best results were observed especially in CPR, mainly due to significantly better leadership. CPR is “strictly” algorithm-driven, and thus it might be easier for the team leader to instruct the crew members and for team members to follow. Our results that are completely different from those of a recently published study of 114 cases of hospital cardiac arrest, where the leadership was evaluated as the worst . This situation can be partly explained by the fact that the most common training, where aspects of NTS and leadership are also explained, are the Advanced Life Support courses of the European Resuscitation Council, which are widespread in the Czech Republic. This result may also indicate that the focus on the standardised approach is also needed in other fields like trauma and general medical care. Except for CPR, other events observed in the study had very different clinical characteristics and the main goal was to evaluate the aspects of NTS in the interplay of multiple field teams rather than the overall competence. However, the authors are aware that the evidence indicates that checklists tend to overlook the more holistic components of clinical competence, suggesting that global ratings of performance are appropriate [11, 19, 20]. In this context, there is the question of how the scale is set in the validated TEAM score and whether differences in team performance or algorithm application are being revealed. So far, most of the works for individual items report an average score or median around 3 and it is very rarely published whether the level of NTS measured by TEAM score correlates with clinical outcome. In this study, the non-technical skills scale for trauma (T-NOTECHS) [21, 22] or other teamwork assessment measures was not used despite NTS being evaluated in the subgroup of trauma patients. This was mainly to have only one tool for evaluation and the TEAM tool is also highly effective for assessing trauma .
When adapting the TEAM forms for this study, the researchers merged some TEAM items and omitted the global score as set by Cooper et al. mainly to simplify the evaluation and focus purely on individual aspects of NTS in real clinical field work in a clinically dynamic environment.
As TEAM can be regarded as a proven method, only a data consistency check was performed prior to statistical evaluation. Interrater variability was determined by evaluating the observer’s video scenarios and not to a certain proportion of actual field cases, where 10% of cases were considered normal and was done in the original validation study . This was not feasible in a multicentric field study due to logistical reasons. The relatively high number of field supervisors (n = 20) could be regarded as a limitation, but their training in TEAM evaluation was extensive and resulted in low interrater variability, which had been checked and proved [11, 23].
A major limitation of this study is that no sample calculation was performed prior to the start of the study. It was only decided to obtain 100 measurements from each participating EMS. This shortcoming in the performed power analysis coupled with the above-mentioned ambiguities in the stratification of the TEAM score scale may cause some uncertainty in the presentation of the results. For now, it is necessary to take these results as a pilot and to follow up the research of other issues such as the correlation of NTS and clinical outcomes, learning effects or standardisation of algorithms.
The overall result of non-technical skills performance can be regarded as very good and can serve for further evaluations. There was no significant difference observed according to the medical conditions of the events except for better leadership in CPR.
Availability of data and materials
All data generated or analysed during this study are included in this published article.
The baseline dataset used and analysed during the current study is available from the corresponding author on reasonable request.
Team Emergency Assessment Measure questionnaire
Emergency Medical Services
Intraclass correlation coefficient
Perkins GD, Graesner JT, Semeraro F, et al. European Resuscitation Council Guideline Collaborators. European Resuscitation Council Guidelines 2021: Executive summary. Resuscitation. 2021;161:1–60. https://doi.org/10.1016/j.resuscitation.2021.02.003 Erratum in: Resuscitation. 2021; 163:97–98.
Association of Emergency Medical Services of the Czech Republic. Statistics on Field Activity of the Czech Emergency Medical Services for the Year 2019 [Online].; 2020. https://www.azzs.cz/data//web/dokumenty/Vybrané ukazatele ZZS/Statistika výjezdové činnosti/AZZS_Statistika-vjezov-innosti-ZZS-R-2019.pdf.
Freytag J, Stroben F, Hautz WE, Schauber SK, Kämmer JE. Rating the quality of teamwork - a comparison of novice and expert ratings using the TEAM emergency assessment measure (TEAM) in simulated emergencies. Scand J trauma Resusc Emerg Med. 2019;27(1). https://doi.org/10.1186/s13049-019-0591-9.
Cant RP, Porter JE, Cooper SJ, Roberts K, Wilson I, Gartside C. Improving the non-technical skills of hospital medical emergency teams: the TEAM emergency assessment measure (TEAM™). EMA - Emerg Med Australas. 2016;28(6):641–6. https://doi.org/10.1111/1742-6723.12643.
Maignan M, Koch FX, Chaix J, et al. TEAM emergency assessment measure (TEAM) for the assessment of non-technical skills during resuscitation: validation of the french version. Resuscitation. 2016;101:115–20. https://doi.org/10.1016/j.resuscitation.2015.11.024.
Posner KL, Sampson PD, Caplan RA, Ward RJ, Cheney FW. Measuring interrater reliability among multiple raters: an example of methods for nominal data. Stat Med. 1990;9(9):1103–15. https://doi.org/10.1002/sim.4780090917.
Karlgren K, Dahlström A, Birkestam A, et al. The TEAM instrument for measuring emergency team performance: validation of the Swedish version at two emergency departments. Scand J Trauma Resusc Emerg Med. 2021;29(1):139. https://doi.org/10.1186/s13049-021-00952-9.
Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global ratings scales for assessing performance on an OSCE-format examination. Acad Med. 1998;73(9):993–7. https://doi.org/10.1097/00001888-199809000-00020.
Husebø SE, Silvennoinen M, Rosqvist E, Masiello I. Status of Nordic research on simulation-based learning in healthcare: an integrative review. Adv Simul. 2018;3(1). https://doi.org/10.1186/s41077-018-0071-8.
The study was approved by the ethics committee of the EMS of the Karlovy Vary Region under Approval Number: 374/19/ZZSKVK on 7 January 2019, before the study started. All participating field supervisors consented to the collection of data from the field as an extension of their routine work on a voluntary basis. The crew members were not aware of the study and the purpose and type of data collection by the field supervisors. Observations were made in accordance with the laws of the Czech Republic. No patient rights were affected, all observations were anonymised, and no patient or employee data were collected at any stage of the study. No signed informed consents were required.
The study is registered at Clinical Trials under the ID: NCT04503369.
Consent for publication
The authors declare no conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Peran, D., Sykora, R., Vidunova, J. et al. Non-technical skills in pre-hospital care in the Czech Republic: a prospective multicentric observational study (NTS study).
BMC Emerg Med22, 83 (2022). https://doi.org/10.1186/s12873-022-00642-4