The study aimed to investigate markers of cognitive skills—situation awareness and decision making—important for team leaders in EMS. The analysis revealed 50 markers that fell under seven elements: gathering information, interpreting information, anticipating states, identifying options, implementing decisions, re-evaluating decisions, and maintaining standards. Each marker was associated with quality and safety at work.
Novelty of markers
The novelty lies in incorporating the additional cognitive element, i.e., maintaining standards and its markers, as an integral part of cognitive skills [4]. Moreover, the analysis suggests that some identified markers of cognitive skills are associated with prioritising, planning, and preparing—it is not surprising since these processes are basically cognitive. A similar approach incorporating these aspects into cognitive skills can be found in previous studies [10, 12], and the approach of separating them from cognitive skills can be found too [11, 27]. The results also yielded markers associated with communication and coordination. We included only those tightly intertwined with cognitive skills—specifics about what communication and coordination should look like were excluded as relevant for social skills.
In the data analysis, our attention was not narrowed to only markers related to the cognition of leaders, their teams, and other necessary elements of the whole sociotechnical system [19]. This is a typical approach in non-technical skills research, though not declared explicitly. In accordance with other studies [3, 20, 22], we also embraced the importance of cognition or shared understanding between leaders on one side and patients, their relatives, or significant others on the other.
All things considered, we have identified a new set of markers; however, it can be claimed that some of the markers are similar, at least in terms of label applied, to previous research in prehospital [10, 11] or hospital settings [27, 28]. This is quite understandable, due to similarities and differences in study designs and across examined settings.
Specificity of markers
In our data analysis and in reporting the results, we proceeded from general categories through less general elements to specific markers. This three-level system of genericity/specificity is common in non-technical skills research, since its purpose is to be applicable in most situations relevant to an occupation. While the categories are usually widely valid, elements and markers vary among occupations/work settings [2]. Thomas [7] pointed out that the usual level of specificity of behavioural markers is limited, and much more powerful assessment tools without difficulties in interpreting such generic behaviours in a specific situation could be created based on a highly specific set of behavioural markers. Of course, that would require creating separate highly specific sets of markers for each individual situation. Another option could be to create at least specific sets of markers for prototypical situations, e.g., for cardiac arrest resuscitation [18]. Anyway, a review has concluded that in contrast to tools designed for assessing very specific actions (checklists), tools designed for assessing overall technical and non-technical performance at a general level (global rating scales) have higher reliability, can be used across multiple tasks, and may better capture nuanced elements of expertise [29].
Observability of markers
We identified several behavioural markers that are probably observable (e.g., communicating interpretations to team members or re-assessing a patient). The issue here arises when it comes to construct validity. Can cognitive skills that are cognitive in their essence be observed and assessed accurately from outside? Scholars answer that although cognitive skills cannot be directly observable, at least they can be inferred from associated actions and communications [2, 9]. The EMS profession is distinguished by many actions and communications with multiple people on and off a scene. Nevertheless, there is still a certain degree of doubt regarding how much of the external behaviour offers an insight into cognitive skills [30] and to what extent subtle behaviour is easily noticeable [27].
Furthermore, we also identified markers that may be rather unobservable (e.g., anticipating a possible course of events or considering advantages and disadvantages of solutions). They represent internal mental activities deemed to be important for EMS according to participants. In contrast with the principle of excluding them from observer-based assessment tools [9], we propose assessing markers and cognitive skills in general by multiple methods. Besides the mentioned, there are alternative methods that are based on the subjective assessment of own skills and the administration of skill-related queries during or after a simulation of a task. As each of the methods has multiple pros and cons [30], combining methods can increase the reliability and validity of assessments.
Applicability of markers
When it comes to settings, our findings are applicable to EMS team leaders working in ground ambulances in urban and rural/remote areas in Slovakia. The comparison of these findings with findings from diverse prehospital and hospital settings in other countries [10, 11, 27, 28] implies partial relevance or overlapping of the identified markers also with other settings.
During the interviews, participants applied their descriptions particularly to patients’ medical or physical problems, which was stimulated by the interview schedule itself, accentuating cognitive difficulty. Therefore, the markers identified here may apply more to such situations than routine ones. Additionally, participants suggested that besides situational factors, leaders’ behaviours towards other team members depend also on team factors, such as familiarity, experience, knowledge, uncertainty, or preferences of team members. Particular behaviours towards them can be viewed as examples of explicit and implicit coordination [31].
This study focused on the leaders’ cognitive skills, and thus the applicability of these markers is self-evident. Whether these markers can also be applied to the followers’ cognitive skills is questionable. However, occasional switches between team roles [32, 33] indicate that there could be cognitive skills’ markers from our results appropriate to both leaders and followers in EMS teams.
Our research sample consisted of EMS physician leaders and paramedic leaders. The findings apply to both—this not surprising for three reasons. First, they talked about the same markers of non-technical cognitive skills. Second, in Slovakia, they work in the same prehospital emergency setting and encounter more or less similar situations, although physician-led teams are usually delegated to more serious cases than paramedic-led teams. Third, non-technical skills are conceptualised as general skills, so their applicability is wider. However, the two kinds of professionals likely differed in medical and technical skills, conceptualised as specific skills, analysis of which was beyond our scope.
From all these points, it follows that the applicability of all markers under any circumstances has its limits, and hence further scrutiny is vital. This issue has already been pointed to in previous research on markers, e.g., by incorporating a “not applicable” option in the assessment tools [10, 11].
Limitations
There are six potential limitations. First, our results apply to the Slovak EMS setting, so their applicability to EMS settings in other countries should be viewed with caution. Second, our study provides findings on markers of cognitive skills that are important from EMS team leaders’ perspective, therefore, we cannot comment on which markers of cognitive skills other team members, expert panels, or patients consider important. Third, we cannot say for sure which of the identified markers are definitely observable or unobservable, since this was not the study’s aim. Fourth, our results can be limited due to individual participants’ ability to accurately recollect specifics about past situations and to make generalisations, as required by the two-part interview schedule. Fifth, although the first author’s greater knowledge base about cognitive skills in EMS helped in navigating the whole research process, alongside this there was a risk of bias due to possible preconceptions. However, the second author, who was less knowledgeable in the topic of the research, functioned as a counterbalance. Sixth, findings from qualitative research may be incorrectly regarded as lacking generalisability. In fact, they are conceptually generalisable, indicating that the identified markers can be used for understanding cognitive skills in emergency medicine in general.