- Study protocol
- Open Access
Testing a videogame intervention to recalibrate physician heuristics in trauma triage: study protocol for a randomized controlled trial
BMC Emergency Medicine volume 16, Article number: 44 (2016)
Between 30 and 40 % of patients with severe injuries receive treatment at non-trauma centers (under-triage), largely because of physician decision making. Existing interventions to improve triage by physicians ignore the role that intuition (heuristics) plays in these decisions. One such heuristic is to form an initial impression based on representativeness (how typical does a patient appear of one with severe injuries). We created a video game (Night Shift) to recalibrate physician’s representativeness heuristic in trauma triage.
We developed Night Shift in collaboration with emergency medicine physicians, trauma surgeons, behavioral scientists, and game designers. Players take on the persona of Andy Jordan, an emergency medicine physician, who accepts a new job in a small town. Through a series of cases that go awry, they gain experience with the contextual cues that distinguish patients with minor and severe injuries (based on the theory of analogical encoding) and receive emotionally-laden feedback on their performance (based on the theory of narrative engagement). The planned study will compare the effect of Night Shift with that of an educational program on physician triage decisions and on physician heuristics. Psychological theory predicts that cognitive load increases reliance on heuristics, thereby increasing the under-triage rate when heuristics are poorly calibrated. We will randomize physicians (n = 366) either to play the game or to review an educational program, and will assess performance using a validated virtual simulation. The validated simulation includes both control and cognitive load conditions. We will compare rates of under-triage after exposure to the two interventions (primary outcome) and will compare the effect of cognitive load on physicians’ under-triage rates (secondary outcome). We hypothesize that: a) physicians exposed to Night Shift will have lower rates of under-triage compared to those exposed to the educational program, and b) cognitive load will not degrade triage performance among physicians exposed to Night Shift as much as it will among those exposed to the educational program.
Serious games offer a new approach to the problem of poorly-calibrated heuristics in trauma triage. The results of this trial will contribute to the understanding of physician quality improvement and the efficacy of video games as behavioral interventions.
clinicaltrials.gov; NCT02857348; August 2, 2016.
Treatment at trauma centers improves outcomes for patients with moderate-to-severe injuries [1–3]. Professional organizations, state authorities, and the federal government endorse the systematic triage and transfer of these patients to trauma centers either directly from the field or after evaluation at a non-trauma center . Nonetheless, between 30 and 40 % of patients with moderate-to-severe injuries still receive treatment at non-trauma centers (under-triage) [5–7]. The problem amplifies among the cohort evaluated by physicians at non-trauma centers—70 to 80 % of whom are under-triaged [8, 9].
Current efforts to change physician decision making in trauma triage focus primarily on education and outreach by opinion leaders . These strategies assume that the problem is one of ignorance. In other words, physicians make the wrong decision because they either lack knowledge of the clinical practice guidelines or do not believe in the validity of the guidelines. However, this approach ignores the influence of intuitive judgments (heuristics) on triage decisions. Heuristics, mental short cuts based on pattern recognition, drive most decision making. They take precedence over rule-based algorithms (analytic cognitive processes), by springing to mind spontaneously and effortlessly. They work well under conditions of time-pressure and uncertainty precisely because they ignore irrelevant information and streamline decision making. Most importantly, they generate useful answers most of the time. However, they can also draw attention to the wrong contextual cues, leading to predictable errors of judgment (biases) [11–13]. Our observational and experimental work suggests that the use of heuristics may explain some patterns of non-compliance with clinical practice guidelines in trauma triage [14–16].
Because they are automatic, the use of heuristics cannot be eradicated. Moreover, the uncertainty and complexity of the clinical environment make well-calibrated heuristics a powerful tool . We propose what we believe to be the first intervention designed to recalibrate physician heuristics. It draws on research from other domains suggesting the potential effectiveness of promoting analogical encoding and narrative engagement. Analogical encoding interventions use case studies to highlight core principles of decision making, emphasizing the identification of relevant and irrelevant contextual cues [18–21]. Narrative engagement interventions use messages designed to resonate with recipients in personally relevant and meaningful ways. Our intervention is an adventure video game that combines these two features [22, 23]. The planned study will compare the impact of the game on simulated physician trauma triage decisions with that of an educational program of equivalent duration.
We developed the video game (Night Shift) in collaboration with Schell Games (Pittsburgh, PA) through an iterative process involving behavioral scientists, cognitive psychologists, trauma surgeons, and emergency medicine physicians. We plan to compare the impact of Night Shift with that of a standard educational program on simulated triage decisions made by a convenience sample of emergency medicine physicians. We will recruit physicians working at non-trauma centers, randomize them to play Night Shift or to review a standard educational program, and assess their decision making on a validated virtual simulation.
The virtual simulation allows us to assess physician performance in terms of the number of severely injured patients not transferred to trauma centers (under-triage). We can also assess the effect of Night Shift on physician heuristics, by experimentally manipulating cognitive load in the simulation. Psychological theory predicts that cognitive load will increase reliance on heuristics, thereby increasing the under-triage rate in situations where those heuristics are inaccurate . Our target heuristic is representativeness. In the context of trauma triage, it would entail forming an initial impression based on how typical a patient appears of the severely injured.
We hypothesize that: a) physicians exposed to Night Shift will have lower rates of under-triage compared to those exposed to the educational program, and b) cognitive load will not degrade triage performance among physicians exposed to Night Shift as much as it will among those exposed to the educational program.
Using a strategy that has proven successful in the past, we will recruit physicians at a national meeting of the American College of Emergency Physicians in the fall of 2016, using a booth in the Exhibition Hall [15, 16]. Physicians will be eligible for participation if they care for adult patients in the Emergency Department of either a non-trauma center or a Level III/IV trauma center in the United States. We will obtain verbal consent from eligible physicians, informing them that the study focuses on evaluating how best to disseminate clinical practice guidelines in trauma.
Randomization and blinding
We will randomize eligible physicians to complete (a) either Night Shift or the educational intervention, myATLS and Trauma Life Support MCQ Review, and (b) either the control or cognitive load version of a virtual simulation [Fig. 1]. We will generate a randomization scheme using Stata 13.0 (Statacorp, College Station, TX, USA) statistical software, using random block sizes of 4 or 8. After registering the participant, study personnel will obtain the intervention assignment from a central database. Although we cannot blind study personnel and participants to the intervention after allocation, we will mask group assignment during the analysis phase.
Upon enrollment, subject physicians receive an iPad mini 2 with their intervention pre-loaded, along with written instructions on how to complete the study protocol. They will receive weekly reminder emails to complete the protocol for the duration of the study. We will ask participants to spend a minimum of one hour completing the intervention task. After completing it, they will be instructed to log in to a website that hosts: a) the virtual simulation; b) a questionnaire about their personal information, and c) a questionnaire about the intervention’s tolerability. Completing the simulation and two questionnaires will take approximately one hour. Participants will be able to complete the two portions of the study protocol at their convenience, within 1 month of enrollment. They will keep the iPad mini 2 as an honorarium (approximate value $260).
Intervention: night shift
The goal of Night Shift is to teach physicians to recognize patients with ‘non-representative’ severe injuries, defined as injuries that are classified by the American College of Surgeons as life-threatening or critical, but that do not fit the archetype for injuries requiring treatment at a trauma center . The intervention is designed to increase analogical encoding by having players experience a case that goes awry because of reliance on the representativeness heuristic, and providing specific feedback on relevant and irrelevant cues (i.e. ones they might have missed and ones that might have misled them). It is designed to facilitate narrative engagement by the providing the information in an emotionally compelling way. After considering different types of games (e.g. puzzles, role-playing), we decided that an adventure video game would provide an environment realistic enough for existing triage guidelines to appear valid, while including an element of fantasy to promote character identification and engagement.
Players take on the persona of Andy Jordan, a young emergency medicine physician who moves home after the disappearance of his estranged grandfather (Robert Jordan) and takes a job in the local emergency department (ED). The introduction tells players that they have two objectives. The first is to diagnose and treat patients who present to their ED. The second is to solve the mystery of Robert’s disappearance: was he murdered or has he chosen to disappear? [Fig. 2].
The patients who present to the ED have both severe injuries and non-trauma complaints. In each case, subject physicians must interview the patients to elicit relevant elements of the history and physical exam. They then have the option of: a) ordering ancillary testing, b) discussing the case with a consultant, or c) discharging the patient home. Cases end when players make disposition decisions (admit, transfer, or discharge).
We categorize cases as ‘relevant’ or ‘non-relevant’ to our objective. ‘Relevant’ patients have non-representative severe injuries. When players fail to transfer one of these patients, he/she returns with complications that require additional treatment. Players also receive feedback on their performance from peers, family members, or their supervisor. The feedback includes both information about the clinical practice guidelines and a personal reprimand. In contrast, when players successfully transfer the patient to a trauma center, they receive an update, stating that the patient has improved, information about the guidelines, and a compliment on their diagnostic and decision-making capability.
‘Non-relevant’ patients either have representative severe injuries or non-trauma complaints. We included these cases because playtesters found that the ‘relevant’ cases did not pose sufficient diagnostic challenge, making game-play tedious. Players do not receive in-game feedback on their treatment of these patients. However, we do provide a summary of their performance at the end of Night Shift, which includes their ability to diagnose and treat the ‘non-relevant’ cases.
The mystery component of Night Shift occurs concurrently with the clinical challenges, and serves to facilitate players’ identification with their character and interest in their task. Players must solve Robert’s disappearance through interactions with other characters, including patients, and their physical environment. Andy Jordan’s background and character are also revealed through these interaction, which are designed to make him and his decisions more appealing and sympathetic [Table 1].
Intervention: educational control
The gold standard educational strategy in trauma is Advanced Trauma Life Support (ATLS). The American College of Surgeons have developed ATLS, a 2-day seminar, to teach participants: to assess the patient’s condition rapidly and accurately; to resuscitate and to stabilize the patient; to determine if the patient’s needs exceeds the capabilities of their facility; to arrange for the definitive care of the patient, and to ensure that optimal care if provided. The course includes lectures on key topics (e.g. assessing the severity of the injury), instruction on skills relevant to the initial stabilization of the patient (e.g. putting in chest tubes), and practice using standardized patients. Participants must complete a multiple choice test before and after the course to receive certification .
We will use a two-part educational program as a surrogate. First, physicians review the myATLS app, designed and produced by the American College of Surgeons as an adjunct to the ATLS course . The app includes a review of each chapter of the ATLS textbook, a series of videos demonstrating common trauma procedures, and clinical resources including checklists for use at the bedside. Second, physicians use the Trauma Life Support MCQ Review, an app designed to help students prepare for the ATLS exam. It includes 550 multiple-choice questions with correct answers and explanations. We will ask physicians to review the myATLS app and then complete questions in the Trauma Life Support MCQ Review, spending at least 1 h on the combined tasks.
Data sources and management
We will install Centrify on each iPad that we distribute, an app that tracks usage of programs, to measure how long physicians spend on their assigned apps (adherence). Data from this app will download daily to a secure server hosted by the University of Pittsburgh. In addition, the Night Shift application collects data on each player’s behaviors and actions (e.g. disposition decisions) during game-play. These data will be summarized using Google Analytics and then downloaded to the server.
After completing their assigned interventions, subjects will log into a website to complete a virtual simulation designed to replicate the task environment of an Emergency Department at a non-trauma center. The primary outcome of the clinical trial will be performance on the simulation, measured as the physician’s rate of under-triage (proportion of severely injured patients not transferred to a trauma center). We previously collaborated with a gaming company (Breakaway Ltd; Hunt Valley, MD) to develop a 2-D simulation to assess physician decision making in trauma triage [Fig. 3]. The simulation has both internal reliability as well as construct validity . Most importantly, in prior studies, we found that physicians make similar decisions for trauma patients on the simulation as they do in real-life. Performance on the virtual simulation will be transmitted from the website to a secure server hosted by the University of Pittsburgh.
The simulation has ten cases: three severely injured patients (one representative, two-non-representative), three minimally injured patients, and four non-trauma cases. Users must evaluate and manage these cases over 42 min, simulating a busy 8-h ED shift. New patients arrive at pre-specified (but unpredictable) intervals, so that physicians must manage multiple patients concurrently. Each case includes a 2-D rendering of the patient, a chief complaint, vital signs which update every 30 s, a history, and a written description of the physical exam. In the absence of clinical intervention by the player, severely injured patients and critically ill distractor patients decompensate and die over the course of the simulation.
Physicians manage patients by selecting from a pre-specified list of 250 medications, studies, and procedures. Some orders affect patients’ clinical status, leading to corresponding changes in their vital signs and physical exam. Other orders generate additional information, presented as reports added to the patients’ charts. The cases end when physicians either make a disposition decision (admit, discharge, transfer) or the patient dies.
There are two versions of the simulation: a control and a cognitive load condition. As mentioned earlier, cognitive load is expected to increase reliance on heuristics . When poorly calibrated, these heuristics result in predictable errors. By varying cognitive load, we can assess the effect of the interventions on physicians’ heuristics (a secondary outcome measure). Specifically, a comparison of the difference in performance under control and cognitive load conditions of physicians in the two arms of the study will provide insight into the mechanism by which the interventions work. If physicians exposed to Night Shift demonstrate both lower under-triage rates, and less effect of cognitive load on performance, we can infer that Night Shift functions by recalibrating heuristics. In contrast, if physicians exposed to Night Shift perform better in general but have the same rate of errors when working under cognitive load as physicians exposed to the educational module, then we can infer that Night Shift functions through some alternative mechanism.
We operationalize cognitive load in two ways. First, we vary the complexity of the non-trauma cases. In the control arm, non-trauma cases have routine complaints (e.g. appendicitis), arrive hemodynamically stable, and do not deteriorate over the course of the simulation. In the cognitive load arm, non-trauma cases are critically ill (e.g. have sepsis), arrive hemodynamically unstable, and deteriorate without adequate management. Second, we reduce the number of rooms that physicians can use to evaluate patients, from eight in the control arm to four in the cognitive load arm, which increases the number of distractors that they receive.
Questionnaire to assess personal characteristics
Each physician will complete a questionnaire querying age, sex, race, ethnicity, educational background (board certification, ATLS certification, years since completion of residency). We will also assess individual differences that might vary between the two samples, and might influence the efficacy of the different interventions. Physicians will complete the Big Five Inventory-10, a ten-item instrument that measures personality traits .
Questionnaire to assess usability and tolerability of interventions
To measure the tolerability of the interventions, we will ask physicians to provide qualitative feedback about the experience. Additionally, we will ask physicians who use Night Shift to complete the Narrative Engagement Scale developed and validated by Busselle and Bilandzic . The instrument has 12 items that measure narrative understanding, attentional focus, narrative presence, and emotional engagement. It will allow us to explore causes for Night Shift’s success or failure at changing physician decision making, in future analyses.
We will calculate the response rate as the proportion of enrolled physicians who start to use their assigned intervention, and the completion rate as the proportion who finish the virtual simulation. We will include physicians in the analysis who do not spend one hour on their assigned intervention (i.e. evaluating it in terms of intention-to-treat). We will exclude physicians who do not complete the virtual simulation (i.e., those for whom we have missing outcome data).
We will summarize physician characteristics (including personality traits) using means (standard deviations) for continuous variables and proportions (%) for categorical variables, and will compare the distribution of characteristics between the two intervention groups using Students t-test and chi-squares as appropriate.
We hypothesize that: a) physicians exposed to Night Shift will have lower rates of under-triage compared to those exposed to the educational program, and b) cognitive load will not degrade triage performance among physicians exposed to Night Shift as much as it will among those exposed to the educational program.
To test these hypotheses, we will first score participants’ performance on each trauma case based on a review of their disposition decisions, using American College of Surgeons guidelines as the gold standard . We will calculate an under-triage rate for each physician:
We will assess the influence of the intervention (game or educational program), cognitive load (absent or present), and the interaction between these factors on physicians’ under-triage rates using an ANOVA.
We will summarize the length of time that physicians in each group spend on their intervention and compare the duration using Students t-tests. Information about adherence will provide insight into the tolerability of the two interventions and will also allow for exploratory analyses into the mechanism of the interventions’ success or failure.
We will categorize qualitative feedback as positive or negative and will compare the tolerability of the two interventions. Among physicians who play Night Shift, we will also summarize responses to the Narrative Engagement Scale.
Human subjects and power calculation
We have designed the experiment to capture a moderate effect (one-half a standard deviation) of the interventions on the difference between the under-triage rates of physicians who complete the game and educational program with an alpha of 0.05 and a power of 80 % . Based on prior studies, we anticipate a 70 % response rate and will recruit 366 physicians [15, 16].
Security, ethics, and dissemination
On enrollment in the trial, participants will receive a unique identifier. They will use that identifier to login to Night Shift (if assigned to that condition) and to the website that hosts the virtual simulation and questionnaires. Only the primary investigator (PI) will have access to the linkage file connecting the identifier to the physician’s name and contact information. This file will be encrypted and stored on a secure server at the University of Pittsburgh.
Dissemination of results
Results from the study will be reported to the public through manuscripts and oral presentations at national meetings. We will provide an abstract of the findings to all participants. Access to the de-identified dataset will be made available upon written request to the study team.
The objective of this research program is to develop a video game intervention that will improve physicians’ triage decisions by engaging them in cases where generally useful heuristic decision rules lead to errors in judgment. Night Shift provides explanations of appropriate decisions, presented in an intuitive way, so as to facilitate integration with their normal thought processes. We will use a randomized controlled trial to compare the efficacy of that video game with that of a standard educational control intervention of similar duration and complexity.
The National Academy of Medicine recently issued a white paper, Improving Diagnosis in Health Care, in which it highlighted the influence of heuristics on diagnostic error . Existing interventions have had limited success at ensuring that physicians use well-calibrated heuristics . These efforts typically focus on increasing physicians’ use of rational processes either implicitly by disseminating rule-based algorithms or explicitly by promoting reflective reasoning. The latter encourages physicians to take a step back, to consider their diagnoses more carefully, and to recognize the shortcomings of judgments made based on intuition [32–34]. A few studies recommend removing the clinician altogether from the decision problem, shifting the burden of judgment to decision tools or external consultants [35, 36]. All of these proposals ignore the potentially adaptive nature of heuristics. Although heuristics can result in predictable errors, they arise because they often produce useful responses to complex questions with relatively little cognitive effort. An intervention that can improve physician heuristics would leverage the strengths and talents of the people at the center of the diagnostic challenge: physicians .
Our intervention focuses on the representativeness heuristic, whereby individuals’ initial judgment of a case reflects how typical it seems of a process. In the trauma triage setting, reliance on representativeness would lead to correct diagnoses of, say, cases involving penetrating injuries, but not cases involving seemingly more benign processes (e.g. falls for elderly individuals). The intervention uses stories designed to immerse participants in playing the role of a physician concurrently solving both clinical and personal problems. It draws on three bodies of research finding the potential power of stories to facilitate behavioral change. One research thread finds that stories facilitate processing and retaining new data, called narrative engagement . In our context, the stories are meant to help physicians integrate their simulated experience into the mental models evoked in normal clinical practice. The second body of research finds that practicing desired behaviors in a safe environment helps people to gain warranted feelings of self-efficacy, providing the confidence needed to deploy newly acquired skills . The third body of research finds that stories can engage players cognitively and emotionally in ways that transcends traditional forms of education [38, 39].
We also exploit the growing appeal and prevalence of serious game technology. Video games are a $22 billion/year industry. 155 million Americans play video games. The average gamer is 34 years old. Twenty-seven percent of players are older than 50; 44 % are female. A majority (80 %) of US households own a device to play games . Statistics do not exist on the number of physicians who play video games. However, states and professional organizations already require between 20 and 50 h a year of continuing medical education as a condition for licensure. Games could easily become part of the roster of accepted educational activities.
This study has potential limitations including its design as a two-arm trial. An alternative design would include four arms: control, game, educational module, game plus educational module. However the constraints of the budget and concerns about recruiting sufficient numbers of physicians precluded this approach. Second, we plan to use a convenience sample to test the efficacy of the game, which may not represent the population at-large. If the game does affect performance, we can then proceed with testing its effectiveness. However, if we find no effect even among motivated participants, it provides a rationale to explore other methods of promoting behavior change. Third, we have chosen an educational intervention without any proven effectiveness. However, the structure (a didactic program that informs users of the clinical practice guidelines accompanied with multiple choice questions to reinforce key decision priniciples) reflects that of ATLS, the current gold standard for continuing medical education in trauma. Furthermore, the American College of Surgeons promotes myATLS as an adjunct to ATLS, giving the intervention additional face validity.
In conclusion, physician heuristics play an important role in diagnostic error. Altering heuristics is a challenge, with no effective solution. Serious games have attributes that make them well suited to address this problem, and therefore offer a new approach. We have chosen the special case of trauma triage to test the effect of a video game intervention to recalibrate heuristics. The results of this trial will contribute to the literature on physician quality improvement and the efficacy of video games as behavioral interventions.
Trial Status: Recruiting
Analysis of variance
Advanced Trauma Life Support
Mackenzie EJ, Rivara FP, Jurkovich GJ, et al. A national evaluation of the effect of trauma-center care on mortality. NEJM. 2006;354:366–78.
Celso B, Tepas J, Langland-Orban B, et al. A systematic review and meta-analysis comparing outcome of severely injured patients treated in trauma centers following the establishment of trauma systems. J Trauma. 2006;60(2):371–8.
Demetriades D, Martin M, Salim A, et al. The effect of trauma center designation and trauma volume on outcome in specific severe injuries. Ann Surg. 2005;242(4):512–7.
Committee on Trauma—American College of Surgeons. Resources for optimal care of the injured patient 2006. Chicago: American College of Surgeons; 2006.
Macias CA, Rosengart MR, Puyana JC, et al. The effects of trauma center care, admission volume, and surgical volume on paralysis after traumatic spinal cord injury. Ann Surg. 2009;249(1):10–7.
Nathens AB, Jurkovich GJ, Mackenzie EJ, Rivara FP. A resource-based assessment of trauma care in the United States. J Trauma. 2004;56(1):173–8.
Delgado MK, Yokell MA, Staudenmayer KL, et al. Factors associated with the disposition of severely injured patients initially seen at non-trauma center emergency departments: disparities by insurance status. JAMA Surg. 2014;149:422–30.
Mohan D, Rosengart MR, Farris C, et al. Assessing the feasibility of the American College of Surgeons’ benchmarks for the triage of trauma patients. Arch Surg. 2011;146:786–92.
Gomez D, Haas B, de Mestral C, et al. Institutional and provider factors impeding access to trauma center care: an analysis of transfer practices in a regional trauma system. J Trauma. 2012;73(5):1288–93.
US Department of Health and Human Services. Model trauma system planning and evaluation. https://www.socialtext.net/acs-demo-wiki/index.cgi?regional_trauma_systems_optimal_elements_integration_and_assessment_systems_consultation_guide. Accessed 14 Feb 2006.
Kahneman D, Tversky A. Judgment under uncertainty: Heuristics and biases. In: Judgment under uncertainty: Heuristics and biases, by Daniel Kahneman, Paul Slovic and Amos Tversky. New York: Cambridge University Press; 2003. p. 3–20.
Sanfey AG, Loewenstein G, McClure SM, Cohen JD. Neuroeconomics: cross-currents in research on decision making. Trends Cogn Sci. 2006;10:108–16.
Shah AJ, Oppenheimer DM. Heuristics made easy: an effort-reduction framework. Psychol Bull. 2010;134:207–22.
Mohan D, Barnato AE, Angus DC, et al. Determinants of compliance with transfer guidelines for trauma patients: a retrospective analysis of CT scans acquired prior to transfer to a Level I trauma center. Ann Surg. 2010;251(5):946–51.
Mohan D, Rosengart MR, Farris C, et al. Sources of non-compliance with clinical practice guidelines in trauma triage: a decision science study. Implement Sci. 2012;7(1):103.
Mohan D, Angus DC, Ricketts D, et al. Assessing the validity of using serious game technology to analyze physician decision making. PLoS One. 2014;9:e105445.
Kahneman D, Klein G. Conditions for intuitive expertise: a failure to disagree. Am Psychol. 2009;64:515–26.
Milkman KL, Chugh D, Bazerman M. How can decision making be improved. Perspect Psychol Sci. 2009;4:379–83.
Idson LC, Chugh D, Bereby-Meyer Y, et al. Overcoming focusing failures in competitive environments. J Behav Decis Mak. 2004;17:159–72.
Moran S, Bereby-Meyer Y, Bazerman M. Stretching the effectiveness of analogical training in negotiations: teaching diverse principles for creating value. Negot Confl Manag Res. 2008;1:99–134.
Thompson L, Gentner D, Loewenstein J. Avoiding missed opportunities in managerial life: analogical training more powerful than individual case training. Organ Behav Hum Decis Process. 2000;82:60–75.
Miller-Day M, Hecht ML. Narrative means to preventative ends: a narrative engagement framework for designing prevention interventions. Health Commun. 2013;28:657–70.
Nabi RL, Green MC. The role of narrative’s emotional flow in promoting persuasive outcomes. Med Psychol. 2015;18:137–62.
Gillard E, van Dooren W, Schaeken W, Verschaffel L. Proportional reasoning as a heuristic-based process: time constraint and dual task considerations. Exp Psychol. 2009;56:92–9.
About Advanced Trauma Life Support. https://www.facs.org/quality%20programs/trauma/atls/about. (n.d.) Accessed 21 Aug 2016.
myATLS. http://www.myatls.com/. (n.d.) Accessed 21 Aug 2016.
Rammstedt B, John OP. A 10-item short version of the Big Five Inventory in English and German. J Res Pers. 2007;41:203–12.
Busselle R, Bilandzic H. Measuring narrative engagement. Med Psychol. 2009;12:321–47.
Cohen J. Quantitative methods in psychology: a power primer. Psychol Bull. 1992;112(1):155–9.
National Academies of Sciences, Engineering and Medicine. Improving diagnosis in health care. Washington DC: The National Academies Press; 2015.
Graber ML, Kissam S, Payne VL, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf. 2012;21:535–57.
Mamede S, Schmidt HG, Penaforte JC. Effects of reflective practice on the accuracy of medical diagnoses. Med Educ. 2008;42:468–75.
Mamede S, Schmidt HG, Rikers RM, et al. Conscious thought beats deliberation without attention in diagnostic decision making: at least when you are an expert. Psychol Res. 2010;74:586–92.
Coderre S, Mandin H, Harasym PH, et al. Diagnostic reasoning strategies and diagnostic success. Med Educ. 2003;37:695–703.
Espinosa JA, Nolan TW. Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study. BMJ. 2000;320:737–40.
Mulvaney SA, Bickman L, Guise NB, et al. A randomized effectiveness trial of a clinical informatics consult service: imact on evidence-based decision-making and knowledge implementation. J Am Med Inform Assoc. 2008;15:203–11.
Gigerenzer G, Todd PM, the ABC Research Group. Simple heuristics that make us smart. New York: Oxford University Press; 1999.
Bartsch A, Oliver MB. Making sense of entertainment: on the interplay of emotion and cognition in entertainment experience. J Med Psychol. 2011;23:12–7.
Chung S, Cheon J, Lee KW. Emotion and multimedia learning: an investigation of the effects of valence and arousal on different modalities in an instructional animation. Instr Sci. 2015;43:545–59.
Entertainment Software Association. Game Player Data. 2015. http://www.theesa.com/wp-content/uploads/2016/04/ESA-Annual-Report-2015-1.pdf. Accessed 21 Aug 2016.
We thank the many University of Pittsburgh faculty and staff who playtested prototypes of the game.
This work was supported by the National Institutes of Health through grants DP2 LM012339 (Mohan) and NHLBI-K08-HL122478 (Wallace). The funding agency reviewed the study but played no role in its design; and will play no role in the collection, analysis or interpretation of data.
Availability of data and material
Access to the de-identified dataset will be made available upon written request to the corresponding author.
Study concept and design: DM, MRR, BF, CF, DCA, DMY, DJW, AEB. Drafting of the manuscript: DM. Critical revision of the manuscript for important intellectual content: MRR, BF, CF, DCA, DMY, DJW, and AEB. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The University of Pittsburgh Institutional Review Board approved this study (PRO16070572). Given our recruitment strategy (a booth in the Exhibition Hall with limited ability to store sensitive documents), we obtained permission to waive written consent for participation. The PI will obtain verbal consent and will explain the study protocol to all physicians who agree to participate in the trial.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Mohan, D., Rosengart, M.R., Fischhoff, B. et al. Testing a videogame intervention to recalibrate physician heuristics in trauma triage: study protocol for a randomized controlled trial. BMC Emerg Med 16, 44 (2016). https://doi.org/10.1186/s12873-016-0108-z
- Trauma triage guidelines
- Physician decision making
- Diagnostic error