Study design
The present study is a mixed-method (qualitative and quantitative) study that was conducted in two phases. In the qualitative phase for (item generation), semi-structured interviews were conducted with Iranian emergency medical technicians who were selected through a purposive sampling method, and a scoping literature review was done to generate an item pool for the preparedness evaluation of EMS in terrorist attacks. In the quantitative phase (item reduction), for validity of tool face, content and construct validity, were performed; for tool reliability, the test and retest and intra-class correlation coefficient were evaluated.
Qualitative phase (Item generation)
To create the items, three major steps were taken: a) interviewing pre-hospital emergency experts using a descriptive phenomenology approach to identify and develop the concept of prehospital preparedness in terrorist incidents, b) scoping review, and c) incorporating steps 1 and 2 for item pool (Synthesis Research).
Step 1: Conduct a Qualitative Study
This qualitative study was conducted in Iran, one of the most disaster-prone countries in the world. The study population included 34 pre-hospital emergency staff and managers who had practical experience of pre-hospital emergency system preparedness in a terrorist attack and had at least one experience of terrorist attacks. Participants were chosen using a purposeful sampling method with maximum diversity. Sampling was carried out through a semi-structured interview until data saturation occurred when the researchers concluded that further interviews would fail to provide new information. Participant experts included 9 pre-hospital managers, 23 pre-hospital emergency personnel, and 2 experts in the dispatch ward of the pre-hospital emergency center. Inclusion criteria for the interview All emergency medical experts who had a role or experience in terrorist attacks experts who had material to say (such as heads of emergency medical centers, assistants in the technical and operations department of emergency medical centers, experts in the emergency operations center, and experienced operational personnel across the country). Finally, these individuals were willing to participate in the study and interviews. Exclusion criteria for the interview: people who did not have expressive points and did not want to participate in the interview.
The interviewees answered a similar set of questions which began with “have you ever experienced the disaster preparedness exercises of the health system?”, “What terrorist incident have you been involved in so far and what was your role in that incident?”, “What problems and challenges have you had in managing a terrorist incident and what were the strengths and weaknesses of the incident?” The data collected in this stage was analyzed using a phenomenological methodological approach by Colaizzi’s method. The coding process is being managed using the trial version of the MAXQDA 16 Software. This study employed strategies recommended by Lincoln and Guba for data trustworthiness [20].
Step 2: A Scoping Review
Search Strategy
We conducted a scoping review in 2020 to identify and evaluate the performance and level of preparedness of pre-hospital emergency teams in terrorist attacks around the world. For this purpose, we studied databases including ISI Web of Science, PubMed, Scopus, Science Direct, Ovid, Pro Quest, Wiley, and Google Scholar from January 1, 2000 to February 13, 2020. Using OR and AND, key words were combined and entered into database search boxes, including ((Terrorist incidents OR Terrorist accidents OR Terrorism attacks OR Terrorism attacks OR Terrorism OR Terrorists) AND (Violence OR Political OR conflict OR criminality OR war) AND (Prehospital emergency OR Prehospital emergency Care OR Emergency Health Service OR Emergency Medical Response OR First medical Responder OR Emergency MedicalAll synonyms of the key words were searched using MESH strategies.
Inclusion and exclusion criteria
The following studies were selected that met the conditions and criteria for inclusion in the study: Studies which were published in English, Studies which studied the performance of pre-hospital emergency medical services (EMSs) during a terrorist attack Studies which considered the use of force, violence, or threats to achieve political ends, studies which were published as originals or reviews. In addition, studies that meet the following exclusion criteria are excluded: studies whose full text was not accessible, Studies which described other areas of aiding during terrorist attacks, including anti-terrorist medical service rendered by military personnel, post-terrorist attack medical care rendered by the fire service, police, or boy scouts, or responses taking place in field hospitals, Studies which considered events including hacking into a government computer or cyber attacks on governmental websites, Studies which focused on other kinds of emergency responses, including Internet security, sanitation, and transportation, studies which were published as letters to the editor, commentary, case reports, case series, expert consensus, published national guidelines, or editorials.
Articles and Documents Selected
After selecting the desired studies, the requested information was collected and analyzed, and all steps were checked and performed according to the principles of the PRISMA checklist [21]. The first preliminary search was performed by two authors (SM and MA) separately; in the next step, independent reviewers (SM and MA) screened abstracts and titles for eligibility. When the reviewers felt that the abstract or title was potentially useful, full copies of the article were retrieved and considered for eligibility by both reviewers. If discrepancies occurred between reviewers, the reasons were identified and a final decision was made based on the third reviewer (HS).
Step 3: combining steps 1 and 2 (Synthesis Research)
In this step, all components and characteristics obtained in the previous two steps were combined; redundant items were removed, and similar ones were merged. Independent of the scoping review and qualitative study, new sub-categories and categories were created. Since the new categories and subcategories served as the foundation for the pool of items, they were evaluated with greater sensitivity. The final table, containing the theme, category, subcategory, and codes, expands the main preparedness evaluation as converted into items. The research team examined questions and eliminated or modified several. Finally, an initial format for the preparedness evaluation tool of pre-hospital emergency medical services was prepared, consisting of 160 questions. Subsequently, the primary questionnaire’s validity was determined. The psychometric properties of the tool were examined for face, content, and construct validity, as well as reliability. For each item, a response scale was considered based on a 3-point Likert scale.
Quantitative phase (Item reduction)
Face validity
We used qualitative and quantitative face validity, qualitative and quantitative content validity, and structural validity to validate this tool. In determining the quality of face validity, 10 participants who were more expert were selected and semi-structured interviews were conducted face to face, and the ease of completion, legibility, grammar, and the writing style of items in terms of ambiguity, level of difficulty, and fitness were examined. In determining the quantity of face validity, 10 participants assigned a value to each item using the five-point Likert scale ranging from five (quite important) to one (not important at all). Frequency (%) Importance = Impact score the impact score was considered to be greater than 1.5 [22]. Participants of both qualitative and quantitative face validity were chosen using a purposeful sampling method [23].
Content validity
For qualitative content validity to be ensured, ten health professionals experienced in terrorist attack incidences were asked to express their corrective views in terms of grammar, the use of appropriate words, proper placement of items, proper scoring and appropriateness of the selected dimensions. The questionnaire items were revised in response to experts’ suggestions. Thus, most of the items were transcribed by adding, substituting more common and understandable words, which led to the clarification of the vague items. Content validity ratio (CVR) and content validity index (CVI) were used to evaluate the quantitative content validity. To begin, the CVR was calculated and a panel of experts was asked to rate each item on a three-point scale: necessary, useful, and not necessary. In this phase, the content validity ratio was calculated using the Lawshe formula (1975), which is acceptable with a score of 0.64 or greater [24]. CVR will be calculated using the following formula:
$$CVR=\frac{n_E-\frac{N}{\Upsilon}}{\frac{N}{\Upsilon}}$$
The criterion of “relevance” was used for each item on the one-point Likert scale to determine the content validity index. For this purpose, 10 experts were asked to determine the correlation between the questionnaire items and the subscales of the questionnaire on a Likert scale ranging from one (not relevant) to four (completely relevant). Finally, K* will be calculated as follows, using the agreement ratio for the relevance of each item (I-CVI) and the probability of the chance agreement. According to Polit, the minimum number of evaluators required to calculate kappa using this method is three; the number of evaluators will be 10 in the present study. Kappa values of 0.59–0.40, 0.74–0.60, and > 0.74 will be considered poor, good, and excellent, respectively. In this study, only items with kappa of at least 0.74 will be accepted [23].
$$\mathrm{CVI}=\frac{\mathrm{number}\kern0.17em \mathrm{of}\kern0.17em \mathrm{raters}\ \mathrm{giving}\;\mathrm{a}\;\mathrm{rater}\;3\;\mathrm{or}\;4}{\mathrm{total}\kern0.17em \mathrm{number}\kern0.17em \mathrm{of}\kern0.17em \mathrm{raters}}\kern0.5em {p}_c=\left[\frac{N!}{A!\left(N-A\right)!}\right]{.5}^N\kern0.5em {k}^{\ast }=\frac{\mathrm{I}\hbox{-} \mathrm{CVI}-{p}_c}{1-{p}_c}$$
Construct validity
Convergent validity, for human cognition, especially within sociology, psychology, and other behavioral sciences, refers to the degree to which two measures that theoretically should be related are in fact related. Convergent validity, along with discriminant validity, is a subtype of construct validity [25]. In general, tools are classified as reflective and formative. In reflective tools, the items that make up the dimensions of the tool are conceptually related and structural validity is required to examine the dimensions of the tool. In formative tools, the items that make up the dimensions of the tool are not conceptually related [26]. For formative tools structural validity cannot be done by factor analysis (exploratory-confirmatory) method because factor analysis requires a linear relationship between variables [27]. designed tool in this study was a checklist with nature of formative and we used convergent validity to perform the construct validity. Convergent validity, one aspect of construct validity, were examined using Spearman’s correlation. Accordingly, values ≥0.40 represented appropriate convergent validity [28] .
In determining the construct validity of this tool we used the Convergent validity and a similar tool with title “Developing of Iranian Pre-Hospital Emergency Preparedness Assessment tool in Emergencies and Disasters” [29] was sent to thirty of Emergency and Incident Management Centers n (MEAIMC) across the country and the centers were asked to rate this tool based on had to be completed (the level of readiness of the centers for each tool item: if the expected function was performed correctly and on time(2), if the expected function was performed, but its quality or timing was improper (1) and the expected function was not performed (0). Then, about a month later, the initial tools designed in this study were sent to the same thirty of MEAIMC and they were again asked to complete and send these tools. The collected data was entered into SPSS 22 and Spearman correlation coefficient test was used.
Tool Reliability
The test-retest method was used to implement the reliability of the instrument. For this purpose, the instrument was first provided to thirty MEAIMC independent of the construct validity stage and the necessary information was collected. Then, about a month later, the initial tools designed in this study were sent to the same 30 MEAIMC. the collected data was entered into SPSS 22 and Pearson correlation test was used. Ten MEAIMC independent of previous stages were chosen for this study to test instrument reliability. Two evaluators evaluated each MEAIMC independently. After the evaluators completed their evaluations of all MEAIMC, the collected data was entered into SPSS 22 and the reliability of the instrument was examined using Intraclass Correlation Coefficient (ICC).