- Research article
- Open Access
- Open Peer Review
Effects of work conditions on provider mental well-being and quality of care: a mixed-methods intervention study in the emergency department
BMC Emergency Medicinevolume 19, Article number: 1 (2019)
Emergency departments (EDs) are highly dynamic and stressful care environments that affect provider and patient outcomes. Yet, effective interventions are missing. This study evaluated prospective effects of a multi-professional organizational-level intervention on changes in ED providers’ work conditions and well-being (primary outcomes) and patient-perceived quality of ED care (secondary outcome).
A before and after study including an interrupted time-series (ITS) design over 1 year was established in the multidisciplinary ED of a tertiary referral hospital in Southern Germany. Our mixed-methods approach included standardized provider surveys, expert work observations, patient surveys, and register data. Stakeholder interviews were conducted for qualitative process evaluation. ITS data was available for 20 days pre- and post-intervention (Dec15/Jan16; Dec16/Jan17). The intervention comprised ten multi-professional meetings in which ED physicians and nurses developed solutions to work stressors in a systematic moderated process. Most solutions were consecutively implemented. Changes in study outcomes were assessed with paired t-tests and segmented regression analyses controlling for daily ED workload.
One hundred forty-nine surveys were returned at baseline and follow-up (response at baseline: 76 out of 170; follow-up: 73 out of 157). Forty-one ED providers participated in both waves. One hundred sixty expert work observations comprising 240 observation hours were conducted with 156 subsequent work stress reports. One thousand four hundred eighteen ED patients were surveyed. Considering primary outcomes, respondents reported more job control and less overtime hours at follow-up. Social support, job satisfaction, and depersonalization deteriorated while respondents’ turnover intentions and inter-professional interruptions increased. Considering the secondary outcome, patient reports indicated improvements in ED organization and waiting times. Interviews revealed facilitators (e.g., comprehensive approach, employee participation) and barriers (e.g., understaffing, organizational constraints) for intervention implementation.
To the best of our knowledge, this is the first study to report prospective effects of an ED work system intervention on provider well-being and patient-perceived quality of ED care. We found inconsistent results with partial improvements in work conditions and patient perceptions of care. However, aspects of provider mental well-being deteriorated. Given the lack of organizational-level intervention research in EDs, our findings provide valuable insights into the feasibility and effects of participatory interventions in this highly dynamic hospital setting.
Emergency departments (EDs) are highly dynamic work environments with particular risks for provider well-being and quality of care . Adverse work factors contribute to high work stress among ED providers . Burnout has been reported by 26% of emergency nurses  and by up to 51% of emergency physicians . Suboptimal patient care was also linked to adverse ED work factors such as poor teamwork or frequent workflow interruptions [5,6,7]. Notwithstanding the need for effective interventions concerning ED work conditions, there is a dearth of reported interventions [1, 2, 8]. Thus far, research on prospective interventions targeting psychosocial ED work factors is limited [2, 4, 8, 9]: First, an intervention study with ED nurses in three Chinese emergency care facilities showed that comprehensive management (nurse manager-led meetings on communication skills, conflicts, efficacy elevation, and emotion control) was related to lower burnout levels over 6 months . Second, a teamwork intervention in California at four ED sites was associated with improved perceptions of the ED work environment among ED nurses and physicians . In this study, EDs participated in a teamwork training curriculum (Emergency Team Coordination Course) where teamwork principles such as maintaining team structure and climate, problem solving strategies, team communication, executing plans and managing workload as well as team skills were practiced. Lastly, an 18-month prospective study of nurses in 15 Belgian EDs observed that changes in work conditions, such as job demands, control, social support, reward, social harassment and work agreements, were associated with job satisfaction, work engagement, emotional exhaustion and turnover intentions . These study findings indicate that job demands were relatively stable whereas social support and material resources showed most variation over time; turnover intentions deteriorated . Yet this observational research omitted any specification of actual intervention measures. Overall, available studies have shortcomings such as lack of theoretical foundation, insufficient methodological rigor for identification of prospective effects, and sole focus on specific ED professions [2, 8].
The Systems Engineering Initiative for Patient Safety (SEIPS) framework  provides a sociotechnical model of interdependencies between several work factors, provider and patient outcomes in healthcare . The underlying premise is that multiple factors (tasks, technologies, persons, environment, organization) conjointly generate stress loads and affect provider and organizational outcomes, i.e., provider health as well as patient safety . Furthermore, participative interventions apply systematic analyses as well as collaborative improvement of work conditions [15,16,17]. Thus ‘front-line’ providers, such as ED physicians and nurses, are best suited to identify and solve problems in their work environment [18, 19]. Although preliminary evidence suggests that organizational-level interventions can be effective in eliciting positive changes in work systems and provider well-being, thorough evaluations concerning intervention effects on quality of care outcomes are scarce .
The aim of our study was to investigate the effects of ED work system factors on provider well-being and quality of ED care, using the SEIPS model as a theoretical framework. Second, we set out to evaluate the feasibility and effectiveness of a multi-professional organizational-level intervention with focus on both ED provider and patient outcomes. Third, we used mixed-methods interrupted time-series (ITS) evaluations to determine intervention effects. An ITS paradigm is recommended as a quasi-experimental surrogate for assessing intervention effectiveness when a randomized controlled trial is not feasible [21, 22].
A prospective intervention study with a mixed-methods ITS design encompassing a 12-month observation period was established. Methods included (1) standardized provider surveys, (2) structured work observation sessions with ED physicians and nurses and concurrent work stress reports, (3) patient surveys, (4) register data, and (5) stakeholder interviews, which are described below. Observation sessions, work stress reports, and patient surveys were conducted on-site on 20 days each at baseline (Dec 15 – Jan 16) and follow-up (Dec 16 – Jan 17), respectively (exact dates are listed in Additional file 1: Table S1). ED providers were informed about the study via presentations and information leaflets. Written informed consent was obtained prior to data collection. The Ethics Committee of the Medical Faculty, Munich University, approved the study (NR 327–15).
Setting and sample
The study setting is a 24-h interdisciplinary ED of a tertiary referral hospital in a major city in Southern Germany. The academic hospital provides major services and medical specialties for an administrative region of almost two million inhabitants. The ED serves adult patients with mean yearly visits of over 85,000. It is organized in three sections according to patient’s chief complaints: i.e., ten separate bays for non-surgical patients, five separate bays for surgical patients, two resuscitation bays, and an observation and clinical decision unit with 24 beds. The ED is regularly staffed with junior and senior physicians from internal medicine, trauma surgery, and neurology, as well as further specialists on call.
The study team approached two hospitals for participation, whereof this ED’s department head, hospital administration, and hospital’s worker council agreed to take part. The study team established first contact directly with the head of the ED. The head discussed the proposal with the head of nursing and introduced it in two team meetings of ED physicians and nurses, respectively. After positive feedback from ED staff, the study was presented at the hospital board meeting and received approval. The specific motivation and decision for participation were not specified prior to study start. However, the ED was described as a high strain work environment with particular challenges for nurses, physicians, and patient care. All professionals working in the ED, i.e., nurses, physicians, and administrative staff, were eligible for participation. At baseline, ED staff consisted of 101 nurses (including assistant nurses), 44 physicians, and 20 administrators. External providers and on-call consultants were not included.
Methods and study outcomes
Proposed intervention effects on (a) work system factors, (b) provider mental well-being, and (c) quality of care were identified using the following methods:
ED providers received surveys at baseline and follow-up that were distributed through internal mail. Pre-stamped envelopes were provided for direct return of questionnaires to the study team. Deadline for survey completion was four weeks and estimated average time for filling in questionnaires was 25 min. Matching across time was ensured through personalized study codes.
(1a) Work system factors were measured with a validated self-report tool for work analysis in hospitals that has been previously applied to ED work settings [23, 24]. Following SEIPS framework, several work factors were surveyed. Task-related factors included scales on patient stressors (i.e., dealing with difficult patients; three items), job control (i.e., autonomous decision making and personal discretion; four items), participation opportunities (i.e., influence on work-related decisions; four items), work overload (i.e., job duties exceeding work time; three items). Organizational factors consisted of personnel resources (i.e., adequate staffing; three items), information problems (i.e., availability and clarity of work-related information; three items), uncertainty (i.e., job insecurity; three items), social support (i.e., support by colleagues and supervisors; three items), and supervisor feedback (i.e., feedback on performance and task behaviors by senior leaders; two items). Self-reported mean weekly overtime (in hours and minutes), profession (ED physician, ED nurse, ED administrator) and professional tenure (in years) were further retrieved.
(1b) Provider well-being included two key burnout dimensions, i.e., emotional exhaustion and depersonalization (four items each)  and a screening tool for depressive symptoms (two items) . Both tools are validated and have been previously applied in healthcare as well as ED provider samples [8, 24]. Conventional cut-off criteria were used to determine providers with elevated levels of emotional exhaustion (scale mean > 3.5)  and depressive symptoms (scale sum score ≥ 3) . Job satisfaction and turnover intentions were measured with one item, respectively.
(1c) Provider perceptions of quality of care were measured with a three-item scale on the frequency of medical errors . Respondents were asked to indicate whether they had experienced a near miss, minor error, or serious error during the past year. For each type of error, a short definition was provided. Additionally, overall patient safety was further assessed with one item (“Please rate the degree of patient safety in your department from your point of view”) .
Work observation sessions and work stress reports
Observation sessions of ED nurses’ and physicians’ workflows were allocated randomly across three ED sections and professions. Randomization and sessions were limited to provider day shifts between 10:00 am and 5:00 pm on 40 pre-defined days of data collection. Trained observers shadowed providers for 90-min sessions using a standardized participant observation approach that has been previously applied to ED settings [6, 29].
(2a) Work system factors were represented by observed interruption rates. An established tool to identify workflow interruptions was applied [6, 29]. Referencing the SEIPS model, we distinguished between interruptions initiated by patients and their relatives (task-related factors), and those by ED colleagues of the same or another profession (organizational factors). Furthermore, duration of personal breaks (e.g., time for personal rests, short respites from work, or regular pauses during the shift) during observation sessions was coded (in % of observed time) .
(2b) Provider well-being was surveyed immediately after each observation session with a short survey on current cognitive, emotional, and physical aspects of work stress in each observed provider .
All ED patients undergoing consultation or treatment on days of on-site data collection were eligible. After information and verbal consent, patients filled in the survey. Patients’ relatives were allowed to fill in the survey by proxy if they accompanied patients throughout their ED stay. Patients with incapability to communicate due to illness severity or other physical and mental constraints were not surveyed. The patient questionnaire was handed out by members of the study team, preferably at the end of patients’ ED treatment. All study team members received prior training in how to approach and interview ED patients.
(3c) Patient perceptions of ED quality of care were obtained with a standardized patient survey (Munich Patient Inventory) with additional translations in English, Russian, and Turkish language. This questionnaire assesses patient-perceived quality of care  and has been tested for reliability and validity in different clinical settings [33, 34], including the ED setting . It contains scales on the quality of interaction (example item: “My problems and complaints are taken seriously by ED providers”), patient information (e.g., “I am comprehensively informed about the course of therapy and treatment”), ED organization (e.g., “I know who of the ED providers is responsible for me”), and waiting time (e.g., “My waiting time until the first consultation with an ED physician was adequate”). Additionally, patients’ overall satisfaction with care was obtained with one question (“Overall, how do you evaluate care in this ED?”).
Register data on ED workload
An approximate measure for daily ED workload was computed with day-level data on patient numbers, patient acuity (ESI, Emergency Severity Index score), and staffing levels. Data was extracted from ED administrative records and staff rosters.
To complement quantitative results, we used ED stakeholder interviews for qualitative process evaluation [35, 36]. Nine stakeholder interviews were conducted at study half-time and follow-up. A semi-structured interview guideline assessed intervention implementation, facilitators and barriers, and providers’ mental models . Questions were derived from Nielsen & Randell (2013)‘s framework for comprehensive process evaluation of organizational-level interventions . The German interview guideline can be obtained from the corresponding author. Interviewees were recruited through convenience sampling. All interviewed stakeholders had a job tenure > 5 years in the ED, except one junior physician. Four physicians (three with leadership responsibilities), four nurses (one in leadership position) and one ED administrator were interviewed. Overall, there were four female and five male interviewees. The department head, nursing supervisor, and administration head were interviewed twice to comprise intervention implementation at ED management level at half-time as well as follow-up. Interviews lasted between 30 and 60 min. We did not apply a prior estimate of expected sample size for data saturation since project resources did not allow for more stakeholder interviews.
The intervention started after feedback of baseline survey results to ED staff during regular internal meetings as well as through internal mail. Ten 90-min meetings, termed health circles , were held at three-week intervals over a period of seven months. Three to seven ED nurses and physicians participated in each meeting. Principally, all ED providers were invited to take part. Decision for participation was completely at the discretion of each ED provider. Practically, the majority of participants consisted of staff members who were on duty on days of respective meetings; for nurses, usually before or after their shift; for physicians, mostly in breaks during their shift. Therefore, participants varied considerably across individual meetings. Participation in health circles was considered work time. Additionally, ED nursing management and the hospitals’ workers council (German: ‘Personalrat’; employee representatives being elected by hospital employees) attended. All meetings were moderated by the study team. In the first meeting, the concept of health circles was introduced. Potential advantages of this approach (i.e., tailored to local needs, employee involvement, intervention process adapted to context) as well as potential problems (e.g., insufficient implementation of solutions, organizational constraints, time-consuming and long-lasting process of structural re-organization) were discussed. Afterwards, participants identified adverse ED work conditions in a systematic process facilitated by moderators: Participants classified problematic work conditions according to their practical importance and potential for change and formed an agenda of issues for improvement for subsequent meetings. Each of the following meetings focused on the development of measures for one of the identified work problems. In these guided health circles, participants collectively analyzed and discussed potential solutions for work problems and defined an action plan with concrete measures, persons responsible, and deadlines . Two meetings were designated to evaluate implementation status of measures and to discuss potential adjustments. Each health circle meeting was documented in written form and made available to the entire ED staff through the intranet information system. Additionally, participating nurses and physicians were assigned to report on health circle meeting outcomes in their regular team meetings.
After each health circle meeting, participants provided short feedback on their satisfaction with and effectiveness of the respective meeting. We used a self-developed tool that measured five aspects: (1) participant’s satisfaction with meeting outputs (“I am satisfied with the results of today’s health circle”), (2) meeting atmosphere (“I am satisfied with today’s group atmosphere”), (3) opportunities to actively develop solutions for work problems (“In this health circle I can actively contribute to improvements of our work situation”), (4) motivation to improve work conditions (“Today’s health circle meeting motivates me to improve processes and contents of my work”), and (5) participant’s expectations of actual implementation of developed solutions in everyday work (“The developed solutions can be implemented in our daily workflow”). All questions were answered on five-point Likert-Scale ranging from 1=“no, not at all” to “5 = yes, exactly”.
In addition to health circles, three meetings of a steering committee were held during the intervention period. These meetings comprised ED management (ED head, head nurse), the hospitals’ work council and health promotion department, and head of ED administration. The steering committee discussed measures developed by ED staff which could not be implemented immediately, needed approval from ED management, or affected inter-departmental or hospital-wide coordination and decisions . In each meeting, the committee reviewed the action plan, monitored project status, and discussed measures developed in health circle meetings with regard to their implementation in routine work organization and processes.
First, to identify proposed intervention effects, changes in provider survey results between baseline and follow-up were calculated using paired t-tests. To assess strength of changes, Cohen’s effect sizes were calculated and classified as weak (0.2–0.4), moderate (0.5–0.7), or strong effects (> 0.8) . Second, ITS data of hourly interruption rates from work observation sessions, work stress reports, and patient perceptions of care were aggregated to mean scores at the day-level. Data was analyzed with segmented regression analysis with 40 available data points, i.e., 20 each pre- and post-intervention . A daily ED workload measure was calculated from mean daily staffing levels and number and acuity of patients as indicated by ESI scores . Autoregressive integrated moving average (ARIMA) models were estimated and controlled for ED workload . In all steps, listwise deletion was used for missing data. All quantitative analyses were conducted with SPSS 24 (IBM, Chicago). Interview data was analyzed applying content analysis. All nine interviews were audio-taped and transcribed verbatim to cluster recurrent main themes .
In the first health circle meeting, participants identified six adverse ED work conditions: (1) lack of personal breaks (i.e., regular work breaks and personal pauses were often omitted due to high workload); (2) work agreements (i.e., unspecified agreements on patient care and tasks that hinder fast and efficient care); (3) high work pressure environment: point of triage (i.e., ED’s location where incoming patients are assessed for the severity of their symptoms based on a standardized process), (4) leadership, (5) staff information (i.e., insufficient information on current reorganizational projects), and (6) staff shortages (i.e., sustained understaffing of ED personnel). Thirteen respective measures were developed focusing on improvements in task-related and organizational work factors (see Table 1). Implementation fidelity varied by the time of follow-up: Eight solutions were implemented or in progress. Five were deemed unfeasible by the steering committee due to financial constraints and personnel shortages and were not pursued.
At baseline, 170 provider surveys were distributed whereof 76 were returned (response rate: 44.7%). At follow-up, 73 out of 157 surveys were returned (46.5%). Forty-one ED providers participated at both waves (29 ED nurses, 5 ED physicians, 7 ED administrators). Tests for panel attrition (between those who returned a complete survey both times and those who only answered at baseline or follow-up) indicated that the final sample reported higher professional tenure, higher ratings of work overload, and higher depersonalization (see Additional file 1: Table S2).
Overall, 160 observation sessions (80 each at baseline and follow-up) were conducted, resulting in 240 observation hours: 99 with ED nurses and 61 with ED physicians. One hundred fifty-six work stress reports were collected after observations (76 at baseline, 80 at follow-up).
Altogether, 1418 ED patients were surveyed; 694 at baseline (survey response rate: 69.2%) and 724 at follow-up (81.2%).
Changes in work system factors
At baseline, respondents reported high levels of patient stressors, work overload, information problems, and uncertainty. Participation opportunities, personnel resources, and supervisor feedback were rated as below average (see Table 2). Considering mean changes over time, job control significantly increased (p = 0.01). Mean weekly overtime significantly decreased from 7.3 to 5.8 h (p = 0.01). Supervisor feedback improved although this change was not significant (p = .058). However, social support deteriorated at follow-up (p < 0.01). Considering Cohen’s delta, effect sizes for changes in work factors were rather weak ranging from Δ = .31 (social support) to Δ = .42 (overtime) (see Table 2).
No significant changes were observed in mean daily ED workload, workflow interruptions by patients, and respondents’ time spent in personal breaks (see Table 3). Intra-professional interruptions (e.g., nurse interrupts nurse) decreased before the intervention (β = − 0.1, p = 0.04), whereas inter-professional interruptions (e.g., nurse interrupts physician) significantly increased after the intervention (β = 0.1, p = 0.03). Mean interruptions by relatives decreased after the intervention (β = − 0.7, p = 0.03).
Changes in respondents’ well-being
At baseline, 61% of respondents reported high emotional exhaustion and 22% depressive symptoms above cut-off. At follow-up, the proportion of ED respondents with reported emotional exhaustion (75.6%) and depressive symptoms increased (34.1%). Both trends were not statistically significant. Depersonalization significantly increased over time (p = 0.01; see Table 2). Respondents further reported less job satisfaction (p = 0.01) and higher turnover intentions at follow-up (p < 0.01). However, mean daily work stress did not change significantly (see Table 3).
Changes in quality of care
Respondents’ reports of the frequency of medical errors and overall ED patient safety remained stable over time (see Table 2).
However, significant changes in patient-perceived quality of care were observed (see Table 3). At follow-up, patient evaluations of ED organization improved (β = 0.02, p = 0.02). Further, ratings of waiting time declined before the intervention (β = − 0.02, p = 0.01), but improved significantly after the intervention (β = 0.03, p = 0.01). Concerning interaction with ED providers, a negative pre-intervention trend was observed (β = − 0.01, p = 0.045), however, no significant subsequent changes were identified. Patient’s overall satisfaction with ED care remained stable at a high level. Daily ED workload negatively predicted overall patient satisfaction (β = 0.1, p = 0.02), such that patients were less satisfied with overall quality of care on days with less favorable patient/provider-ratios.
Evaluation of intervention fidelity and implementation process
Forty-one surveys were collected after health circle meetings. Participants indicated high satisfaction with meeting outputs (mean, M = 4.2; standard deviation, SD = 1.0), meeting atmosphere (M = 4.7, SD = 0.5), and opportunities to actively develop solutions for work problems (M = 4.1, SD = 0.7). Participants also reported high motivation to improve work conditions (M = 4.0, SD = 1.0). However, developed measures were deemed only partially realizable in everyday work (M = 3.3, SD = 1.1).
Stakeholder interviews revealed that provider surveys and work observation sessions addressed relevant aspects of the ED environment and that sufficient information was available prior to the intervention. Despite extensive staff information and communication efforts prior to study start, some ED providers felt not well informed about the project purpose and process; among interviewees, two out of nine respondents did not take full note of surveys and expressed limited time capacities to deal with available information.
Contextual factors and mental models
Nurses’ intervention participation was deemed successful while there was concern about lack of physician involvement. Two interviewees explicitly mentioned the intervention’s potential to improve ED work conditions. However, only two respondents expressed motivation to actively engage in improvement activities. Few expressed reservations that implementation of measures might be difficult due to organizational constraints (e.g., shortage of personnel resources). Main stakeholder expectations concerning the study were to raise awareness for ED workload in other wards and at hospital management level.
When asked about changes in their work environment, three interviewees reported a deterioration of their work situation, while others reported no changes or slight improvements, especially relating to personal breaks. Three respondents reported a general increase in work stressors (i.e., increased patient numbers, insufficient staffing). Nonetheless, after study completion, the hospital’s health promotion department decided to roll out the intervention to other hospital units. This was partly motivated by the overall positive feedback of ED providers concerning the participatory approach and involvement of representatives from the worker’s council. Moreover, health circle meetings were considered a feasible opportunity for the evaluation of psychosocial risk factors at work.
To the best of our knowledge, this study is the first to systematically investigate prospective effects of a multi-professional organizational-level intervention on ED work conditions, provider well-being, and quality of care. Patient perceptions of ED organization and waiting times, and survey respondents’ self-reported job control and overtime hours improved while some indicators of provider well-being deteriorated. Given the lack of organizational-level intervention research in EDs , our results generate first valuable insights into the feasibility and effects of participatory interventions on ED provider and patient outcomes.
Theoretical assumptions of our study were based on the SEIPS model which links multiple factors of the work system with care processes and provider and patient outcomes [13, 14]. However, observed intervention effects in our study were inconsistent across different outcomes. Considering work factors, job control is a key resource for provider well-being and performance [1, 2]. Participatory interventions were shown to increase job autonomy partly due to their inclusive approach and employee-oriented focus [15, 20]. Our results confirm this assumption for the ED context since survey respondents reported significantly higher job control at follow-up. Furthermore, workflow interruptions are a major work stressor in EDs [29, 41]. In our study, inter-professional interruptions increased after the intervention, which suggests more face-to-face communication and information exchange across professions. Yet, we cannot infer about the underlying reasons and consequences of this increase, e.g., if additional interruptions were more helpful or necessary. Further, interruptions by relatives decreased which might indicate better information of patients and relatives about ED procedures resulting in less need to interrupt ED providers. Accordingly, a key finding of our study was that patients reported significant improvements in ED organization and waiting times. Generally, EDs are interrupt-driven environments and excessive interruptions mitigate provider well-being and performance [6, 42, 43]. However, disruptions can also contribute to efficient and timely patient care [6, 44]. Our results hint to this double-edged sword: frequent interruptions among providers may promote patient-perceived ED organization and shorter waiting times but also contribute to inferior provider well-being.
Our results further corroborate that burnout is a chronic work-related hazard of the ED work environment that affects a significant proportion of ED physicians and nurses [2,3,4]. In our study, job satisfaction decreased while turnover intentions and depersonalization increased at follow-up. Available evidence on effects of organizational interventions on healthcare provider mental well-being is inconsistent [18, 45]. Based on stakeholder interviews, we assume that observed deteriorations in well-being were related to provider’s disappointment about shortcomings in the implementation of developed measures . Furthermore, mental well-being might have affected ED providers’ willingness to engage with intervention measures . Low job satisfaction and low affective well-being were shown to predict intervention participation and evaluation of intervention effectiveness in elder care providers . Moreover, since duration of personal breaks was unchanged over time, opportunities for respite or recovery from high work strain during ED shifts remained limited .
Finally, an unexpected study finding was that social support from colleagues decreased at follow-up. Designing sociotechnical work systems which promote effective teamwork is crucial for positive provider and patient outcomes . However, in our study, ED providers reported concerns that initiated measures were not pursued due to resource limitations, institutional boundaries, and organizational constraints, i.e., financial cuts and a long-standing surge in patient load. Adverse contextual conditions as well as insufficient support by colleagues and managers in implementing measures might thus have led to disappointment and decreased trust within the ED team [17, 36, 48]. Moreover, those interventions with the highest progress of implementation at follow-up were related to improving communication and organization between ED providers, while solutions that involved more resources (i.e., personnel) were among those that were not (or not yet) implemented. Despite consistent participation and support of ED management throughout the study, providers may have developed a perception of limited support since effects did not exert on the ED organizational level as anticipated [17, 36].
We established a mixed-methods ITS study design that allows robust inferences concerning prospective changes of outcomes between pre- and post-intervention assessments . However, pragmatic improvement studies in dynamic clinical settings comprise multiple limitations. First, although our study setting features a typical urban ED setting and relies on elaborated analyses, it lacks a control group. This limits inferences concerning causation as well as secular trends. We describe a realist approach that aimed to change ED work factors. This comprehensive approach targeting several task- and organizational-level aspects over one year does not allow for attribution of effects to single interventions or steps. Our participatory approach consists of several interventions of collective intertwined initiatives that occur in the course of a multitude of everyday concurrent events in patient care . Therefore, we cannot attribute effects of specific measures to primary and secondary outcomes nor specify time lags of measures being effective. Second, our results strongly depend on local contextual factors and the process of intervention implementation . In the study period, other process changes occurred, i.e., reorganization of the triage process and preparation of constructional expansion. ED providers thus might have perceived limited capacity to engage with intervention measures on top of high daily workloads. Nevertheless, our approach combined quantitative results with qualitative information from stakeholder interviews to shed light on these potentially relevant facilitators and barriers in intervention implementation [36, 46]. Third, we acknowledge the rather small longitudinal sample of provider surveys. Although high commitment in data collection and recruitment was undertaken, follow-up bias occurred. This might partially be due to high staff turnover rates which are generally characteristic of EDs . To offset limitations of provider ratings which are prone to subjective bias, we further used objective and independent methods such as expert observation sessions, patient surveys, and register data to measure study outcomes. Yet, observations were confined to day-time shifts which limit inferences concerning provider workflow routines during evening or night shifts . The patient survey tool has been previously applied in various populations including ED patients where it proved its reliability . Specific investigations into the validity of the instrument for ED patient surveys are not yet reported. Fourth, our timing of follow-up assessment needs careful consideration . For practical reasons, we used a one-year time lag. However, ongoing improvements might have failed to reach their full impact on work system factors and provider well-being at the time of our follow-up measurement [17, 51]. Finally, we acknowledge that ED work systems comprise multiple factors and that system interventions should comprehensively address various components to improve provider and patient outcomes [13, 17, 42, 52]. Yet, although preliminary evidence points to positive effects of comprehensive organizational-level interventions for provider outcomes , simultaneous effects for patient care need to be elicited. In our study, solutions concerning other domains such as changes in technologies, tools, or environmental factors, were considered by ED providers in meetings but not prioritized for implementation. This refers particularly to resource constraints such as understaffing or structural provisions of the physical environment. Although we carefully introduced the intervention approach to providers and sought to manage stakeholder expectations in the beginning of the study, post-hoc, we cannot infer on specific anticipations or ‘implicit theories’  of involved stakeholders, e.g., employee assumptions that hospital management would provide additional resources or prioritization of staff shortages.
Implications for practice and further research
Implementation of organizational-level interventions is time-consuming and evaluation of intervention effects is challenging . We used a multi-disciplinary intervention that included both ED nurses and physicians in collaborative meetings and implementation of solutions. Yet, this partly resulted in perceptions of imbalanced involvement, efforts, and contributions of both professions. Future attempts should therefore seek opportunities to implement interventions that take account of the multi-disciplinary nature of ED work as well as consider unique expectations and needs of each profession in the course of participatory work design in clinical care . Concerning methodological aspects, future studies should consider applying cluster-randomized and controlled designs across various ED settings as well as realist evaluation for intervention evaluation . Varying follow-up measurement intervals should be considered to capture potentially time-delayed intervention effects . With regard to intervention content, future studies should expand the scope of assessed work system factors as well as their differential effects on processes, provider and patient outcomes . Furthermore, ED practitioners could expand or adapt existing tools and practices in performance management to include continuous improvement of work system factors . Finally, although our intervention approach was well-accepted by ED providers, partial improvements in work conditions and patient evaluations of care were accompanied by deteriorations in provider mental well-being. Future studies of work system interventions should thus elucidate beneficial concomitants of provider well-being before and during intervention implementation in high stress care environments.
This study provides valuable first insights into the feasibility of organizational-level interventions in EDs to improve work conditions, provider well-being, and quality of care. Our findings indicate that interprofessional approaches targeting work system factors are well-accepted by ED physicians and nurses. Improvement measures developed by ED providers largely focused on changes in organizational work factors. Improvements in job control, overtime hours, and patient perceptions of ED organization and waiting times were observed. However, provider well-being deteriorated over time. Future studies should further identify to what extent and under which circumstances work system interventions are beneficial for provider outcomes in high stress care environments.
Autoregressive integrated moving average
Emergency Severity Index
Systems Engineering Initiative for Patient Safety
Johnston A, Abraham L, Greenslade J, Thom O, Carlstrom E, Wallis M, Crilly J. Review article: staff perception of the emergency department working environment: integrative review of the literature. Emerg Med Australas. 2016;28:7–26. https://doi.org/10.1111/1742-6723.12522.
Basu S, Qayyum H, Mason S. Occupational stress in the ED: a systematic literature review. Emerg Med J. 2017;34:441–7. https://doi.org/10.1136/emermed-2016-205827.
Adriaenssens J, de Gucht V, Maes S. Determinants and prevalence of burnout in emergency nurses: a systematic review of 25 years of research. Int J Nurs Stud. 2015;52:649–61. https://doi.org/10.1016/j.ijnurstu.2014.11.004.
Bragard I, Dupuis G, Fleet R. Quality of work life, burnout, and stress in emergency department physicians: a qualitative review. Eur J Emerg Med. 2015;22:227–34. https://doi.org/10.1097/MEJ.0000000000000194.
Nielsen KJ, Pedersen AH, Rasmussen K, Pape L, Mikkelsen KL. Work-related stressors and occurrence of adverse events in an ED. Am J Emerg Med. 2013;31:504–8. https://doi.org/10.1016/j.ajem.2012.10.002.
Weigl M, Müller A, Holland S, Wedel S, Woloshynowych M. Work conditions, mental workload and patient care quality: a multisource study in the emergency department. BMJ Qual Saf. 2016;25:499–508. https://doi.org/10.1136/bmjqs-2014-003744.
Jepson ZK, Darling CE, Kotkowski KA, Bird SB, Arce MW, Volturo GA, Reznek MA. Emergency department patient safety incident characterization: an observational analysis of the findings of a standardized peer review process. BMC Emerg Med. 2014;14:20. https://doi.org/10.1186/1471-227X-14-20.
Schneider A, Weigl M. Associations between psychosocial work factors and provider mental well-being in emergency departments: a systematic review. PLoS One. 2018;13:e0197375. https://doi.org/10.1371/journal.pone.0197375.
Büssing A, Perrar K-M. Die Messung von Burnout. Untersuchung einer deutschen Fassung des Maslach Burnout Inventory (MBI-D) [Measuring burnout: a study of a German version of the Maslach Burnout Inventory (MBI-D)]. Diagnostica. 1992;38:328–53.
Wei R, Ji H, Li J, Zhang L. Active intervention can decrease burnout in ED nurses. J Emerg Nurs. 2017;43:145–9. https://doi.org/10.1016/j.jen.2016.07.011.
Ajeigbe DO, McNeese-Smith D, Leach LS, Phillips LR. Nurse-physician teamwork in the emergency department: impact on perceptions of job environment, autonomy, and control over practice. J Nurs Adm. 2013;43:142–8. https://doi.org/10.1097/NNA.0b013e318283dc23.
Adriaenssens J, de Gucht V, Maes S. Causes and consequences of occupational stress in emergency nurses, a longitudinal study. J Nurs Manag. 2015;23:346–58. https://doi.org/10.1111/jonm.12138.
Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M, Flatley Brennan P. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15:i50–8. https://doi.org/10.1136/qshc.2005.015842.
Carayon P, Wetterneck TB, Rivera-Rodriguez AJ, Hundt AS, Hoonakker P, Holden R, Gurses AP. Human factors systems approach to healthcare quality and patient safety. Appl Ergon. 2014;45:14–25. https://doi.org/10.1016/j.apergo.2013.04.023.
Aust B, Ducki A. Comprehensive health promotion interventions at the workplace: experiences with health circles in Germany. J Occup Health Psychol. 2004;9:258–70. https://doi.org/10.1037/1076-89188.8.131.528.
Leis JA, Shojania KG. A primer on PDSA: executing plan–do–study–act cycles in practice, not just in name. BMJ Qual Saf. 2017;26:572–7. https://doi.org/10.1136/bmjqs-2016-006245.
Montano D, Hoven H, Siegrist J. Effects of organisational-level interventions at work on employees’ health: a systematic review. BMC Public Health. 2014;14:135. https://doi.org/10.1186/1471-2458-14-135.
von Thiele Schwarz U, Nielsen KM, Stenfors-Hayes T, Hasson H. Using kaizen to improve employee well-being: results from two organizational intervention studies. Hum Relat. 2017;70:966–93. https://doi.org/10.1177/0018726716677071.
Stab N, Hacker W. Participatory redesign of work organisation in hospital nursing: a study of the implementation process. J Nurs Manag. 2018;26:382–92. https://doi.org/10.1111/jonm.12545.
Nielsen K, Randall R, Holten A-L, González ER. Conducting organizational-level occupational health interventions: what works? Work Stress. 2010;24:234–59. https://doi.org/10.1080/02678373.2010.515393.
Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Qual Saf. 2015;24:748–52. https://doi.org/10.1136/bmjqs-2014-003756.
Fretheim A, Zhang F, Ross-Degnan D, Oxman AD, Cheyne H, Foy R, Goodacre S, Herrin J, Kerse N, McKinlay RJ, et al. A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation. J Clin Epidemiol. 2015;68:324–33. https://doi.org/10.1016/j.jclinepi.2014.10.003.
Büssing A, Glaser J. Das Tätigkeits-und Arbeitsanalyseverfahren für das Krankenhaus - Selbstbeobachtungsversion (TAA-KH-S) [Work analysis instrument for hospitals - self report version (TAA-KH-S)]. Göttingen: Hogrefe; 2002.
Weigl M, Schneider A. Associations of work characteristics, employee strain and self-perceived quality of care in emergency departments: a cross-sectional study. Int Emerg Nurs. 2017;30:20–4. https://doi.org/10.1016/j.ienj.2016.07.002.
Kroenke K, Spitzer RL, Williams JBW. The patient health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41:1284–92. https://doi.org/10.1097/01.MLR.0000093487.78664.3C.
Kleijweg JHM, Verbraak MJPM, Van Dijk MK. The clinical utility of the Maslach burnout inventory in a clinical population. Psychol Assess. 2013;25:435–41. https://doi.org/10.1037/a0031334.
Waterman AD, Garbutt J, Hazel E, Dunagan WC, Levinson W, Fraser VJ, Gallagher TH. The emotional impact of medical errors on practicing physicians in the United States and Canada. Jt Comm J Qual Patient Saf. 2007;33:467–76. https://doi.org/10.1016/S1553-7250(07)33050-X.
Pfeiffer Y, Manser T. Development of the German version of the hospital survey on patient safety culture: dimensionality and psychometric properties. Saf Sci. 2010;48:1452–62. https://doi.org/10.1016/j.ssci.2010.07.002.
Weigl M, Müller A, Vincent C, Angerer P, Sevdalis N. The association of workflow interruptions and hospital doctors' workload: a prospective observational study. BMJ Qual Saf. 2012;21:399–407. https://doi.org/10.1136/bmjqs-2011-000188.
Füchtbauer LM, Nørgaard B, Mogensen CB. Emergency department physicians spend only 25% of their working time on direct patient care. Dan Med J. 2013;60:A4558.
Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger state-trait anxiety inventory (STAI). Br J Clin Psychol. 1992;31:301–6. https://doi.org/10.1111/j.2044-8260.1992.tb00997.x.
Büssing A, Glaser J. Employee and client orientation in health care services. Zeitschrift für Arbeits- und Organisationspsychologie. 2003;47:222–8. https://doi.org/10.1026//0932-4089.47.4.222.
Weigl M, Hoffmann F, Müller A, Barth N, Angerer P. Hospital paediatricians’ workflow interruptions, performance, and care quality: a unit-based controlled intervention. Eur J Pediatr. 2014;173:637–45. https://doi.org/10.1007/s00431-013-2232-z.
Weigl M, Hornung S, Angerer P, Siegrist J, Glaser J. The effects of improving hospital physicians working conditions on patient care: a prospective, controlled intervention study. BMC Health Serv Res. 2013;13:401. https://doi.org/10.1186/1472-6963-13-401.
Nielsen K, Randall R. Opening the black box: presenting a model for evaluating organizational-level interventions. Eur J Work Org Psych. 2013;22:601–17. https://doi.org/10.1080/1359432X.2012.690556.
Aust B, Rugulies R, Finken A, Jensen C. When workplace interventions lead to negative effects: learning from failures. Scand J Public Health. 2010;38:106–19. https://doi.org/10.1177/1403494809354362.
Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale: Erlbaum; 1988.
Bernstein SL, Verghese V, Leung W, Lunney AT, Perez I. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med. 2003;10:938–42. https://doi.org/10.1197/S1069-6563(03)00311-7.
Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27:299–309. https://doi.org/10.1046/j.1365-2710.2002.00430.x.
Miles MB, Huberman AM. Qualitative data analysis: a sourcebook of new methods. Beverly Hills: Sage Publications; 1984.
Ramlakhan S, Qayyum H, Burke D, Brown R. The safety of emergency medicine. Emerg Med J. 2016;33:293–9. https://doi.org/10.1136/emermed-2014-204564.
Werner NE, Holden RJ. Interruptions in the wild: development of a sociotechnical systems model of interruptions in the emergency department through a systematic review. Appl Ergon. 2015;51:244–54. https://doi.org/10.1016/j.apergo.2015.05.010.
Jeanmonod R, Boyd M, Loewenthal M, Triner W. The nature of emergency department interruptions and their impact on patient satisfaction. Emerg Med J. 2010;27:376–9. https://doi.org/10.1136/emj.2008.071134.
Sasangohar F, Donmez B, Trbovich P, Easty AC. Not all interruptions are created equal: positive interruptions in healthcare. Proc Hum Factors Ergon Soc Annu Meet. 2012;56:824–8. https://doi.org/10.1177/1071181312561172.
Ruotsalainen JH, Verbeek JH, Marine A, Serra C. Preventing occupational stress in healthcare workers. Cochrane Database Syst Rev. 2015:CD002892. https://doi.org/10.1002/14651858.CD002892.pub3.
Nielsen K, Miraglia M. What works for whom in which circumstances? On the need to move beyond the ‘what works?’ Question in organizational intervention research. Hum Relat. 2017;70:40–62. https://doi.org/10.1177/0018726716670226.
Nielsen K, Randall R. The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work Stress. 2012;26:91–111. https://doi.org/10.1080/02678373.2012.682721.
Pannick S, Sevdalis N, Athanasiou T. Beyond clinical engagement: a pragmatic model for quality improvement interventions, aligning clinical and managerial priorities. BMJ Qual Saf. 2016;25:716–25. https://doi.org/10.1136/bmjqs-2015-004453.
Kirkegaard ML, Kines P, Nielsen HB, Garde AH. Occupational safety across jobs and shifts in emergency departments in Denmark. Saf Sci. 2018;103:70–5. https://doi.org/10.1016/j.ssci.2017.11.014.
Dormann C, Griffin MA. Optimal time lags in panel studies. Psychol Methods. 2015;20:489–505. https://doi.org/10.1037/met0000041.
Ford MT, Matthews RA, Wooldridge JD, Mishra V, Kakar UM, Strahan SR. How do occupational stressor-strain effects vary with time? S review and meta-analysis of the relevance of time lags in longitudinal studies. Work Stress. 2014;28:9–30. https://doi.org/10.1080/02678373.2013.877096.
Holden RJ, Carayon P, Gurses AP, Hoonakker P, Hundt AS, Ozok AA, Rivera-Rodriguez AJ. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics. 2013;56:1669–86. https://doi.org/10.1080/00140139.2013.838643.
Wears RL. Improvement and evaluation. BMJ Qual Saf. 2015;24:92–4. https://doi.org/10.1136/bmjqs-2014-003889.
We would like to thank Joana Beck, Tobias Augenstein, Tobias Filmer, Jeannette Weber, Theresa Steeb, and Nikolaus Bürger for their support in data collection.
The study was supported by the Munich Centre of Health Sciences (MC-Health). The funding body did not have any role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
Anonymized and de-identified data will be made available to other investigators upon request. Interested readers should contact the corresponding author.
Ethics approval and consent to participate
The Ethics Committee of the Medical Faculty, Munich University, approved the study (NR 327–15). Informed consent was obtained from participants in written (provider survey) and verbal form (participants in standardized work observations; participants in patient survey). Verbal consent was obtained because no personal data was collected from participants in work observations and from patients. The ethics committee approved this procedure.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1. Dates of data collection. Dates of data collection at baseline and at follow-up. Table S2 Panel attrition: Loss to follow-up analysis. Analysis of loss to follow-up in provider survey participants from baseline to follow-up. (PDF 36 kb)
About this article
- Work conditions
- Emergency department
- Mental well-being
- Quality of care
- Patient survey
- Emergency care
- Emergency service