This article has Open Peer Review reports available.
A patient survey of out-of-hours care provided by Emergency Care Practitioners
© Halter et al; licensee BioMed Central Ltd. 2007
Received: 25 September 2006
Accepted: 15 June 2007
Published: 15 June 2007
Emergency Care Practitioners (ECPs) have recently been deployed to provide out-of-hours primary care home visits – a practice development that has been supported by policy makers. The aim of the study was to evaluate the care provided to patients receiving out-of-hours home visits from ECPs in London from the patients' perspective and to assess their wellbeing following the visit.
A bespoke telephone-administered questionnaire was designed to survey all patients who had received out-of-hours care in Bromley Primary Care Trust from ECPs during a ten week period in 2005 (n = 174).
Sixty three patients (36.2%) were excluded because: no telephone number was available; they had a diagnosis of dementia; or had not received a study information sheet. The remainder (n = 111) were contacted 3–5 days after the home visit, and 81 of these (73.0%) completed the survey. Of those respondents treated at home who gave unequivocal answers (n = 60), all but one (8.3%) reported that they felt that their treatment had been 'right' and/or had followed any advice given. However, overall only 86.4% reported that they had been clear about their ECP's assessment, and only 58.0% reported that their health was now 'better'. Those who reported that they were not clear about their assessment were less likely to report that their health was 'better' (p = 0.03) and more likely to have subsequently used hospital-based health services (p = 0.03).
Most patients treated at home by ECPs appeared satisfied and compliant with the care provided, according to the measures used in this study. However, it appears that a sizeable minority of patients were unclear about ECP assessments and it remains to be seen whether these patients had pre-existing health complaints which made them less likely to recover and more likely to seek hospital care, or whether the lack of clarity about their assessment undermined their subsequent recovery and necessitated hospital care. Further research is required to establish if the assessments provided by ECPs are less clear than those provided by other practitioners, and whether it is possible to ensure that all such assessments are clear to all patients.
Patients hold a mainly positive view of out-of-hours home visit care provided by ECPs, although a lack of clarity about their assessment was evident, with a possible impact on their continuing health.
The delivery of out-of-hours primary health care has changed substantially in recent years and now includes a range of service models, including: deputising services; telephone triage; primary care centres; walk-in centres; emergency departments; and/or cooperatives . A new model, which has been positively received by policy makers [2, 3], involves Emergency Care Practitioners (ECPs) working with GPs, particularly in the delivery of primary care through out-of-hours home visits. For this and their other enhanced roles, ECPs undertake additional training to develop autonomous practice which aims to enable them to assess and treat the patient at the point of access (wherever possible) and thereby avoid the use of hospital-based emergency departments (wherever appropriate), with subsequent gains in capacity in the hospital sector. Additionally, in the GP environment, the aim has been to reduce waiting times in primary care by visiting patients for GPs .
There has been a paucity of research into alternative models of out-of-hours primary care, leading to a lack of evidence about clinical outcomes . What studies there have been have focused on patient satisfaction and have found that patients were less satisfied when they did not receive the care they were expecting [5–7]. Related research on alternative models of pre-hospital emergency care, where Emergency Medical Technicians or Paramedics have adopted an ECP-type role (which allowed them to 'treat and release' patients at the scene – albeit without the additional training that ECPs receive), suggested that there were a number of unresolved concerns about patient safety [8–10].
The aim of the present study was therefore to evaluate the care provided to patients receiving out-of-hours home visits from ECPs in London from the patients' perspective and assess their wellbeing following the visit. We anticipated that the potential existed for ECPs to make inappropriate care decisions, given that ECPs are comparatively inexperienced providers of out-of-hours care. And although this was not intended to be a survey of patient satisfaction, we expected that patients' experiences of care might be negatively affected by the fact that this was delivered by an ECP, given that patients who call their out-of-hours GP service would not have expected an ECP to carry out their home visit.
The ECPs currently practising in London are all Emergency Medical Technicians or Paramedics who have undertaken additional diploma-level training, part-time over a two year period. During this time they continue to practice, primarily by responding to emergency ambulance calls. The diploma involves dedicated modules on physical assessment, clinical decision-making, minor illness, chronic conditions, pharmacology, paediatric care and the health of older people. At the same time, ECPs also undertake supervised clinical placements with a variety of other practitioners as part of their continuing professional development, including: GPs; emergency department physicians; community nurses; and social care professionals. More recently they have started to use 'Patient Group Directives' to administer a limited number of medicines, although this had not been introduced at the time the present study took place.
Since December 2004 ECPs have also been deployed to conduct out-of-hours home visits between 09h00 and 21h00 at weekends and on public holidays. When the present study took place (February to April 2005) London ECPs were working alongside GPs in just one such out-of-hours service, in a single NHS-defined geographical area – Bromley Primary Care Trust. This service involves patients who contact their out-of-hours GP service by telephone. Each such call is logged by a call handler who passes the call on to a GP based at the out-of-hours primary care centre. This GP telephones the patient back to elicit any clinical information required and to make an initial assessment of any clinical needs. At this point the patient may be given advice on the telephone, or asked to attend the out-of-hours primary care centre. But if a home visit is deemed necessary, the GP decides if the patient's condition is suitable for an ECP, or whether a GP is required (although the patient will not ordinarily know which type of practitioner will be visiting them). This decision is based on strict criteria, developed by the GP lead for the ECP out-of-hours scheme, which determines which conditions are suitable for home visits by ECPs.
At the time of the present study any information provided by the patient calling the out-of-hours service, together with the assessment made by the GP based at the out-of-hours centre, was transmitted electronically to a computer in the ECP's response car. And after every home visit, whether by a GP or ECP, the patient's own GP received a faxed copy of the 'call sheet' which contained the attending practitioner's record of the assessment they had made, any treatment they had provided and any further care they recommended. This could include a recommendation to the patient's GP that they should be followed up urgently, but this was not always the case. The management information available for analysis in the present study therefore included: the age and gender of the patient; their presenting complaint; subsequent disposal (i.e. treatment at home with/without referral, or conveyance to the out-of-hours primary care centre/hospital emergency department); and the time spent with the patient.
A bespoke questionnaire was developed by members of the research team (MH and TM), drawing on discussions with the ECP-, GP- and management-leads for out-of-hours care in Bromley Primary Care Trust, and focusing on concerns raised before the scheme became operational. The questionnaire was subsequently examined by these leads who concluded that it had both face and content validity, and had successfully covered all of their principal concerns. The questionnaire used both closed- and open-ended questions and focussed on the following aspects of out-of-hours care: what had happened to the patient during and after the ECP's visit, with questions tailored to two groups -those patients who had received advice (had they followed the advice) and those who had been treated, referred or taken to another facility (had this treatment felt 'right'); clarity about the ECP's assessment (i.e. what the outcome of care would be and when any subsequent care might happen); and whether their health had felt 'better' following the ECP's visit. Data collected during the first week of the study were used to pilot and evaluate the questionnaire, but no issues arose with administering the survey during this period, and since no modifications were made, the data from this initial week were included in the final analyses.
All patients who had received an ECP out-of-hours home visit between 26th February and 15th May 2005 were eligible for recruitment into the study. ECPs were asked to give these patients an information sheet at the end of their home visit, which explained that they might receive a telephone call from a researcher, at which point the research governance approval allowed the researchers to contact the participants and request informed consent to participate in the study. The records of each of these patients were then collected from the GP out-of-hours centre and potential participants were telephoned during office hours within 3–5 days of their ECP home visit (i.e. as soon as possible thereafter).
Responses to the questionnaire were transcribed verbatim during each telephone interview, and were subsequently categorised by two of the authors (MH and TM) prior to analysis. Data were managed in Microsoft Access and were statistically analysed using SPSS version 12 to conduct t tests, χ 2 tests and Fisher's exact tests.
Research governance approval for the study was granted by the London Ambulance Service and Bromley Primary Care Trust.
Reasons for excluding potential participants and non-respondents
Reason for exclusion
% within stage
Prior to contacting patients (of n = 174)
No telephone number recorded
No record that the patient had received a study information sheet1
On patient contact (of n = 111)
No reply to telephone call
Patient declined to participate
Sociodemographic and service-related characteristics of respondents and non-respondents
Aged less than 60
Aged 60 and over
Pain (other than chest or back)
Treated at home2
Treated at home and referred
Conveyed to the out-of-hours primary care centre
Conveyed to the hospital emergency department
Time spent with the patient (minutes)
Median (where known, n = 81)
47 (15 – 94)
From Table 2 it is clear that most of the respondents were female (67.9%) and aged 60 or above (59.3%). More than two thirds had presented with respiratory infections (27.2%), urological complaints (17.3%), back pain (12.3%) or medical conditions (12.3%; including: arthritis; hypertension; influenza; and heart failure), while comparatively few had gastro-intestinal complaints (7.4%), pain (other than chest or back pain; 6.2%), minor injuries (5.0%), dizziness (2.5%) or neurological complaints (2.5%). Few of these conditions resulted in immediate conveyance to the hospital emergency department (12.3%) or out-of-hours primary care centre (3.7%) and most were treated at home, with (32.1%) or without (51.9%) referral to additional services. Finally, there was substantial variation in the duration of ECP home visits, which ranged from just 11 minutes to more than two hours (148 minutes), and averaged just under an hour (50 minutes).
Self-reported outcomes following the ECP home visit
Remained at home (n = 68)
Followed advice given or treatment felt right (n = 68)
Unclear from responses
Overall services used (n = 68)
GP or nurse
Follow up to the ECP's referral (n = 26)1
Not seen yet (GP = 9, physiotherapist = 1)1
Seen and remained at home (GP = 11, nurse = 2, GP and nurse = 1, emergency department = 1)1
Seen and admitted to hospital (GP)
No referral reported but services used (n = 16)
Seen and remained at home (GP = 14, nurse = 1)
Seen and admitted to hospital (GP)
Conveyed (n = 13)
Outcome following conveyance
Discharged from out-of-hours centre
Discharged from the emergency department
Admitted to hospital
All (n = 81)
Clear about the outcome at the end of the assessment
No response to this question
Clear about when the next steps might happen?
Unclear from responses
No response to this question
How are you now?
Before laying too much store by the results of the present study there are two potential limitations that need to be taken into account. First and foremost, the survey drew on a modest sample of respondents, a disproportionate number of whom were women, treated at home and aged ≥60. However, since older people are the principal recipients of home visits in this out-of-hours service, and since most of those attended at home by GPs remain there after their visit, it seems likely that respondents were broadly similar to patients receiving out-of-hours home visits from GPs. And although there were significant differences in presenting complaint amongst those excluded and included in the study, and it is unclear whether patients selected for ECP visits presented with the same level of complexity or severity of presenting condition, the subsequent diagnoses of respondents in our study suggests that they were broadly representative of patients receiving similar care elsewhere. Indeed, the respondents in the present study had a similar range of diagnoses  and outcomes (such as subsequent self-reported health  and hospital admission ) to those observed by previous studies of out-of-hours care. Nonetheless, the findings of this study are only strictly applicable to patients with similar presenting conditions subject to similar selection for subsequent care, and who were primarily treated at home rather than in hospital.
Meanwhile, a second potential limitation is that the study used a bespoke questionnaire which relied on self-reported outcomes rather than more objective measures which might have provided a more reliable assessment of the quality of care provided by ECPs . However, the use of self-reported outcomes is entirely appropriate for addressing the aim of the present study, which was to assess patients' experiences of the care they had received, and this approach was crucial for identifying an important minority of patients who were unclear about the assessments provided by ECPs. As such the questionnaire successfully captured sufficient variation in perceived care and subsequent wellbeing to identify important differences in these outcomes amongst different groups of respondents, even though there was insufficient variation in their response to advice, services used or conveyance to hospital to permit a detailed investigation thereof (see Table 3).
Notwithstanding these potential limitations, the present study does provide a degree of reassurance that ECPs were capable of delivering care that was considered acceptable, or advice that was followed by patients, at out-of-hours home visits, according to the measures we used. Moreover, most of our respondents were treated at home which suggests that our interpretation of findings is particularly relevant to this group. Setting aside the eight respondents who provided equivocal answers, all but one of the remaining 60 respondents who had not been conveyed to hospital felt that the care they had received from their ECP had been 'right' or that they had followed the advice the ECP had given. This high level of agreement with the care provided by ECPs, and of self-reported compliance with the advice offered by ECPs, mirrors the high level of satisfaction found in a recent evaluation of ECPs working in emergency care . This suggests that the ECP model, particularly its focus on treating patients at the point of contact and avoiding conveyance to hospital wherever appropriate, is well-received by most patients in both contexts. And while the present study was not intended to be a survey of general satisfaction with ECP-delivered out-of-hours primary care, it is worth considering such studies in similar settings to inform what we might have expected to find here. In particular, while patient satisfaction is generally higher with home visits than with other models of out-of-hours primary care [4, 5], we did not expect such a positive response to the ECP home visit because the patients were likely to have been expecting a GP, and previous studies have found higher satisfaction with out-of-hours care when this met patients' prior expectations [5–7]. With this in mind the present study suggests that patients were sufficiently satisfied with the care received from ECPs to overcome any disappointment at not being visited by a GP.
Despite these reassuring findings, the present study did identify a sizeable minority of respondents who were unclear about their ECP's assessment. These patients were less likely to report that their health was 'better' and more likely to have subsequently sought hospital care. While it is not possible to establish the reasons for these associations from the cross-sectional survey used in the present study, it is possible to identify two possible explanations and suggest fruitful avenues for future research. The first explanation is that the ECPs did not always provide a clear assessment of what would happen next for each of the patients they visited, and that a lack of clarity undermined the recovery of these patients and increased their risk of requiring or seeking hospital-based care. The second explanation is that health conditions which are more difficult to assess and explain to patients are less likely to improve and more likely to require additional, hospital-based care. Certainly, patients with lower self-reported health status have been found to be significantly less satisfied with their out-of-hours care in the past , and it is likely that it is difficult for ECPs and GPs to clearly assess or explain all conditions to all patients. Nonetheless, the present study found that ECPs' assessments can be unclear to out-of hours primary care patients, despite the fact that they spend longer with them than GPs. Further research is therefore warranted to establish whether this might reflect ECPs' relative inexperience in out-of-hours primary care, and whether this might improve with appropriate experience, clinical supervision or training. Further research is also required to establish if a lack of clarity is only found when ECPs provide such care, or whether it might be expected from any practitioner dealing with difficult conditions or patients with confusion. Either way, additional support and training might be appropriate for ECPs and GPs to improve patient understanding of the out-of-hours assessments they provide.
The present study found high levels of agreement with the care provided or self-reported compliance with advice provided by ECPs during out-of-hours primary care home visits. However, it also found a sizeable minority of patients who were not clear about their ECP's assessment, and this was associated with subsequent wellbeing and use of services.
We are grateful to the participating patients; to Richard Mintern and the Bromley ECP team at London Ambulance Service for providing data and informing patients of the study; and to Mary Cooke at EMDoc for providing access to data. Caro van Uden, Gerard Bury and Edward Krupat all provided helpful suggestions for improving the manuscript following review.
- Department of Health: Raising standards for patients: new partnerships in out-of-hours care An independent review of GP out-of-hours services in England commissioned by the Department of Health. 2000, London , Department of HealthGoogle Scholar
- NHS Modernisation Agency: Right Skill, Right Time, Right Place The ECP Report. 2004, London , Department of Health,Google Scholar
- Department of Health: Taking Healthcare to the Patient. Transforming NHS Ambulance Services. 2005, London , Department of HealthGoogle Scholar
- Leibowitz R, Day S, Dunt D: A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Family Practice. 2003, 20 (3): 311-317. 10.1093/fampra/cmg313.View ArticlePubMedGoogle Scholar
- Thompson K, Parahoo K, Farrell B: An evaluation of a GP out-of-hours service: meeting patient expectations of care. Journal of Evaluation in Clinical Practice. 2004, 10 (3): 467-474. 10.1111/j.1365-2753.2004.00446.x.View ArticlePubMedGoogle Scholar
- McKinley RK, Roberts C: Patient satisfaction with out of hours primary medical care. Quality in Health Care. 2001, 10 (1): 23-28. 10.1136/qhc.10.1.23.View ArticlePubMedPubMed CentralGoogle Scholar
- McKinley RK, Stevenson K, Adams S, Manku-Scott TK: Meeting patient expectations of care: the major determinant of satisfaction with out-of-hours primary medical care?. Family Practice. 2002, 19 (4): 333-338. 10.1093/fampra/19.4.333.View ArticlePubMedGoogle Scholar
- Snooks HA, Dale J, Hartley-Sharpe C, Halter M: On-scene alternatives for emergency ambulance crews attending patients who do not need to travel to the accident and emergency department: a review of the literature. Emergency Medicine Journal. 2004, 21 (2): 212-215. 10.1136/emj.2003.005199.View ArticlePubMedPubMed CentralGoogle Scholar
- Pringle RP, Carden DL, Xiao F, Graham DD: Outcomes of patients not transported after calling 911. Journal of Emergency Medicine. 2005, 28 (4): 449-454. 10.1016/j.jemermed.2004.11.025.View ArticlePubMedGoogle Scholar
- Hauswald M, Raynovich W, Brainard AH: Expanded emergency medical services: the failure of an experimental community health program. Prehospital Emergency Care. 2005, 9 (2): 250-253. 10.1080/10903120590924942.View ArticlePubMedGoogle Scholar
- Brogan C, Pickard D, Gray A, Fairman S, Hill A: The use of out of hours health services: a cross sectional survey. BMJ. 1998, 316 (7130): 524-527.View ArticlePubMedPubMed CentralGoogle Scholar
- McKinley RK, Cragg DK, Hastings AM, French DP, Manku-Scott TK, Campbell SM, Van F, Roland MO, Roberts C: Comparison of out of hours care provided by patients' own general practitioners and commercial deputising services: a randomised controlled trial. II: The outcome of care. BMJ. 1997, 314 (7075): 190-193.View ArticlePubMedPubMed CentralGoogle Scholar
- Salisbury C: Observational study of a general practice out of hours cooperative: measures of activity. BMJ. 1997, 314 (7075): 182-186.View ArticlePubMedPubMed CentralGoogle Scholar
- Rao M, Clarke A, Sanderson C, Hammersley R: Patients' own assessments of quality of primary care compared with objective records based measures of technical quality of care: cross sectional study. BMJ. 2006, 333 (7557): 19-10.1136/bmj.38874.499167.7C.View ArticlePubMedPubMed CentralGoogle Scholar
- Halter M, Marlow T, Tye C, Ellison GTH: Patients' experiences of care provided by emergency care practitioners and traditional ambulance practitioners: a survey from the London Ambulance Service. Emerg Med J. 2006, 23 (11): 865-866. 10.1136/emj.2005.032912.View ArticlePubMedPubMed CentralGoogle Scholar
- Glynn LG, Byrne M, Newell J, Murphy AW: The effect of health status on patients' satisfaction with out-of-hours care provided by a family doctor co-operative. Family Practice. 2004, 21 (6): 677-683. 10.1093/fampra/cmh616.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-227X/7/4/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.