The persistent growth in demand on emergency ambulance services is an international problem which has little to do with changing patterns of disease and injury but appears to reflect a range of societal and organisational factors.¹ Between 11% and 52% of emergency ambulance call outs are to patients with non-serious problems.^1–15 Excessive demand may lead to delays in the arrival of ambulances to those with life threatening needs, potentially contributing to avoidable morbidity and mortality.¹⁶

There is limited evidence about the impact and acceptability of alternatives to immediate emergency ambulance despatch.¹ It has been suggested that telephone assessment using computerised decision support could enable identification of whether alternatives to ambulance despatch could be safely offered. Telephone assessment systems have been piloted and evaluated in other contexts and have been shown to be acceptable to the public and to be safe.^17,18

We recently reported results from a pragmatic randomised controlled trial to assess the potential impact of telephone triage and advice to emergency ambulance callers—to our knowledge, the first study to test this intervention as an alternative to ambulance despatch. Of 635 emergency (999) ambulance service calls prioritised by call takers using priority despatch criteria as presenting with non-serious (category C) problems, approximately half (52.0%) were triaged by nurses and paramedics using computerised clinical decision support as not requiring emergency ambulance despatch.¹⁹ The trial was conducted in shadow form—that is, with ambulances continuing to be despatched to all calls at the commencement of prioritisation by the call taker—because of uncertainty about the safety of the intervention. Nurses were more likely than paramedics to triage calls as not requiring an ambulance, odds ratio 1.28 (95% CI 1.12 to 1.47) after controlling for age, case mix, time of day, day of week, season, and ambulance service. Telephone assessment was found to be acceptable to callers.²⁰

The main trial found that a significant proportion (almost 10%) of patients triaged as not requiring immediate despatch of an emergency ambulance were subsequently admitted to inpatient hospital care, raising concerns about the safety of the intervention.¹⁹

The purpose of the study reported here is to re-analyse the trial data to assess the safety of the assessment and advice that had been provided in the light of health care needs identified in the days following the call to the emergency ambulance service. In-depth review of patients’ records was undertaken by an independent expert clinical panel to identify whether patients’ lives may have been put at risk had their access to emergency care been delayed by the intervention.

METHODS

The recruitment of patients to the main trial, the intervention applied, and the data collection methods have been described elsewhere¹⁹ and are summarised in box 1. Data collection was conducted between April 1998 and May 1999. The trial was conducted across two ambulance service sites that together covered the whole of Greater London, Birmingham, Coventry, the Black Country, and South Staffordshire (a total population of approximately 10 million).

Nurses and paramedics trained in telephone assessment and advice worked within the ambulance service control rooms during randomly sampled 4 hour sessions between 07.00 and 23.00 hours across the entire data collection period. Emergency (999) calls were prioritised by control room call takers using a priority despatch system (box 2). Callers that were classed as category C were first reassured that the ambulance was on its way and then, during intervention sessions, transferred to a nurse or paramedic for further assessment and advice.

The subsample of patients assessed by nurses and paramedics as not requiring an emergency ambulance—that is, considered to require moderately urgent, routine care, or home care advice—were the subjects of the study reported here. The care pathway for each patient was defined from a variety of sources (box 1): follow up questionnaires posted to all patients 4–7 days after the call,²⁰ ambulance service control room records and “at scene” records for patients attended by ambulance personnel, and episode records of all patients who attended accident and emergency departments or general practice within 4 days of the call.

Assessment of safety of triage decisions: independent specialist review panel

Panel

An independent specialist review panel was recruited to assess the clinical safety of the triage decision making. It was constituted to reflect clinical expertise at the interface between primary care and emergency care, and included a heterogeneous group as the aim of the study was to explore areas of uncertainty. To minimise institutional and professional bias, panellists were recruited from a variety of different organisational and institutional settings with nominations being sought from national professional bodies (Faculty of Accident and Emergency Medicine, Royal College of General Practitioners, Royal College of Nursing), and the London Ambulance Service. As a result, two accident and emergency department consultants, two accident and emergency department nurses, two general practitioners, and two paramedics were recruited, with one of each being employed in London and the West Midlands. All worked in urban settings.

Rating instrument

At the first two meetings of the panel the nominal group technique²¹ was used, facilitated by EG, to develop and refine a set of questions for rating the safety and appropriateness of the telephone triage decision. The panel was instructed that “the classification of need for an emergency ambulance should be based on the patient’s clinical condition as documented in ambulance, accident and emergency department or general practice records available for this episode”. Following presentation of and discussion about the study’s background, panellists were asked independently to develop and write down criteria that they would use for judging the appropriateness of the assessment and advice that had been provided by nurses and paramedics. These ideas were gathered systematically, followed by a group discussion to clarify them. Panellists then privately ranked each idea, the ranking was tabulated and presented, and the overall ranking was discussed and re-ranked. The final rankings were tabulated and used to develop an assessment schedule which was then discussed and approved by all members of the panel (Appendix 1).

The schedule was piloted on 10 cases and a high level of agreement was found between the panel’s ratings for safety, with at least 75% of the panellists agreeing for seven cases. Panellists found that they were unable to judge the appropriateness of the triage decision for cases where accident and emergency department or GPs’ notes were unavailable. For this reason, only cases with this information available were included in the main assessment process.

Assessment of cases

Of the 330 cases triaged in the main trial¹⁹ into categories indicating an absence of need for an emergency ambulance response, 239 (72%) cases had sufficient data to meet the requirements of the review process. The remaining 91 (28%) cases had either not attended an accident and emergency department or been in contact with a GP or, if they had attended, clinical records about this contact could not be retrieved.

Panellists were blind as to whether the call had been assessed by a nurse or a paramedic, and were unaware that they were only to receive cases where patients had been triaged as not requiring an emergency ambulance. For the purpose of confidentiality all cases were rendered anonymous.

The staticised group technique²² was used with panellists working independently using the agreed assessment schedule.

Reliability

After collation of the data from ratings made by panellists in their initial assessment of cases, a second review was undertaken to reassess all cases where one or more panellists indicated that they believed an emergency ambulance should have been despatched. Additional questions were included to clarify panellists’ perceptions about the level of risk to the patient and the reason(s) for ambulance need. Intrarater reliability was tested on a further 10 randomly selected cases.

Panellists were unaware of the findings of the first round and, as in the first round, worked independently. In addition to the information made available on each case for the first review round, the panel was also provided with the call reports produced by the decision support system for each of these cases. This was to enable comparison of the information documented during the telephone assessment with the clinical findings documented subsequently, and so make an assessment of the reliability of the initial assessment.

Statistical analysis

For the purpose of analysis the patients triaged as not requiring an ambulance—that is, those triaged as either “moderate”, “routine”, or “home care advice”—were re-coded into a single group. The analysis is based on the majority views of the panel (that is, at least five in agreement). The kappa statistic was used to test the reliability of majority views over the first and second assessment for cases rated as having required an emergency ambulance. Percentage agreements were calculated for intrarater reliability.

Case studies

The final stage of the review process involved analysis of transcripts of audiotaped recordings of any call for whom a majority of the panel (at either the first or second review) judged that an emergency ambulance should have been despatched to arrive within 14 minutes. This was to confirm whether or not mis-triage had occurred and to identify the extent to which these patients’ needs had been identified during the telephone consultation.

The recordings were transcribed by ambulance service personnel and rendered anonymous. The transcripts were summarised by the researcher (SW) and the case transcripts and transcript summaries were sent to the chair of the review panel (EG) to confirm that the summaries were an accurate account of each call and to make an overall assessment of the safety of the triage decision.

Ethical approval

Ethical approval was granted by the multicentre regional ethics committee for South Thames and the 39 local research ethics committees responsible for the populations covered by the two ambulance services.

RESULTS

Table 1 indicates that there were no significant differences in age, sex, or presenting problems between the cases reviewed by the panel and those for whom insufficient data were available. However, there was a significant difference in the care pathway, with 192 (80.3%) patients in the group sent to the panel having attended an accident and emergency department compared with only 12 (14.0%) in the group for whom insufficient data were available to allow review by the panel.

Review panel findings: initial assessment

The initial assessment round achieved a 100% response rate from panellists. As shown in table 2, in 231 cases (96.7%) the majority rated the patient as not needing an emergency ambulance.

There were 35 cases (14.6%) for whom at least one member of the panel classified the patient as needing an ambulance within 14 minutes. Table 3 provides details of these cases. These included four (1.6%) where a majority of the panel classified the patient as needing an emergency ambulance within 14 minutes (these cases are described in table 4), but for none of these was this view held unanimously. In addition, there were four cases (1.6%) without a majority—that is, the panel was split 4:4 over the patient’s need/absence of need for an ambulance within 14 minutes.

One member of the panel (an accident and emergency consultant) appeared to have a lower threshold for rating patients as needing an emergency ambulance and for 12 (34.2%) of the 35 cases rated as needing an emergency ambulance he/she was the only panellist to state that this was necessary.

Review panel findings: second assessment

The panel were asked to re-review independently the 35 cases for whom at least one panellist had classified the patient as needing an ambulance within 14 minutes (table 3). The results of a cross tabulation revealed two cases (nos 1 and 2, table 4) who were reported at both assessment rounds by a majority of the panel as needing an emergency ambulance within 14 minutes.

There were 28 cases where the majority rated the patient as not requiring an emergency ambulance on reassessment and there was no majority view on the remaining five cases. This suggests good reliability from one assessment to the next (κ = 0.88, p<0.0001), although the subsample of patients reassessed was not entirely representative of the full sample as they consisted of cases who had been identified by at least one panellist on the first assessment as being in need of an emergency ambulance.

The intrarater reliability results for the review panel on 10 randomly selected cases showed the following percentage agreement between the first and second assessments: (1) A&E consultant, 60%; (2) A&E consultant 40%; (3) GP, 100%; (4) GP, 90%; (5) paramedic, 90%; (6) paramedic, 90%; (7) A&E nurse, 70%; (8) A&E nurse, 80%. There were generally high levels of agreement between the two assessments for the majority of the panel, the exceptions being the two accident and emergency consultants.

Level of risk

For the 35 cases sent for re-review, panellists were asked to grade the level of risk to the patient’s health had an emergency ambulance not been despatched. There were four cases where at least one panel member thought the patient would have been at “life risk” without an emergency ambulance (patients nos 1, 2, 4 and 20; table 3), but none where a majority of the panel felt that a patient would have been at “life risk”.

In table 5 the risk assessment results for the two cases for whom a majority of the panel felt an emergency ambulance should have been sent at both the initial and second review are presented. For neither case did the majority of the panel view that the patient’s life would have been put at risk had an emergency ambulance not been despatched. Overall, the nurse and paramedic members of the panel rated patients as having potentially been placed at greater risk than did the medically qualified panel members.

Case studies

The final stage of the review process involved analysis of transcripts of the audiotaped recordings of the consultations of the four cases described in table 3. As the summaries show in Appendix 2, it appears that in case 1 the need for an urgent medical assessment of the patient (he had fallen) had been recognised in the consultation, but the advice had been given that the patient’s GP might be best placed to do this in the first instance. Similarly, for case 4 the need for an urgent medical assessment was recognised (the patient had been unwell and deteriorating for several weeks), but again it was felt that this assessment should have been undertaken by a GP as he appeared to be stable at the time of the call to the emergency service.

The call transcripts showed that in neither of the remaining two cases was the caller advised that the emergency ambulance was not required. For case 3 the caller was specifically advised that hospital care was required and that he should not move until the ambulance crew arrived. It is therefore probable that, for cases 2 and 3, the triage decisions were incorrectly entered into the decision support system and the intended triage decision was that the patient needed urgent care.

DISCUSSION

The findings from this study support the view that telephone assessment, triage, and advice of non-urgent ambulance service callers may be a safe alternative to despatch of an emergency ambulance. An independent expert panel reviewed a range of data from the clinical records of 239 patients who had been assessed by nurses and paramedics using clinical decision support as not being in need of emergency ambulance despatch. The panel members, although blind to the decisions reached by the nurses and paramedics, showed a very high level of agreement with the decisions that had been reached and, for 96.7% of reviewed cases, the majority of members agreed with the view that emergency ambulance despatch was not needed. There were initially four cases (1.6%) that were rated by a majority of members as needing an emergency ambulance, and a further four (1.6%) where the panel was evenly split. However, on secondary review the number felt to need an emergency ambulance dropped to two cases (0.8%), but there were no cases where a majority of the panel considered that the patient’s life would have been at risk as a result of emergency ambulance despatch having been delayed as a result of the intervention. Analysis of transcripts revealed that for both of these cases it appeared that the triage decision had been incorrectly recorded—that is, from the transcript it was evident that the caller had been advised of the need for an emergency ambulance and that one was on its way. Hence, overall there was a very high level of agreement with the triage decisions made in the trial.

The review process revealed that it is often far from straightforward to decide from ambulance service, hospital, and general practice records whether or not an emergency ambulance is necessary. As in much of primary care, a clear diagnosis was often lacking. The purpose of telephone assessment of category C calls is to decide what response is likely to be in the patient’s best interests, given the information that can be elicited about the patient’s condition and knowledge about the services that are available. For example, an elderly person who falls from his/her chair would require assessment (face to face) and assistance back into the chair. It is questionable whether the immediate despatch of an emergency ambulance travelling with “lights and sirens” and manned by a paramedic is warranted, but telephone advice alone in this situation is unlikely to be sufficient. On the other hand, a patient who has been unwell and deteriorating for several weeks (as in case 4, Appendix 2) might reasonably be considered to be best assessed by a GP familiar with their history unless their condition had clearly reached a point where emergency intervention was necessary, even though there is a likelihood that, following such assessment, they might be referred for admission.

The group of emergency ambulance callers who can best benefit from telephone advice needs to be more clearly defined through further evaluation and research. Priority despatch systems enable lay call takers to classify the urgency of patients’ needs according to simple algorithms to ensure that those with the most urgent needs receive the swiftest response. They are designed to have high sensitivity for life threatening needs but tend to have low specificity; only a relatively small proportion of patients classified as category A tend to have life threatening needs. Just as there may be many category C calls for whom telephone advice is not suitable, there may be some calls assessed by call takers as being category B (“urgent”) for whom telephone advice could offset the need for attendance by an emergency ambulance. The potential for refining priority despatch systems and for offering telephone assessment and advice to different types of callers should be explored.

In this study the clinical panel was only asked to consider the patient’s clinical condition in their assessment of need for emergency ambulance attendance. In the real world, in addition to clinical indices, environmental and/or social circumstances influence the need for hospital admission. It is not clear from this study how these other factors might impinge on the triage decision making of nurses and paramedics in a live situation.

Methodological limitations

In interpreting these findings there are a number methodological limitations that need to be considered. As previously discussed,¹⁹ it was beyond the scope of this study to consider the extent to which the nurses and paramedics involved in the main trial were representative of their professional groups, or the extent to which outcomes were dependent on aspects of the computerised decision support system used. At the time of the study the decision support system was in wide use in the UK and had been the subject of published studies that had demonstrated its effectiveness and safety in out of hours care.^18,23,24

In terms of the clinical review process, 91 cases (28%) were excluded because of a lack of information. This was because the patient had not attended an accident and emergency department or GP following the emergency ambulance call—that is, there was no clinical information available against which to judge the safety of the triage decision—or because clinical records could not be retrieved. There was no evidence to suggest that any of these cases had experienced serious adverse consequences, but this could not be ruled out.

The validity of the clinical information available to the panel also needs to be considered. The data were derived from a variety of sources to provide an overview of the clinical needs of each patient at the time that the emergency ambulance service was called. In addition to patients’ clinical needs, such records are likely to reflect a variety of other factors including the experience and grade of the consulting clinician. For example, junior medical staff in accident and emergency departments tend to over-investigate, treat, and refer for admission.²⁵

The panel was composed of people who were expert in the appropriate area, were suggested for their experience and interest by professional organisations, had local knowledge of the health services being studied, and were likely to have credibility with their peers.^26–28 The extent to which differences in panellists’ views reflected professional or other factors could not be judged, although there was some evidence of variation according to professional group in the ways that risk were assessed, with the GP members of the panel being the least risk averse. The differences in views between individual professionals, between professional groups, and between individuals over time are in themselves interesting and reflect the difficulty of working in an area like this without an agreed gold standard. It is possible that the results would have differed if the panel had consisted of members from only a single specialty, or simply different panellists.²⁹ As the purpose of the study was to identify whether the intervention was safe enough to be recommended for full implementation in a live trial, we felt justified to focus on the majority view rather than on the views of individual panellists.

In conclusion, the results from this study suggest that the provision of computer assisted telephone assessment and advice within the context of a trial may be a safe method of responding to category C emergency ambulance service calls. There was no evidence that safety had been seriously compromised. However, the disagreement observed between panellists about the need for attendance of an emergency ambulance leaves some uncertainty about the definitiveness of the conclusions of this study. Furthermore, the possibility of rare adverse events cannot be discounted, particularly given the extent to which there were missing data. The findings need to be interpreted within the context of the setting, the method of training of the paramedics and nurses, and the particular decision support software. It is now national policy that NHS Direct, the national nurse led telephone advice service, will in future triage emergency calls that are prioritised as being of a non-urgent nature.³⁰ We consider that widespread adoption will be premature without further evaluation of safety. The findings reported here provide sufficient evidence of safety to support undertaking a full trial.

APPENDIX 1: ASSESSMENT OF PATIENT’S NEED FOR AN AMBULANCE

APPENDIX 2: AUDIOTAPED RECORDINGS OF THE CONSULTATIONS OF THE FOUR CASES DESCRIBED IN TABLE 4

CASE 1 (69 YEAR OLD MAN LIVING IN RESIDENTIAL HOME WHO HAD FALLEN);

Caller

The caller was a district nurse who was visiting the patient. She did not want to move the patient until help arrived and did not have anyone to assist her. She had not spoken to the patient’s GP prior to calling the ambulance service.

Presenting problem

The patient had been found on the floor of a residential home and was conscious but not responding verbally. There were no apparent injuries. He was not having difficulty breathing. The district nurse thought that he had mobility problems and had previously suffered a stroke.

Assessment and advice

On receiving the call from the call taker the advisor first reassured the caller: “Hello it’s [name], the nurse. The ambulance is on the way.” The nurse established the age of the patient and contact details and carried out telephone assessment.

After completing assessment of the patient, the nurse suggested the caller might contact a colleague (another district nurse) to attend to help lift the patient into a chair. The caller stated that this was not possible because of nurses being off sick, and the nurse again reassured the caller that the ambulance was on its way.

The nurse went on to suggest that it would have been more appropriate to have contacted the GP, and asked “..have you spoken to his doctor?....morning surgery would have finished and I’m sure the doctors will be off doing their visits and one can come round and examine him.” The caller stated that she did not know whether they would attend. The nurse suggested that it was not really appropriate in the first instance for an emergency ambulance, but again reassured the caller that “there is an ambulance on the way, so the team will be there shortly and they will certainly be able to help you.”

The nurse also asked if there was any lifting equipment in the house. The caller explained that there was no equipment. The nurse asked about neighbours and the caller explained that they were all elderly. The nurse responded by reassuring the caller: “Right OK, and you can’t get him up. Well I think really then that this was the only option wasn’t it, to call someone for help? But you’ll probably find when the ambulance arrives that they’ll suggest you call the GP.” The nurse stated that the patient needed to see a doctor but that it would probably be better if he saw his own GP. The caller said that she would call the doctor.

Total word count of call: advisor = 499, caller = 381.

CASE 2 (2 Year Old Child With A High Fever)

Caller

The caller was the mother of the child. Before calling the ambulance service the mother had contacted the GP and, since the GP could not attend immediately, the doctor told her to call for an ambulance.

Presenting problem

The child’s temperature was high and, according to the mother, she had just started shivering and shaking, but not fitting. Her eyes were not rolling and her breathing was fine. She had a stomach ache and a headache. The child was also crying. She did not have cold symptoms. At the time of the call the child could not keep Calpol (paracetamol) down, but was not vomiting. The child was thirsty and drinking frequently. Her bowel movements were normal. The patient had not been taken abroad recently and had no history of convulsions. The stomach ache had started 5 days previously and the child had been seen by a doctor then and had been diagnosed as having a viral gastric condition. There had been no prescription of antibiotics given. The child had also been seen by the GP at 18.00 hours the previous evening with the stomach ache and a temperature of 104. A urine sample had been taken. The mother had been able to control the child’s temperature through the evening with Calpol.

Assessment and advice

On receiving the call from the call taker the advisor said “Hello there, paramedic here.” The mother explained that she had been told by the GP to call for an ambulance. After taking additional history, the paramedic reassured the mother that the ambulance was on the way: “… there’s an ambulance on its way, we’re not delaying the ambulance, don’t worry”. The paramedic through the course of the assessment reassured the caller at times and repeated that there was an ambulance on its way.

Whilst carrying out the assessment the paramedic provided home care advice and information on regulating the child’s temperature (e.g.“Kids … are not brilliant at controlling their own body temperature…so we need to, don’t overheat her but don’t overcool her, you just need to make sure she’s OK and not too hot but not too cool OK … is she dressed? “She’s got her pyjamas on, that’s fine, alright well don’t warm her up any more than that OK.”).

Total word count of call: advisor = 1082, caller = 491.

CASE 3 (83 Year Old Woman Who Had Fallen)

Caller

The caller was the patient who was alone at home at the time of the call.

Presenting problem

She had fallen in the hallway of the house after passing out within the previous 2 hours. The patient was on her way to the toilet and had urinated on herself after the fall. She was unable to get up or move and had great pain high up in her right leg but could move her toes. There was no bleeding and no injury to her head. The patient was dressed only in night clothes, the heating was turned off, and she was very cold. She did not have a history of falls and had no difficulty breathing.

Assessment and advice

The advisor explains that the caller was speaking with a paramedic. The paramedic explained that there would be no delay in the ambulance arrival and through the course of the call further reassurance was given to the patient that the ambulance was on the way.

The paramedic advised the patient not to move her leg. The paramedic explained “We’ll wait for the ambulance to get there and pick you up. We’ll let them assess you properly..to make sure you’ve not hurt your leg …”. The paramedic further explained “… the ambulance crew will have a look at you and sort you out …and they’ll probably pop you into hospital.” The paramedic asked if she could open the door and said if there was any difficulty to let the ambulance service know and they would find another way of getting into the house. The paramedic also asked many questions unrelated to making a decision on the need for an ambulance.

Total word count of call: advisor = 662, caller = 335.

CASE 4 (65 YEAR OLD WOMAN WHO HAD BEEN SHOWING INCREASING SELF-NEGLECT OVER PREVIOUS WEEK)

Caller

The caller was the daughter of the patient and was visiting the mother’s house at the time of the call. The patient lived alone. The caller had been going to call the GP 2 days earlier but the patient requested her not to call the doctor. The patient had not wanted her daughter to call for an ambulance.

Presenting problem

The patient had reportedly not eaten over the previous 2 weeks and appeared to be neglecting herself. She could speak but was unresponsive and moribund. The patient had been deteriorating over the past 4 weeks but particularly over the previous week. She was lethargic and had only been drinking cold drinks. She appeared to have lost weight and was weak. The patient had been depressed since moving house nearly 2 years previously. She had not seen a doctor or been out of the house for over a year. The daughter did everything for her mother. The patient was not on medication and had not been suicidal or shown signs of self-neglect in the past.

Assessment and advice

The advisor said to the caller “It’s OK, you’re through to a nurse”. The advisor suggested that the patient may suffer from agoraphobia and explained: “it’s a condition where people don’t want to go outside”. The caller agreed with this. The nurse also suggested the patient may be depressed and the daughter confirmed this and reported that she was also herself suffering from depression for which she was on medication. The nurse suggested that the patient may have “just lost interest in everything” as a result of depression.

The nurse explained that the patient needed to see her own GP who could provide nourishing fortified drinks to build up the patient’s strength. She suggested that, if the patient was taken to A&E by ambulance, she could be waiting a long time to be seen and this might be off-putting to the patient. She suggested that the caller should take the patient to the GP in future. The advisor instructed the caller to contact the patient’s own GP the next day.

Total word count of call: advisor = 419, caller = 751.