Table 1

Summary of data sources

Source and type of dataDatesNature of full datasetSubset of data analysed for this paperStrengthsLimitations
Safety incident dataset
Safety incidents from NHS England complaintsJanuary 2021 to July 2022Quarterly primary care complaints review slide sets, manually searched for remote encounters.69 complaints relating to remotely delivered care (28 pages total).Verified cases. Large numbers.Skewed sample (only cases with complaints), and data preselected by NHS England, so unable to verify if representative of all such complaints. Safety issues which take years to emerge will have been missed.
Safety incidents from NHS Resolution claimsJanuary 2015 to May 2023Closed claims with an incident description which included at least one of telephone, phone, online, remote or triage. 22 cases identified.16 cases included (160 pages total) (6 cases excluded, 3 because the complaint was unrelated to a remote consultation, and 3 due to insufficient information).In-depth information on cases, often containing clinical records and local investigations such as RCAs.Skewed sample (rare cases involving death or serious harm). Including only closed cases means some occurred several years ago. Complex cases may take longer to reach closure.
Safety incidents from investigation into COVID-19 telephone advice7Mostly March to June 2020; also considered later COVID peaks in 2021/2022National investigation by the Healthcare Safety Investigation Branch into set-up, design and delivery of COVID-19 telephone triage service accessed via 111 and into the patient pathway delivered by 111.Full investigation report (86 pages; 4 cases highlighted as exemplars of wider issues).In-depth investigation and analysis of a specific safety period in the early months of the pandemic.Cases only reflect the earliest period of the pandemic (which was highly atypical) and are skewed towards the most severe examples.
Safety incidents from NHS 1112021–2023List of safety events relating to remote consulting in NHS 111 Wales with case examples provided by a senior clinician after a stakeholder interview.List of key themes in patient safety, illustrated by fictionalised cases (2 pages; 6 cases).National provider; large number of remote consultations, allowing rare cases to be identified.Small number of cases. Procedures and systems for out-of-hours telephone provider differ from other parts of primary care.
Safety culture and safety practices dataset
Multisite longitudinal case study of remote care in general practice2322September 2021 to August 202312 general practices in England, Wales and Scotland followed for 24 months. Ethnography, staff and patient interviews and documents such as websites and leaflets. Data transcribed and coded on NVivo.Fieldnotes and interviews where staff had highlighted safety concerns, fieldnotes relating to safety issues (30 pages total). Includes in-depth ethnographic observation of reception areas where decisions about mode of consultation are made.Detailed ethnographic material; diverse sample. Insights into how and why staff make situated judgements and ‘break the rules’ to protect patient safety.Practice-level information unlikely to capture rare events that may only be detected at scale.
Interviews with out-of-hours providers and trainers2022Interviews with 8 clinical leads working out of hours or in NHS 111 (default modality telephone) in England, Scotland and Wales.Extracts from interviews and training materials shared by these providers (20 pages total).Insights into a system where the default modality is telephone and the service has years of experience and an established system of supervision and training.Out-of-hours primary care is not directly comparable to in-hours general practice.
Multistakeholder workshopSeptember 2022Online workshop with 61 participants including clinicians, national clinical leads, representatives from arm’s-length bodies, practice staff and laypeople. Plenaries and breakout groups videotaped.Interdisciplinary group discussions about safety, presentation by the 111 COVID Clinical Assessment Service, South Central Ambulance Service. 4 hours of video footage (11 pages of transcribed extracts).Very rich and nuanced discussions among large number of participants across multiple sectors. Breakout groups allowed a wide range of views to be captured.Clinical details of cases mentioned in discussions could not be verified (hence were not included in our sample of safety incidents).
Interviews with GP trainers and trainees2023Interviews with 10 GP trainers and 10 GP registrars who had trained during the pandemic, undertaken by recently trained GP.Trainers’ comments on trainees’ ability to assess clinical risk in the remote environment; registrars’ perceptions on their own confidence in this regard.Insights into how the rapid shift to remote had compromised training and supervision of clinicians.Sample only included GPs; study was not resourced to do similar studies on other clinicians.
Expert interview and sense check of draft findings with national stakeholdersMay to June 2023Stakeholders in strategic safety roles and subject matter experts sampled from arm’s-length bodies, government and health boards.Extracts of data from 6 interviews including 3 representatives of arm’s-length bodies, 1 Department of Health, 2 health boards (12 pages total).Informants worked at senior levels with an oversight at system level. Included representation from 111/OOH expert familiar with remote consults.Risk of bias in recall.
  • GP, general practitioner; OOH, out of hours; RCAs, root cause analyses.