Article Text

PDF

Developing a patient measure of safety (PMOS)
  1. Sally J Giles1,
  2. Rebecca J Lawton1,2,
  3. Ikhlaq Din1,
  4. Rosemary R C McEachan1
  1. 1Quality and Safety Research, Bradford Institute for Health Research, Bradford, UK
  2. 2Institute of Psychological Sciences, University of Leeds, Leeds, UK 
  1. Correspondence to Dr Sally J Giles, Quality and Safety Research, Bradford Institute for Health Research, Temple Bank House, Bradford Royal Infirmary, Duckworth Lane, Bradford BD9 6RJ, UK; sally.giles{at}bthft.nhs.uk

Abstract

Background Tools that proactively identify factors that contribute to accidents have been developed within high-risk industries. Although patients provide feedback on their experience of care in hospitals, there is no existing measure which asks patients to comment on the factors that contribute to patient safety incidents. The aim of the current study was to determine those contributory factors from the Yorkshire Contributory Factors Framework (YCFF) that patients are able to identify in a hospital setting and to use this information to develop a patient measure of safety (PMOS).

Methods Thirty-three qualitative interviews with a representative sample of patients from six units in a teaching hospital in the north of England were carried out. Patients were asked either to describe their most recent/current hospital experience (unstructured) or were asked to describe their experience in relation to specific contributory factors (structured). Responses were coded using the YCFF. Face validity of the PMOS was tested with 12 patients and 12 health professionals, using a ‘think aloud’ approach, and appropriate revisions made. The research was supported by two patient representatives.

Results Patients were able to comment on/identify 13 of the 20 contributory factors contained within the YCFF domains. They identified contributory factors relating to communication and individual factors more frequently, and contributory factors relating to team factors, and support from central functions less frequently. In addition, they identified one theme not included in the YCFF: dignity and respect. The draft PMOS showed acceptable face validity.

Discussion Patients are able to identify factors which contribute to the safety of their care. The PMOS provides a way of systematically assessing these and has the potential to help health professionals and healthcare organisations understand and identify, safety concerns from the patients’ perspective, and, in doing so, make appropriate service improvements.

  • Health services research
  • Patient safety
  • Patient-centred care
  • Safety culture

Statistics from Altmetric.com

Introduction

The elicitation of feedback from patients about satisfaction with their care or their experience of care is relatively well established. Patient satisfaction surveys1 ,2 are often criticised for producing mostly positive ratings from patients3 which are not comparable with the lower levels of satisfaction revealed through interviews with the same patients.4 ,5 More recently, measures of patient experience have been developed to capture data on specific aspects of healthcare processes and events.6–8 While patient experience measures such as the widely used Picker Patient Experience Survey7 ask some questions that are relevant to patient safety (eg, about medication side effects and communication with patients), to date, no tool has been developed that asks patients to provide feedback on the safety of their care, particularly as a way of capturing information that can be used as a basis for improving safety at ward/unit level. There is growing evidence, however, that patients can be an important source of knowledge in reducing avoidable harm and improving healthcare.9–11

Previous research has demonstrated that, while patients may be reluctant to directly challenge healthcare professionals,12 they are willing to engage in patient safety initiatives. This engagement includes providing feedback about their experiences of care via surveys, but also includes identifying factors that contribute to patient safety incidents.13 A recent systematic review14 generated a comprehensive taxonomy of the factors contributing to patient safety incidents, the Yorkshire Contributory Factors Framework (YCFF) (figure 1). These include factors such as physical environment, communication, leadership and teamwork. Using the YCFF as a starting point, this study aimed to explore the extent to which patients are able to provide feedback about the contributory factors represented in this framework.

Figure 1

The Yorkshire Contributory Factors Framework.

A second aim was to develop indicators of each of these contributory factors in the form of questionnaire items and to test the face validity of this questionnaire (Patient Measure of Safety or PMOS) with staff and patients. These aims were achieved iteratively in two stages. Stage one involved a series of qualitative interviews with patients to identify which contributory factors they were able to identify, using the YCFF as a basis. These were used to inform the development of items for the PMOS. Stage two involved testing the PMOS with health professionals and patients using a ‘think aloud’15 approach. A multidisciplinary panel of experts including policy makers, health professionals, academic researchers and patients informed the design of the study, and the patient panel aided the development of the questionnaire.

Methods

Participants

Patients in stage 1 (interviews) and patients and health professionals in stage two (think-aloud) were recruited from six units (maternity, renal, physiotherapy outpatients, vascular surgery, ear, nose and throat, and cancer services) in a National Health Service trust in the north of England. These units were selected and purposive sampling to ensure the views of a broad range of patients was elicited (eg, those with regular interaction with the health service (renal patients) and those with relatively short one-off stays (maternity), young and old, men and women and patients of different ethnic backgrounds). The data were collected by two researchers, one of whom was able to speak Mirpuri (a dialect spoken by three-quarters of the Pakistani population in Bradford16).

Procedure

Ethical approval for this study was gained from a local Research Ethics Committee (ref: 09-H1302-115). The PMOS was developed in two stages. The process is described below.

Stage 1—qualitative interviews

Qualitative interviews were used as the basis for identifying which contributory factor domains patients could identify, and for developing PMOS questionnaire items. The project steering group, consisting of a multidisciplinary panel of experts was consulted in the development of interview schedules. This panel recommended that while the YCFF14 (figure 1) could be used to define the interview questions, some of the interviews should take a more unstructured approach to ensure that the views of the patients were fully represented and not constrained by an a priori framework. Thus, two approaches to interviewing were used: unstructured and structured. In the first case (unstructured) interviews (n=18) were based on a narrative approach17 where participants were asked to describe their most recent/current hospital experience. These interviews were preceded by three pilot interviews. Participants were asked to describe their hospital experience with an emphasis on patient safety. However, during the pilot interviews it became clear that using the term ‘patient safety’ (which was not familiar to many patients) appeared to discourage participants from engaging in the interview. Thus, it was decided to omit the phrase ‘patient safety’ from any subsequent interviews, for the interviewer to explore any experiences that related to patient safety (eg, delays in waiting for medication, insufficient information given to patients regarding their condition/treatment/procedure, delays in treatment/procedures/operations, poor communication), should participants describe such experiences. In the case of the structured interviews the patient panel for the project was consulted and asked to select which of the contributory factors (contained within the YCFF) they felt that patients would definitely not be able to identify/comment on. Based on this assessment, patients were not asked to comment on safety culture, policy and procedures, external policy context, task characteristics and design of equipment and supplies. Active failures were also excluded from the structured interviews as the PMOS was designed to assess those factors contributing to error, but not the errors themselves. The structured interviews (n=15) asked patients, which of the remaining 13 factors they felt that they were able to comment on or were in a position to notice and/or make judgments about. They were encouraged to provide examples based on their own experience.

Patients (except those deemed too unwell by staff or those having undergone a general anaesthetic in the preceding 24 h) were approached in the ward by the researcher who explained the study and gave them an information sheet. Willing participants were then consented. Where possible, interviews took place in the unit, often at the bedside. Some interviews were conducted elsewhere, for example, in the hospital canteen or in the waiting room. Interviews ranged from 15 min (for the structured interviews) to 2 h for the unstructured interviews. The interviews continued until no new themes emerged and theoretical saturation was achieved.18 All interviews were recorded using a digital recorder and fully transcribed.

Stage two—think aloud

The ‘think aloud’15 process took place after the PMOS was produced. The purpose of this phase of the research was to test the face validity of the PMOS. Twenty-four ‘think aloud’ interviews (12 patients and 12 health professionals) were conducted in the six units described above. As staff and patients are potential users of the survey, the views of both groups were important. Participants were asked to talk aloud about their thoughts and feelings as they read, and decided how to respond to each question in the draft PMOS. Participants were also asked to comment on: (1) perceived barriers to completing the PMOS, (2) the timing of completion during the care pathway and (3) the questionnaire format. Minor revisions to the PMOS were made following the ‘think aloud’ procedure.

Recruitment of the patients took place within the six units in the same way as for stage 1. The health professionals were identified using existing contacts within each of the areas. These think aloud sessions were arranged via email or telephone and took place at a location chosen by the participants. Interviews were between 10 min and 30 min long. All interviews were digitally recorded and transcribed in full.

Data analysis

Stage 1—qualitative interviews

Interview transcripts were imported into NVivo 8 and then coded using content analysis.19 The YCFF was used as a coding framework with a particular focus being on the 13 domainsi deemed relevant to patients. The frequency with which each of the domains was coded was recorded. Any text that could not be coded using the YCFF was coded separately and new themes created. To begin, three members of the research team (SG, RL and ID) reviewed three of the transcripts each in order to ensure there was consensus in the interpretation of the factors within the YCFF. Following this, the remainder of the transcripts (n=30) were divided equally between two of the researchers (SG and ID), who used the above process to code the transcripts.

Stage two—think aloud

The ‘think aloud’ transcripts were imported into NVivo 8. Two of the researchers listened to the recordings and read through the transcripts (SG and ID) to identify and code comments made when completing the draft PMOS. In addition, both researchers collated the responses to the short questionnaire that took place following each ‘think aloud’ session. These, together with the transcripts, informed any changes that were made to the PMOS questionnaire. The final PMOS was then tested for readability using the Flesch Reading Ease and Flesch-Kincaid Grade level.

Results

Participants –stage 1

All participants approached for interview at both stages agreed to take part. This high uptake was due to patients who were too unwell or who did not have the capacity to complete the interview being filtered out by staff. In the stage 1 interviews, participants ranged from 18–83 years and included 14 male patients and 19 female patients (table 1).

Table 1

Interview participants phases 1 and 2

Stage 1—qualitative interviews

Can patients identify contributory factors within the YCFF?

Table 2 shows the number of times patients highlighted a contributory factor domain and gives examples, in the form of interview excerpts, of the way in which patients talked about these factors. As expected, none of the five domains excluded by the expert panel were identified during the qualitative interviews. All remaining 13 domains were identified by patients. Domains most frequently identified were ‘communication’ (identified 80 times) and ‘individual factors’ (in relation to staff, for example, stress, and patients, for example, attitudes of carers, identified 62 times). Domains such as ‘team factors’ and ‘support from central functions’ were identified less frequently. Although the domain ‘active failures’ was excluded from the structured interviews, participants did identify active failures during the unstructured interviews. There appeared to be no major differences in the nature of information collected using the structured and unstructured approach to the interviews. A further theme not captured within the YCFF was identified: dignity and respect. Little is known about the relationship between 'dignity and respect‘ and patient safety outcomes, but it is an important factor in patients’ experience of their care, being associated with feeling comfortable, in control and valued.20 Although not identified as a contributory factor within the literature, our patient panel members felt that patients who were not treated with dignity and respect may be reluctant to communicate important information about their wellbeing (including symptoms) to health professionals which might compromise their safety.

Table 2

Frequency of contributory factors domains identified, and number of patient measure of safety items developed

All these domains, with the exception of ‘active failures’ (deemed to be an outcome rather than a contributory factor) were used as a basis for the PMOS.

Developing the items for inclusion in the draft PMOS

The interview transcripts (structured and unstructured) were used to develop the individual items for each domain. For some domains, particularly those which patients referred to less in the interviews, only a small number of questionnaire items were necessary to adequately represent the codes (eg, training was represented by two items). For other domains, such as communication, which was a broad category, more items were necessary to reflect the assigned codes. Forty-two questionnaire items were developed in total. A mixture of positive and negative items were included in order to avoid acquiescence response set bias (the tendency to give the same response to questions where the direction of wording is the same21), and items were designed to be responded to using a Likert scale. Two pairs of domains, ‘management of staff and staffing levels’ and ‘staff workload’, and ‘patient factors’ and ‘individual factors’ were merged based on data from the interviews which suggested that patients did not distinguish between these domains. One item was also included in the questionnaire, under the heading of ‘dignity and respect’ to capture this aspect of the safety of care.

Consulting with the research team and patient panel

The items in the draft PMOS were developed and revised based on discussions with the research team (consisting of psychologists, social scientists and clinicians) and the patient panel for the project to maximise content validity.22 A small number of changes were suggested by both groups.ii This resulted in a draft version of the PMOS to be used in stage 2, the ‘think aloud’ process.

Stage 2—think aloud

All those involved in the ‘think aloud’ process (n=24) felt that, on the whole, patients would be interested and willing to complete the PMOS. Participants identified some potential barriers to completion of the PMOS, such as eyesight, language and age (ie, elderly patients finding it a challenge).

During the ‘think aloud’, the majority (n=20) of participants were able to complete the questionnaire within 15 min and were able to understand and respond to the items. The same participants also stated that they were satisfied with the length of the PMOS. Only a small number expressed concerns with understanding or found it difficult to respond to questions (see online supplementary appendix 1).

There were three key areas of concern that arose during the ‘think aloud’ process. First, negative statements appeared to present a problem for a small number of participants (n=5). Second, some participants (n=2) were unfamiliar with terminology used within the questionnaire (eg, care plan). Finally, some patients found it difficult to answer questions that they had no experience or knowledge of, but were often able to use the ‘not applicable’ option if this was the case. These issues were discussed and addressed in consultation with the patient panel, and where necessary, revisions to items were made.

Finalising the PMOS

Following the ‘think aloud’ process and consultation with the patient panel, some changes were made to the draft PMOS, including the format (ie, more space between questions and space to comment after each) and changes to some of the wording used in the questionnaire (eg, care plan was changed to plan of care). Once the changes to the draft PMOS had been made, this resulted in the main outcome of this study, the PMOS (see online supplementary appendix 2).This was then tested for readability using the Flesch Reading Ease (65.7%, ie, easily understandable by 13–15-year-old students) and Flesch-Kincaid Grade level (6.9, ie, indicates that the text is expected to be understandable by an average student in the 6th grade) tests.

Discussion

The current study is innovative, in that it clarifies the types of contributory factors that patients can identify in hospital settings, and it details the development of the first healthcare questionnaire to assess safety from the perspective of the users of healthcare: patients. The fact that patients identified and described contributory factors from 13 domains of the YCFF also provides evidence of the face validity of this framework. Patients were best able to identify contributory factor domains inherent in ‘local working conditions’ (eg, communication, availability of equipment) as these are often issues that a patient can explicitly observe while on a ward. Those contributory factors at a more upstream organisational level (eg, six of the 20 original domains) appear to represent factors that patients do not recognise as impacting directly on their safety or, even more generally on their hospital experience (eg, policy context or design of equipment). Being outside the scope of experience for most patients these factors were not measured within the PMOS. However, a similar questionnaire could be designed for staff which included items to tap these more upstream contributory factors. In addition, the study identified one further area which patients felt was strongly linked to safety in hospital settings: ‘dignity and respect’, therefore confirming that patients are able to give a more holistic view of issues relating to their safety. Future work using the PMOS will allow the relationship between ‘dignity and respect’ and the other contributory factors to be investigated.

The PMOS is designed to be a useful diagnostic tool which, when used at a ward level, has the potential to allow health professionals to proactively identify areas of strength and weakness, and intervene to prevent errors occurring. To date, patients have had minimal opportunity to input into traditional incident-reporting and risk procedures for managing safety despite being able to offer valuable feedback.13 ,23 In addition, unlike other patient reported outcome measures7 patients have played a more central role in its development, as research participants and part of the research team. With the increasing emphasis on patient reported outcome measures in the UK24 and USA,25 the PMOS provides a useful and timely addition to current tools which focus on safety culture.26 The PMOS is part of a larger programme of work27 which will explore the use of the tool, combined with patient incident reporting, as a mechanism for improving safety within wards.

The potential role of patients in promoting patient safety has been emphasised,28–31 particularly for those patients who have ongoing treatment and may be better placed to identify errors or lapses as they become more knowledgeable and familiar with the details of their care.30 PMOS is a specific example of how this role can be realised and how patients have the potential to provide valuable quality and safety improvement data at a micro level. It is clear from this study that patients could identify contributory factors within the hospital setting with ease. In fact, all patients identified at least 10 different factor domains. It is hoped that by providing a mechanism for this information to be systematically gathered, the PMOS will provide a useful source of information which service providers can use to aid organisational learning. Indeed, the PMOS could be seen as an adjunct to other patient safety tools, such as incident reporting systems. Current information on quality and safety comes predominantly from incident reporting systems that rely on healthcare professionals to report, and suffer from high levels of underreporting.32

Limitations

The development of the PMOS was based on the YCFF, which, in turn was based on a review of studies conducted almost exclusively with healthcare professionals. This means there is a danger that PMOS may not truly reflect the views of patients if patients tend to identify different contributory factors. However, taking an unstructured approach to half of the qualitative interviews meant we were not constrained by this a priori structure. It is worth noting that, aside from the ‘dignity and respect’ domain, all contributory factors patients identified mapped onto the YCFF model.

In addition, the PMOS contains only those domains from the YCFF that patients were able to comment on, which means that five of the domains within the YCFF are not represented. Although a major strength of the PMOS is that it allows identification of contributory factors from a patients’ perspective, it must be recognised that it does not capture all possible contributory factors. It must therefore be used in conjunction with other tools, such as staff safety culture surveys and Failure Modes Effects Analysis,33 to explore upstream organisational failures.

PMOS may not represent the views of the general population internationally because despite being developed with a diverse ethnic population, the sample was drawn from a single hospital in the north of England. However, future work will explore the utility of the PMOS with a larger number of patients across a number of NHS trusts within England.

Conclusion

Patients are in a very good position to observe the safety of their care and that of others on the same ward/unit and are able to respond to items in the PMOS that are indicative of the safe organisation of their care. In this study we have developed a PMOS that patients can complete during their hospital stay. The next step in this research programme is to test the reliability and validity of the questionnaire and to develop an intervention based on data that are produced to help staff improve patient safety on their wards.27

Acknowledgments

We are grateful to all the patients and health professionals who took part in this study, and to the members of the dedicated ‘Patient involvement in patient safety’ patient panel: Mick Bonallie, Ted Clarke, Mike Conway, Sharin Goodchild, Dave Green, Ashfaq Gulab, Nigat Parveen, Linda Lovett.

References

View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors SG, RL and RM developed the draft and final versions of the PMOS. SG and ID conducted all the qualitative interviews. SG, RL and ID were involved in the analysis of the qualitative interviews. All authors contributed to draft versions of the manuscript. All authors read and approved the final manuscript.

  • Funding This article presents independent research commissioned by the National Institute for Health Research (NIHR) under a ‘Patient Involvement in Patient Safety Programme Grant’. The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

  • Competing interests None.

  • Ethics approval Bradford Local Research Ethics Committee -ref: 09-H1302-115.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • i Communication, individual factors, physical environment, scheduling and bed management, management of staff and staffing levels, staff workload, training and education, lines of responsibility, design of equipment and supplies, equipment and supplies, patient factors, supervision and leadership, team factors and support from central functions.

  • ii These included changes to the format (ie, more space to write comments), size and font of the text.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.