You are here

Article Text

Article menu
Original research
Developing a reliable and valid patient measure of safety in hospitals (PMOS): a validation study
  1. Rosemary R C McEachan1,
  2. Rebecca J Lawton2,
  3. Jane K O'Hara3,
  4. Gerry Armitage4,
  5. Sally Giles5,
  6. Sahdia Parveen3,
  7. Ian S Watt6,
  8. John Wright3,
  9. on behalf of the Yorkshire Quality and Safety Research Group
  1. 1Bradford Institute for Health Research, Bradford Teaching Hospitals NHS Foundation Trust, Bradford Royal Infirmary, Bradford, UK
  2. 2Institute of Psychological Sciences, University of Leeds, Leeds, UK & Quality and Safety, Bradford Institute for Health Research, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
  3. 3Quality and Safety, Bradford Institute for Health Research, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
  4. 4School of Health Studies, University of Bradford & Quality and Safety, Bradford Institute for Health Research, Bradford Teaching Hospitals NHS Foundation Trust, Bradford, UK
  5. 5NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, University of Manchester, Manchester, UK
  6. 6Department of Health Sciences/Hull York Medical School, University of York, York, UK
  1. Correspondence to Professor Rebecca Lawton, Institute of Psychological Sciences, University of Leeds, Leeds, LS2 9JT, UK; r.j.lawton{at}leeds.ac.uk

Abstract

Introduction Patients represent an important and as yet untapped source of information about the factors that contribute to the safety of their care. The aim of the current study is to test the reliability and validity of the Patient Measure of Safety (PMOS), a brief patient-completed questionnaire that allows hospitals to proactively identify areas of safety concern and vulnerability, and to intervene before incidents occur.

Methods 297 patients from 11 hospital wards completed the PMOS questionnaire during their stay; 25 completed a second 1 week later. The Agency for Healthcare Research and Quality (AHRQ) safety culture survey was completed by 190 staff on 10 of these wards. Factor structure, internal reliability, test-retest reliability, discriminant validity and convergent validity were assessed.

Results Factor analyses revealed 8 key domains of safety (eg, communication and team work, access to resources, staff roles and responsibilities) explaining 58% variance of the original questionnaire. Cronbach's α (range 0.66–0.89) and test-retest reliability (r=0.75) were good. The PMOS positive index significantly correlated with staff reported ‘perceptions of patient safety’ (r=0.79) and ‘patient safety grade’ (r=−0.81) outcomes from the AHRQ (demonstrating convergent validity). A multivariate analysis of variance (MAMOVA) revealed that three PMOS factors and one retained single item discriminated significantly across the 11 wards.

Discussion The PMOS is the first patient questionnaire used to assess factors contributing to safety in hospital settings from a patient perspective. It has demonstrated acceptable reliability and validity. Such information is useful to help hospitals/units proactively improve the safety of their care.

  • Risk Management
  • Patient Safety
  • Human Factors

https://doi.org/10.1136/bmjqs-2013-002312

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    In order to effectively learn from error, it is important that organisations systematically identify contributory factors which impact on patient safety. In the main, hospitals take a reactive approach to learning from error, relying on incident-reporting systems. These systems have been criticised due to under-reporting1 and a tendency to focus on the proximal causes of incidents.2 ,3 In a recent systematic review of empirical studies reporting contributory factors, Lawton et al,4 found that active failure and individual factors (characteristics of staff, such as tiredness, or lack of knowledge) were disproportionately represented in such systems. Without guidance and adequate training on what constitute contributory factors, individuals may have a tendency to focus on the more proximal determinants of patient safety incidents.4 ,5 While more proactive systems exist to explore systems weaknesses in organisations (eg, Failure Modes and Effect Analysis6) these are often resource intensive, and as such difficult to routinely implement.7 There is a clear need to develop a proactive, low-cost system for identifying and learning from systems weaknesses in hospital settings.

    There is growing evidence to suggest that patients are willing and able to provide feedback on the quality and safety of their care.8–13 This is particularly valuable, as patients can identify safety issues that staff may not notice, be willing or able to report.14 Patients are also uniquely placed to observe the processes of their care (eg, scheduling of procedures), treatment (eg, inappropriate drug administration) and physical environment (eg, temperature, cleanliness).15 Recent reports focusing on patient safety in the British National Health Service (NHS) have highlighted the importance of listening to and acting on patient concerns about patient safety issues.16–18 Despite the increase in tools, such as the NHS inpatient survey19 used to assess patients’ experience of care, there are no such routinely implemented surveys which give patients the opportunity to provide feedback about the safety of their care. This, along with the international impetus for the mobilisation and empowerment of patients with regards to their safety20 and increasing emphasis on the importance of patient feedback as key indicators of patient experience makes the time ripe for the development of a systematic way of collecting safety information from patients.21–23 Furthermore, within the UK, there is a clear call for hospitals to engage patients and collect their feedback on services in real-time, and consistently across different organisations.16 Thus, a tool to allow patients to directly report on their safety is timely and important in the drive for hospitals to proactively manage safety.

    Previous research has explored in detail the extent to which patients are able to identify systems weaknesses. The Yorkshire Contributory Factors Framework (YCFF4) was developed as a tool to help explore latent systems weaknesses based on Reason's model of human error,24 and describes 20 key contributory factor domains. Using the YCFF as an underlying structure, Giles et al15 found that patients were able to identify safety issues related to 14 of 20 key contributory factor domains. The types of contributory factor domains patients were able to identify centred on those that manifest in local working conditions (eg, communication, support from central resources). They were less able to report those that related to higher-order organisational and external factors (eg, external policy context). Based on these findings, Giles et al15 reported the development of a questionnaire tool—the Patient Measure of Safety (PMOS), a brief questionnaire which patients complete during their hospital stay and which provides feedback on these key contributory factor domains.

    The aim of the current study was to test the reliability and validity of the PMOS in a hospital setting. The objectives were to explore the factor structure and internal reliability of the scale (Cronbach's α, test-retest reliability), the extent to which the scale discriminates among wards (discriminant validity), and the extent to which it converges with staff measures of patient safety (convergent validity).

    Method

    Design

    The study used two separate cross-sectional surveys (one with patients, and one with staff) within a large acute teaching hospital in the North of England. Data were collected between 1 September 2011 and 30 November 2011 (10 wards) and 3rd and 5th April 2012 (1 ward). In line with recommendations, a minimum sample size of 250 was considerable acceptable for the patient survey (in relation to planned factor analysis).25 There was no a priori required sample size for the staff survey, although we aimed to sample a minimum of 50% of staff on participating wards.

    Patient survey

    Participants

    Four hundred and two patients or their parents/carers across 11 wards (including medical, surgical, maternity and paediatric) were approached to take part in the study. Of these patients, 344 consented and 297 valid PMOS questionnaires were collected (n=47 patients consented to take part in the study but did not return a questionnaire). Reasons for refusal were not recorded. The mean age of patients across the 10 adult wards was 54 years (SD 18.13 years).i Within the paediatric ward, the mean age of children admitted was 6 years (SD 4.45 years); the age of parents completing the questionnaire was not recorded. Table 1 contains a summary of demographic characteristics by ward. The majority of the sample was classed as white British (78%, n=254), 14% (n=49) were of Asian origin (Pakistani, n=37, Indian, n=7, Bangladeshi, n=2, other Asian or Asian British, n=3). Seventy-three per cent (n=238) had been admitted to the hospital at least once previously in the past 5 years. Fifty-five per cent of patients were receiving ongoing treatment as outpatients at the time of stay.

    View this table:
    Table 1

    Response rates and sample description

    Those who did not complete the PMOS questionnaire after consenting (n=47) were more likely to be female, x2(1)=6.84, p=0.009), but did not differ on age,ii ethnicity,iii previous hospitalisation or on-going treatment.

    In order to assess test-retest reliability, 55 patients were sent a second PMOS questionnaire at home approximately 1 week after they completed the first. Of these, 25 (45%) were returned.

    Procedure

    Staff on wards identified patients to be approached by the research team. Exclusion criteria included being too ill to talk to the research team (although relatives/carers could take part on a patient's behalf), or being unable to consent. A member of the research team then approached the patient and/or their relatives/carers and asked if they would be willing to complete a brief questionnaire about safety. Once consented, the patients’ demographic information was recorded, along with a unique patient identification code. Patients were then asked to complete the PMOS either on their own or with a member of the research team. Those who wished to complete the questionnaire on their own were provided with a sealable envelope. Locked post boxes were available on each ward for patients to post their completed questionnaires.

    A subsample of 55 respondents who were due for discharge either on the day they were recruited or imminently afterwards were sent a second questionnaire to their home address approximately 1 week after they completed the first to assess test-retest reliability. They were instructed to complete the questionnaire as soon as possible and return them to the researchers in the postage-free envelope enclosed.

    Staff survey

    Participants

    In order to assess convergent validity, all staff working on 10 wards (n=118 medical staff, including 54 staff working in obstetrics and gynaecology or paediatrics who had no clear designated ward, and n=321 nursing, midwifery or auxiliary staff) were asked to complete the outcomes measures of the AHRQ hospital survey on patient safety culture questionnaire.26 Two hundred and twelve (48.3%) responses were returned. Of these, 22 were excluded as they indicated that staff worked across more than one of the wards included in the study (n=19), or in a ward not included in the study (n=3). Responses rates per ward varied from 17.5% to 95.1%.

    Procedure

    In November 2011, staff in 10 wards were sent personalised envelopes containing the AHRQ hospital survey on patient safety culture, a pen and an internally addressed reply envelope. Questionnaires were either placed in pigeon holes or distributed by the ward manager. Return of questionnaires was incentivised such that the ward returning the most questionnaires (as a percentage of total staff) would receive a £200 prize. Staff were unaware of patient scores on the PMOS scales when completing the safety culture questionnaire.

    Measures

    Patient measure of safety

    Patients completed the PMOS, whose development has been described in detail in Giles et al.15 The questionnaire assesses patients’ perception of the factors contributing to patient safety. These include latent, local and situational factors across a number of hypothesised domains:

    • Communication (eg, ‘I got answers to all the questions I had regarding my care’).

    • Individual factors (eg, ‘I felt that the attitude of staff towards me was good’).

    • Physical environment (eg, ‘There was not enough space on the ward’, reverse coded).

    • Scheduling and bed management (eg, ‘My treatment/procedure/operation did not always happen on time’, reverse coded).

    • Training and education (eg, ‘On at least one occasion a member of staff was not able to carry out a task that they should have been able to do’, reverse coded).

    • Lines of responsibility (eg, ‘I have always known which person/team was responsible for my treatment’).

    • Management of staff and staffing levels/workload (eg, ‘Too few staff meant that things didn't get done on time. Eg, attending to call bells, removing bodily fluids, toileting patients, feeding patients’, reverse coded).

    • Equipment and supplies (eg, ‘Equipment needed for my care was always working properly’).

    • Supervision and leadership (eg, ‘It was clear who was in charge of staff’).

    • Team factors (eg, ‘A doctor changed my plan of care and other staff didn't know about it’, reverse coded).

    • Support from central functions (eg, ‘My test results were always available when required eg, scans, blood tests, X-rays’).

    All 42 items were responded to on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). Respondents could also select a ‘not applicable’ option. Items relating to active failures (errors and violations) were not included in this tool, as the focus was on the identification of the factors contributing to these failures.

    AHRQ hospital survey on patient safety culture

    Staff on participating wards were asked to complete the AHRQ staff safety culture questionnaire,26 whose validation has been reported elsewhere.27 ,28 In line with guidance, four outcome scales were calculated: (A) overall perceptions of safety (mean of 4 items, eg, ‘our procedures and systems are good at preventing errors from happening’, 1: strongly disagree, to 5: strongly agree, higher scores=safer); (B) frequency of event reporting (mean of 3 items, eg, 'when a mistake is made, but is caught and corrected before affecting the patient, how often is this reported?’ 1: never, to 5: always); (C) patient safety grade (please give your work area/unit in this hospital an overall grade on patient safety, coded as Excellent=1, Very Good=2, Acceptable=3, Poor=4, Failing=5) and (D) number of events reported in the past 12 month (0=no event reports, 1=1 to 2 event reports; 3=3 to 5 event reports; 4=6 to 10 event reports; 5=11 to 20 event reports; 6=21 event reports or more).

    Analysis

    Principal components analyses (PCA) were performed on correlation matrices, with pair-wise deletion using the PASW Statistics (for Windows, V.17), using orthogonal varimax rotation to explore the internal structure of the questionnaire. PCA was chosen due to the exploratory nature of the PMOS questionnaire.

    Kaiser's criterion was used for choosing the number of factors to retain, as it has been shown to be relatively accurate when samples sizes are above 250, and when average communalities are greater than 0.60. Loadings above 0.40 were retained as significant in line with Stevens’29 recommendations. Internal reliability of the retained factors was inspected via Cronbach's α where ≥0.8 was interpreted as good, ≥0.7 acceptable, and ≥0.6 questionable.30 For factors containing fewer than 7 items, average interitem correlations were calculated based on the recommendation of Briggs and Cheek.31

    Pearson's correlations were used to assess test-retest reliability of the PMOS questionnaire across participants and to explore convergent validity of the PMOS with the AHRQ hospital survey on patient safety culture at ward level. In line with Cohen's guidelines,32 correlations of 0.1 were interpreted as a small effect, 0.3 as medium and 0.5 as large. A MANOVA was used to assess discriminant validity with ward as the independent factor, and the PMOS scales as dependent factors. Missing data were excluded list-wise.

    Results

    Factor analysis

    Using Kaiser's criterion, nine factors were extracted explaining 58.15% variance. The content of these factors are shown in online supplementary appendix table 1. Seven of the original items were complex, loading above 0.4 on more than one factor. Cross-loading variables were assigned to scales on the basis of theoretical considerations. Two items did not load on any factor.

    Constructing the PMOS index and factors

    Using the items retained from the factor analysis, an overall PMOS ‘positive index’ was constructed by summing the number of items that patients responded to by using one of the two positive response options (eg, strongly agree, or agree for a positively worded item; strongly disagree, disagree for a negatively worded item). Thus, patients could have a score out of 35 where higher responses equated to better safety. The mean PMOS index score for the entire sample was 24.43 (SD=7.28), meaning that on average patients responded positively to around 24 items out of the retained 35 in the PMOS questionnaire (see table 2).

    View this table:
    Table 2

    Descriptive scale statistics and test-retest correlations for PMOS scales

    A mean score was calculated for each of the PMOS factors, taking into account reverse coded items. In order to calculate a mean score patients were required to rate at least half the items relevant to the scale (eg, on a 5-item scale a minimum of 3 responses required). Scores could range from 1 to 5 with higher scores indicating better patient safety. These mean scores can be seen in table 3. Mean scores were generally positive across domains. Patients rated wards higher on the communication and teamwork (mean 4.24), information flow (mean 4.07) and equipment factors (mean 4.06). Factors rated as poorer included the access to resources factor (mean 3.66).

    View this table:
    Table 3

    Scores on PMOS scales by ward (list-wise deletion)

    Test-retest reliability

    The second PMOS questionnaire was completed on average around 2 weeks after the first (median=12 days, IQR=9.5 to 16 days). The PMOS was found to have acceptable test-retest reliability on the positive PMOS index (r=0.75, see table 2). Test–retest was acceptable for all PMOS factors (≥0.47, a large effect size, according to Cohen's32 guidelines). The two retained items relating to delays did not exhibit acceptable test-retest reliability.iv

    Discriminant validity

    A MANOVA was performed to assess the extent to which the PMOS discriminated among the 11 wards. The PMOS positive index, the PMOS factors and the two retained items relating to delays were entered as dependent variables with ‘ward’ entered as a fixed factor. Data were deleted list-wise, leaving a total sample of n=221. The mean scores on each PMOS factor by ward are shown in table 3.

    The MANOVA showed an overall main effect of ward (Wilks’ λ=1.67, df=110, 1510, p<0.001, partial η2=0.08). Tests of between-subject effects revealed that three factors discriminated between hospital units. Specifically, significant differences were apparent in the ‘staff roles and responsibilities’ factor (F=3.03, df=10, 210, p=0.001, partial η2=0.13); ‘ward type and layout’ factor (F=2.26, df=10 210, p=0.016, partial η2=0.10), and ‘equipment’ factor (F=1.89, df=10, 210, p=0.048, partial η2=0.08). Item 26, which referred to having enough staff to get things done just missed the standard cut-off for significance (F=1.87, df=10, 210, p=0.051, partial η2=0.082).

    Posthoc tests using the Bonferroni correction were used to explore significant differences between wards, see table 3. Some significant differences were apparent which were encouraging given the reduced power and relatively small sample sizes within each ward. Within the ‘staff roles and responsibilities’ factor, the admissions ward appeared to be significantly worse on this scale compared with eight of the 10 remaining wards. For the equipment factor, maternity ward 2 scored lowest on this scale (mean=3.00), and was significantly lower than three other wards scoring near the upper end of the scale (medicine 3, surgery 1, surgery 4), although care should be taken in this interpretation due to the low sample size within this ward (n=6, due to list-wise deletion). Finally, there was one significant difference in the ‘access to resources’ factor, with medical ward 3 scoring significantly higher than surgery ward 2 (p<0.08), and one difference on retained item 26 (too few staff meant that things DIDN'T get done on time… (reversed)) with medical ward 2 scoring worse than medical ward 3 (p=0.051).

    Convergent validity

    In order to assess the convergent validity of the PMOS, the mean PMOS positive index for each ward was correlated with the four ‘patient safety outcome’ measures of the AHRQ across 10 wards. The PMOS positive index correlated highly with the perceptions of safety outcome scale (r=0.79, p=0.007, k=10) indicating that the more positive PMOS scores among patients, the higher staff rated the ward on perceptions of safety. Additionally, the PMOS positive index correlated highly with the patient safety grade (r=−0.81, p=0.005, k=10, note that patient safety grade was coded so that higher scores indicate better patient safety), indicating that the more positive the patients’ evaluation of the ward, the better ‘grade’ staff gave their ward on patient safety. There was no relationship between the PMOS positive index and either overall staff frequency of event reporting (r=0.25, p=n/s, k=10), or individual staff event reporting (r=-0.46, p=n/s, k=10).v

    Discussion

    Patients are an important source of information with regards to improving healthcare through reducing preventable harm. The current study explored the reliability and validity of the PMOS; a tool which allows patients to proactively identify potential risks to safety in hospitals settings. The results of the study suggest that the measure shows acceptable validity and reliability, although further development is necessary to refine items and to ensure that each construct is adequately represented (7 items cross-loaded and three domains just missed the 0.7 Cronbach's α criterion of ‘acceptable’ reliability). Additionally, only three factors discriminated among wards, although this may have been due to the fact that wards were all from one hospital site (implying a shared safety culture). Future research should explore differences between wards in different hospital settingsvi.

    As far as we are aware, this is the first tool developed to allow acute health services to systematically and routinely collect information from patients about the safety of their care. Given the current spotlight upon patient safety,16–18 and the potential role of patients as an extra piece in the ‘error detection jigsaw’,9 ,33 it is felt that this tool shows promise as a means for hospitals to collect and act on this important new angle on safety intelligence. Our current work aims to use this new tool as part of an intervention, to support hospitals to make ward-based patient safety improvements on the basis of patient feedback about the safety of their care. However, this intelligence might also prove useful for health services in other ways, perhaps as part of a safety dashboard, or to help systematise the process of getting feedback from patients about safety as part of the wider clinical governance processes. Authors have recently described the need for patients to be more involved in complaints or serious incidents,33 and this measure could perhaps allow wards where safety events have occurred, to understand from a range of patients how widespread specific safety issues or concerns are.

    The PMOS was based on the YCFF, with Reason's error causation model24 as an underlying theoretical tenet. However, there were some differences in structure between the YCFF and what was observed within the PMOS. For example, items relating to communication and team working co-occurred within the current questionnaire, despite being separate factors in the original framework. The ‘access to resources’ factor contained items relating to lack of support from other staff, as well as equipment and other resources, while within the original framework these were separated into ‘support from central functions’ and ‘team factors’. These differences may stem from the fact that the YCFF was developed from empirical studies of staff-reported data, whereas, the current measure is rooted within the patient's perspective. Unsurprisingly, staff and patients, having a different vantage point within the ward, appear to conceptualise local conditions and safety differently. However, it was reassuring that some clear similarities remained across the PMOS measure and the YCFF (eg, roles and responsibilities, training and equipment).

    Patient safety is a complex multidimensional concept. A key strength of the PMOS is that it assesses eight key domains related to safety which can be used to help staff proactively identify areas of weakness within their clinical areas. An attempt was made to name factors in such a way that they would be meaningful to staff to support the use of the tool in practice. Two additional items were retained as single indicators of delays. The factors were found to demonstrate acceptable levels of internal reliability and test-retest reliability. The measure also demonstrated convergent validity (correlating strongly with staff perceptions of safety). This is an exciting result and supports recent suggestions that patients have a valuable contribution to make in assessing the safety of the wards on which they are cared for. Moreover, responses by patients to PMOS might be a useful ‘smoke-detector’34 for safety problems on a ward, particularly as the confirmation of discriminatory validity here suggests that the tool is sensitive to differences among wards.

    Limitations

    A number of limitations within the current study were apparent. Some of the dimensions within the PMOS failed to achieve the recommended Cronbach's α guidelines of 0.70. Despite this, the average inter-item correlations suggest that the dimensions are located within the optimum range of good reliability (0.20–0.40). The PMOS positive index was also found to only relate to two of the AHRQ patient safety culture survey outcomes; no significant correlations were found with the two ‘incident reporting’ outcomes. However, given the unclear relationship between levels of incident reporting and actual patient safety incidents,1 one might not expect to see a clear relationship between the number of incident reports and a measure of safety. The AHRQ hospital survey of patient safety questionnaire was chosen as a measure of convergent validity as it has been used and validated a number of times, however, recent research has questioned its reliability and validity in a UK setting.35 The response rates for this staff survey also varied widely, with an overall response rate of 48%, much lower than responses to the patient survey, which has implications for assessing the convergent validity of the measure. The two retained single indicator items exhibited poor test-retest reliability, and as such, caution should be exercised in their interpretation. Finally, the PMOS could only be completed with patients (or their relatives/carers) who understood English. Despite the above limitations, patients were found to be very willing to engage with the measure, demonstrated by the high consent rate into the study (86%). The PMOS was developed specifically for patients in hospital settings which may limit its generalisability to other settings. Further work should explore whether this tool can be used for different patient groups in a range of settings (eg, community-based outpatient services or general practice).

    Conclusion

    The current study tested the reliability and validity of the PMOS. Overall, the tool exhibited acceptable reliability and validity, subject to the caveats mentioned above. Further work is necessary to refine the tool. Based on a clear theoretical framework, the PMOS uses the patient's perspective with the intention of identifying ‘latent’ weaknesses which could contribute to future patient safety incidents. The development of this tool has implications for practice and research. Clinicians can use this tool to gain an important patient perspective on safety, capturing areas of weakness that might otherwise go unreported or unidentified. There is no other such tool available at present to collect this type of information from patients. The tool also offers an opportunity to allow clinicians and hospitals the ability to track changes in safety over time by repeated assessments within wards. We are currently exploring how clinicians can use this tool to enact and measure positive change, within a cluster randomised controlled trial (ISRCTN 07689702, http://www.controlled-trials.com/ISRCTN07689702/).

    Acknowledgments

    We are grateful to all the patients and health professionals who took part in this study, and to the members of the dedicated ‘Patient involvement in patient safety’ patient panel: Mick Bonallie, Ted Clarke, Mike Conway, Dave Green, Ashfaq Gulab, Nigat Parveen, Linda Lovett. We wish to thank Caroline Reynolds, Claire Coulson, Ikhlaq Din and Liz Thorp for recruiting patients and staff.

    References

    1. ↵
      1. Sari AB,
      2. Sheldon TA,
      3. Cracknell A,
      4. et al
      . Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 200;334:79.
    2. ↵
      1. Shojania KG
      . The frustrating case of incident-reporting systems. Qual Saf Health Care 2008;17:400–2.
      OpenUrlFREE Full Text
    3. ↵
      1. Vincent C
      . Incident reporting and patient safety. BMJ 2007;334:51.
      OpenUrlFREE Full Text
    4. ↵
      1. Lawton R,
      2. McEachan RRC,
      3. Giles SJ,
      4. et al
      . Development of an evidence-based framework of factors contributing to patient safety incidents in hospital settings: a systematic review. BMJ Qual Saf 2012;21:369–80.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Lawton R,
      2. Parker D
      . Barriers to incident reporting in a healthcare system. Qual Saf Health Care 2002;11:15–18.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      Management American Society for Healthcare Risk. Strategies and Tips for Maximising Failure Mode and Effect Analysis in Your Organization. 2002. Retrieved Nov 2013 from. http://www.ashrm.org/ashrm/education/development/monographs/FMEAwhitepaper.pdf
    7. ↵
      1. Ashley L,
      2. Armitage G,
      3. Neary M,
      4. et al
      . A practical guide to failure mode and effects analysis in health care: Making the most of the team and its meetings. Jt Comm J Qual Patient Saf 2010;36:351–8.
      OpenUrlPubMed
    8. ↵
      1. King A,
      2. Daniels J,
      3. Lim J,
      4. et al
      . Time to listen: a review of methods to solicit patient reports of adverse events. Qual Saf Health Care 2010;19:148–57.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Ward JK,
      2. Armitage G
      . Can patients report patient safety incidents in a hospital setting? A systematic review. BMJ Qual Saf 2012;21:685–99.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Ward JK,
      2. Giles SJ,
      3. Hrisos S,
      4. et al
      . Involving patients in their own safety. In: Lawton R, Armitage G. eds Innovating for patient safety in medicine. London: SAGE, learning matters, 2012:42–60.
    11. ↵
      1. Weingart SN,
      2. Pagovich O,
      3. Sands DZ,
      4. et al
      . What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. J Gen Int Med 2005;20:830–6.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Davis RE,
      2. Sevdalis N,
      3. Vincent CA
      . Patient involvement in patient safety: how willing are patients to participate? BMJ Qual Saf 2011;20:108–14.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Hall J,
      2. Peat M,
      3. Birks Y,
      4. et al
      . Effectiveness of interventions designed to promote patient involvement to enhance safety: a systematic review. Qual Saf Health Care 2010;19:e10.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Weissman JS,
      2. Schneider EC,
      3. Weingart SN,
      4. et al
      . Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not? Ann Intern Med 2008;149:100–8.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Giles SJ,
      2. Lawton RJ,
      3. Din I,
      4. et al
      . Developing a patient measure of safety (PMOS). BMJ Qual Saf 2013;22:554–62.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Francis R
      . Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry: Executive Summary. TSO Shop, 2013.
    17. ↵
      1. Berwick D
      . On behalf of the National Advisory Group on the Safety of Patients in England, Improving the safety of Patients in England: A promise to learn—a commitment to act. Department of Health.
    18. ↵
      1. Keogh B
      . Review into the quality of care and treatment provided by 14 hospital trusts in England: Overview report. England: NHS, 2013.
    19. ↵
      Care Quality Commission. National Summary: Inpatient Survey Results 2012. Picker Institute Europe, 2013.
    20. ↵
      World Health Organization. World Alliance for Patient Safety: Forward Programme. World Health Organization, 2004.
    21. ↵
      Department of Health. Equity and Excellence: Liberating the NHS. London: Department of Health, 2010.
    22. ↵
      1. Hor S,
      2. Godbold N,
      3. Collier A,
      4. et al
      . Finding the patient in patient safety. Health 2013;17:567–83.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Greaves F,
      2. Pape UJ,
      3. King D,
      4. et al
      . Associations between web-based patient ratings and objective measures of hospital quality. Arch Intern Med 2012;172:435–6.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Reason J
      . Human error: models and management. BMJ 2000;320:768.
      OpenUrlFREE Full Text
    25. ↵
      Catell (1978) in MacCallum RC, Widaman KF, Zhang S, et al. Sample size in factor analysis. Psychol Methods 1999;4:84–99, p84.
      OpenUrlCrossRefWeb of Science
    26. ↵
      1. Sorra JS,
      2. Nieva VF
      . Hospital Survey on Patient Safety Culture (Prepared by Westat, under contract no. 290-96-0004. 2004 (AHRQ Publication No. 04-0041).
    27. ↵
      1. Sorra JS,
      2. Dyer N
      . Multilevel psychometric properties of the AHRQ Hospital Survey on Patient Safety Culture. BMC Health Serv Res 2010;8:199.
      OpenUrl
    28. ↵
      1. Sarac C,
      2. Flin R,
      3. Mearns K,
      4. et al
      . Hospital survey on patient safety culture: psychometric analysis on a Scottish sample. BMJ Qual Saf 2011;20:842–8.
      OpenUrlAbstract/FREE Full Text
    29. ↵
      1. Stevens JP
      . Applied Multivariate Statistics for the Social Sciences. Hillsdale, NJ: Erlbaum, 1992.
    30. ↵
      1. George D,
      2. Mallery P
      . SPSS for Windows step by step: a simple guide and reference. 11.0 update. 4th edn. Boston: Allyn & Bacon, 2003.
    31. ↵
      1. Briggs SR,
      2. Cheek JM
      . The role of factor analysis in the development and evaluation of personality scales. J Pers 1986;54:106–48.
      OpenUrlCrossRefWeb of Science
    32. ↵
      1. Cohen J
      . A power primer. Psyc Bull 1992;112:115–59.
      OpenUrl
    33. ↵
      1. O'Hara J,
      2. Isden R
      . Identifying risks and monitoring safety: the role of patients and citizens. London: The Health Foundation, 2013.
    34. ↵
      1. Bacon N
      . A smoke-alarm for patient safety and healthcare quality, in Neil Bacon Blog. 2010. Retrieved August 2013. http://neilbacon.wordpress.com/2010/12/13/a-smoke-alarm-for-patient-safety-andhealthcare-quality/
    35. ↵
      1. Waterson P,
      2. Griffiths P,
      3. Stride C,
      4. et al
      . Psychometric properties of the Hospital Survey on Patient Safety Culture: findings from the UK. Qual Saf Health Care 2010;19:e2.
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Collaborators on behalf of the Yorkshire Quality and Safety Research Group.

    • Contributors RJL, GA, JW and ISW conceived the original research proposal. RRCM, JKOH, SG, RJL, GA, JW and ISW developed the protocol and managed the project. RRCM, RJL and SP conducted and interpreted analyses. RRCM and RJL drafted the manuscript. All authors commented on and approved the final draft.

    • Funding This paper presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (Improving safety through the involvement of patients: grant reference number RP-PG-0108-10049).

    • Competing interests None.

    • Ethics approval The study was approved by the Bradford Local Research Ethics committee (09/H1302/115).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement We are happy to share anonymised data.

    • ↵i Unfortunately demographic data was not available for one ward (n=14).

    • ↵ii Excluding the paediatric ward.

    • ↵iii Comparing only white British versus Asian patients.

    • ↵iv One outlying participant was identified from the test-retest calculations who scored highly on the PMOS positive index at baseline (31) but very low at time 2 (3). This participant was excluded from the test-retest calculations. Values including this individual were as follows: PMOS positive index r=0.57** (n=25); communication and team work r=0.51** (n=25); organisation and care planning r=0.55** (n=25); access to resources r=0.65*** (n=25); ward type and layout r=0.41* (n=24); information flow r=0.52** (n=25); staff roles and responsibilities r=0.61*** (n=25); staff training r=0.48* (n=20); equipment r=0.35 (n=21); item 8 r=0.40 (n=23); item 26 r=0.05 (n=22). ***p≤0.001, **p≤0.01, *p≤0.05.

    • ↵v The analyses were repeated excluding 3 wards for which there were less than 14 responses to the questionnaire, leaving a sample size of 7 wards. The pattern of results were the same (perceptions of safety, r=0.78, p=0.039; patient safety grade, r=−0.75, p=0.053), although the reduction in power rendered the correlation with patient safety grade non-significant.

    • ↵vi We thank an anonymous reviewer for this comment.