We conducted a year-long quality improvement project involving internal medicine residents at an academic medical centre. Resident champions designed and implemented a discharge summary improvement bundle, which employed an educational curriculum, an electronic discharge summary template, regular data feedback and a financial incentive. The timeliness and quality of discharge summaries were measured before and after the intervention. Residents and faculty were surveyed about their perceptions of the project; primary care providers were surveyed about their satisfaction with hospital provider communication.

With implementation of the bundle, the average time from patient discharge to completion of the discharge summary fell from 3.5 to 0.61 days (p<0.001). The percentage of summaries completed on the day of discharge rose from 38% to 83% (p<0.001) and this improvement was sustained for 6 months following the end of the project. The percentage of summaries that included all recommended elements increased from 5% to 88% (p<0.001). Primary care providers reported a lower likelihood of discharge summaries being unavailable at the time of outpatient follow-up (38% to 4%, p<0.001). Residents reported that the systems changes, more than the financial incentive, accounted for their behaviour change.

Our discharge summary improvement project provides an instructive example of how residents can lead clinically meaningful quality improvement projects.

At a 510-bed Danish university hospital, a multidisciplinary change project inspired by Kotter's Eight Steps of Change was designed. In addition to revision of antimicrobial guidelines and restriction of selected antimicrobials, the complex, managed, multi-faceted intervention comprised training and education, enhanced isolation precautions, and a series of actions to improve the infection control measures and standardise procedures across the hospital. A prospective interrupted time series design was used to analyse data collected at hospital level from January 2008 through December 2011.

Though overall antimicrobial consumption remained unaffected, the intervention led to intended, immediate and sustained reduction in the use of cefuroxime, and an increase in the use of ertapenem, piperacillin/tazobactam and β-lactamase sensitive penicillin. Moreover, a postintervention reduction in the rate of ESBL-KP in diagnostic samples and in the incidence of ESBL-KP infections was observed. The intervention may also have reduced the need for isolation precautions and may have shortened each isolation period.

The results indicate that an immediate and sustained change in the antimicrobial consumption and a decreasing rate of ESBL-KP are achievable through the application of a managed, multi-faceted intervention that does not require ongoing antibiotic stewardship.

We analysed the results of AIPS for 2010 (n=56 931 returns) by specialty (medicine, surgery, orthopaedics, renal medicine, neurosurgery, obstetrics-gynaecology and oncology) to determine whether validity is retained at a suborganisational level.

Criterion validity and internal consistency of AIPS were retained for most specialty subgroups. When small local samples were excluded, the results for Trust level specialty groups were similar over a 2-year period, indicating test stability. For oncology there was poor internal consistency in the ‘doctors’ domain and criterion validity, expressed as the relationship elements of experience and overall rating of care, was less than for other specialties.

The AIPS is suitable for use within many specialties, but our findings question some elements of validity for oncology inpatients. We recommend that future surveys are administered and reported by specialty, to inform local improvement and permit comparison of specialty units.

Data of the Dutch Surgical Colorectal Audit were used. Case-mix factors and treatment-related factors were identified from the literature and their association with AL and mortality were analysed with logistic regression. Hospital differences in observed AL and mortality rates, and adjusted rates based on the logistic regression models were shown. The reduction in hospital variance after adjustment was analysed with Levene's test for equality of variances.

17 of 22 case-mix factors and 4 of 11 treatment factors related to AL derived from the literature were available in the database. Variation in observed AL rates between hospitals was large with a maximum rate of 17%. This variation could not be attributed to differences in case-mix but more to differences in treatment factors. Hospital variation in observed mortality rates was significantly reduced after adjustment for differences in case-mix.

Hospital variation in AL is relatively independent of differences in case-mix. In contrast to ‘postoperative mortality’ the observed AL rates of hospitals evaluated in our study were only slightly affected after adjustment for case-mix factors. Therefore, AL rates may be suitable as an outcome indicator for measurement of surgical quality of care.

To investigate the reliability of injury coding, injury severity scoring and survival status in a regional TR. The feasibility of the format that was developed for this study was also investigated.

A random sample, without replacement, was taken from the TR of a Dutch regional trauma care network. All 343 patients in the sample were then recoded by another trauma registrar (rater). Reliability was expressed in the percentage agreement between the raters.

In the total study sample of 333 patients, the reliability of the number of Abbreviated Injury Scale (AIS) codes was substantial (intraclass correlation coefficient (ICC)=0.70); and the reliability of the Injury Severity Score (ISS) (ICC=0.84) and survival status were ‘almost perfect’ (Cohen's =0.82). Both raters had given 129 patients one AIS code. The reliability of the body region of the AIS was ‘almost perfect’ (Cohen's =0.91); and the reliability of the severity of the injury and the ISS were ‘almost perfect’ (weighted =0.88 and ICC=0.90). The reliability of the ISS in the patients who were assigned at least two AIS codes (n=128) was ‘almost perfect’ (ICC=0.86). The reliability of the number of AIS codes and the number of body regions was ‘moderate’ (ICC=0.56 and Cohen's =0.52).

The reliability of injury coding in a regional trauma registry was ‘substantial’ and the reliability of the ISS and survival status was ‘almost perfect’. The format and design of this study were feasible and could be used to investigate the quality of (trauma) registries.

This paper offers practical advice about organising and writing a manuscript reporting quality improvement or patient safety research for submission to a peer-reviewed journal.

Each section of the paper discusses a specific manuscript component—from title, abstract and each section of the manuscript body, through to reference list and tables and figures—explaining key principles, offering content organisation tips and providing an example of how this section may read. The paper also offers a checklist of common mistakes to avoid in a manuscript.

13 incident questions were included in a national patient-experience survey in Norway during the spring of 2011. All questions and a composite incident index were assessed by calculating missing-item rates, test–retest reliability and hospital-level reliability. A multivariate linear regression on a global item regarding incorrect treatment was used to assess the main sources of variation in patient-perceived incorrect treatment at hospitals.

Five of the 13 patient-incident questions had a missing-item rate of >20%. Only one item met the criterion of 0.7 for test–retest reliability (wrong or delayed diagnosis), seven items had a score of >0.5, while the remainder had a reliability score of <0.5. However, the reliability was >0.7 for six of 10 items tested at the hospital level, and >0.6 for the remaining four items. A patient-incident index based on 12 of the incident items had no missing data, the test–retest reliability was 0.6 and the hospital-level reliability was 0.85.

The PRIH-I comprises 13 questions about patient-perceived incidents in hospitals, and can be easily and cost-effectively included in national patient-experience surveys with an acceptable increase in respondent burden. Although the missing-item rate and test–retest reliability were poor for several items, the hospital-level reliability was satisfactory for most of the items. The incident items contribute to a patient-reported incident index, with excellent data quality and hospital-level reliability.

We conducted an observational cross-sectional study of readmissions of children in 225 hospitals by linking nurse surveys, inpatient discharge data and information from the American Hospital Association Annual Survey. Registered Nurses (N=14 194) providing direct patient care in study hospitals (N=225) and children hospitalised for common conditions (N=90 459) were included.

Each one patient increase in a hospital's average paediatric staffing ratio increased a medical child's odds of readmission within 15–30 days by a factor of 1.11, or by 11% (95% CI 1.02 to 1.20) and a surgical child's likelihood of readmission within 15–30 days by a factor of 1.48, or by 48% (95% CI 1.27 to 1.73). Children treated in hospitals with paediatric staffing ratios of 1 : 4 or less were significantly less likely to be readmitted within 15–30 days. There were no significant effects of nurse staffing ratios on readmissions within 14 days.

Children with common conditions treated in hospitals in which nurses care for fewer patients each are significantly less likely to experience readmission between 15 and 30 days after discharge. Lower patient-to-nurse ratios hold promise for preventing unnecessary hospital readmissions for children through more effective predischarge monitoring of patient conditions, improved discharge preparation and enhanced quality improvement success.

To compare prescribing information approved by the US Food and Drug Administration with the UK, Canada and Australia to identify content differences in safety warnings.

For 20 top-selling prescription drugs, we used an automated natural language processing tool to calculate the number and severity of reported adverse drug reactions (ADRs). We fit hierarchical Poisson models and included fixed effects for other prescribing information characteristics. Separately, we analysed the appearance and content of ‘black box’ warnings.

There was substantial variation in safety content of approved prescribing information. Canada had the highest median ADRs per drug (138 (IQR 86–234)) and the UK had the lowest (84 (IQR 51–111)). The number of ADRs reported was on average 50% higher in Canada compared with the USA (ratio of ADRs/document: 1.5, 95% CI 1.4 to 1.6, p<0.001). By contrast, there were on average 15% fewer ADRs listed in the UK compared with the USA (ratio of ADRs/document 0.85 (95% CI 0.78 to 0.93, p<0.001), and 21% fewer ADRs listed in Australia compared with the USS (ratio of ADRs/document 0.79, 95% CI 0.74 to 0.85, p<0.001). There were no variations in ADR severity. The presence and qualitative content of boxed warnings also showed substantial diversity.

International variations exist in the presentation of safety data in drug prescribing information, which may have important implications for patient safety. Better international coordination is necessary to enhance use of this information for patient decision-making.

We analysed diagnosis-related claims from the National Practitioner Data Bank (1986–2010). We describe error type, outcome severity and payments (in 2011 US dollars), comparing diagnostic errors to other malpractice allegation groups and inpatient to outpatient within diagnostic errors.

We analysed 350 706 paid claims. Diagnostic errors (n=100 249) were the leading type (28.6%) and accounted for the highest proportion of total payments (35.2%). The most frequent outcomes were death, significant permanent injury, major permanent injury and minor permanent injury. Diagnostic errors more often resulted in death than other allegation groups (40.9% vs 23.9%, p<0.001) and were the leading cause of claims-associated death and disability. More diagnostic error claims were outpatient than inpatient (68.8% vs 31.2%, p<0.001), but inpatient diagnostic errors were more likely to be lethal (48.4% vs 36.9%, p<0.001). The inflation-adjusted, 25-year sum of diagnosis-related payments was US$38.8 billion (mean per-claim payout US$386 849; median US$213 250; IQR US$74 545–484 500). Per-claim payments for permanent, serious morbidity that was ‘quadriplegic, brain damage, lifelong care’ (4.5%; mean US$808 591; median US$564 300), ‘major’ (13.3%; mean US$568 599; median US$355 350), or ‘significant’ (16.9%; mean US$419 711; median US$269 255) exceeded those where the outcome was death (40.9%; mean US$390 186; median US$251 745).

Among malpractice claims, diagnostic errors appear to be the most common, most costly and most dangerous of medical mistakes. We found roughly equal numbers of lethal and non-lethal errors in our analysis, suggesting that the public health burden of diagnostic errors could be twice that previously estimated. Healthcare stakeholders should consider diagnostic safety a critical health policy issue.

We compared the pattern of vital signs and VitalPAC Early Warning Score (ViEWS) data collected from admissions to all adult inpatient areas (except high care areas, such as critical care units) of a NHS district general hospital from 1 May 2010 to 30 April 2011, to the hospital's clinical escalation protocol. Main outcome measures were hourly and daily patterns of vital signs and ViEWS value documentation; numbers of vital signs in the periods 08:00–11:59 and 20:00–23:59 with subsequent vital signs recorded in the following 6 h; and time to next observation (TTNO) for vital signs recorded in the periods 08:00–11:59 and 20:00–23:59.

950 043 vital sign datasets were recorded. The daily pattern of observation documentation was not uniform; there were large morning and evening peaks, and lower night-time documentation. The pattern was identical on all days. 23.84% of vital sign datasets with ViEWS ≥ 9 were measured at night compared with 10.12–19.97% for other ViEWS values. 47.42% of patients with ViEWS=7–8 and 31.22% of those with ViEWS ≥ 9 in the period 20:00–23:59 did not have vital signs recorded in the following 6 h. TTNO decreased with increasing ViEWS value, but less than expected by the monitoring protocol.

There was only partial adherence to the vital signs monitoring protocol. Sicker patients appear more likely to have vital signs measured overnight, but even their observations were often not followed by timely repeat assessments. The observed pattern of monitoring may reflect the impact of competing clinical priorities.

The objective of this article is to describe the scientific discipline of human factors and provide common ground for partnerships between healthcare and human factors communities.

The primary goal of human factors science is to promote efficiency, safety and effectiveness by improving the design of technologies, processes and work systems. As described in this article, human factors also provides insight on when training is likely (or unlikely) to be effective for improving patient safety. Finally, we outline human factors specialty areas that may be particularly relevant for improving healthcare delivery and provide examples to demonstrate their value.

The human factors concepts presented in this article may foster interdisciplinary collaborations to yield new, sustainable solutions for healthcare quality and patient safety.

Totally 18 clinicians, licensed practical nurses, registered nurses and medical doctors, at two Swedish emergency departments were observed during clinical work for 2 h each. A semistructured interview was conducted directly after the observation to explore their perceptions of interruptions. Data were analysed using non-parametric statistics, and by quantitative and qualitative content analysis.

The interruption rate was 5.1 interruptions per hour. Most often the clinicians were exposed to interruptions during activities involving information exchange. Calculated as percentages of categorised performed activities, preparation of medication was the most interrupted activity (28.6%). Face-to-face interaction with a colleague was the most common way to be interrupted (51%). Most common places for interruptions to occur were the nurses’ and doctors’ stations (68%). Medical doctors were the profession interrupted most often and were more often recipients of interruptions induced by others than causing self-interruptions. Most (87%) of the interrupted activities were resumed. Clinicians often did not regard interruptions negatively. Negative perceptions were more likely when the interruptions were considered unnecessary or when they disturbed the work processes.

Clinicians were exposed to interruptions most often during information exchange. Relative to its occurrence, preparation of medication was the most common activity to be interrupted, which might increase risk for errors. Interruptions seemed to be perceived as something negative when related to disturbed work processes.

An institutional ethnographic study was undertaken. The study was conducted over three hospital sites in Melbourne, Australia. In total, 125 healthcare professionals from the disciplines of surgery, anaesthesia and nursing participated in the study. Data were generated through 350 h of observation, two focus groups and 20 semi-structured interviews. An institutional ethnographic analysis was undertaken.

Analysis revealed healthcare professionals adapted the content, timing and number of team members involved in the time-out procedure to meet the demands of the theatre environment. Habitually, the time-out procedure was partially completed, conducted after surgery had commenced and involved only a few members of the surgical team. Communication was restricted and stifled by asynchronous workflows, time restrictions, a hierarchical culture and disinclination by surgeons and anaesthetists to volunteer information and openly communicate with each other and nurses. Healthcare professionals became normalised to performing an abbreviated time-out procedure.

Patient safety was relegated in importance as productivity, professional and hierarchical discourses configured the communication practices of surgical team members to limit active, open and direct communication. Examining how the time-out procedure was conducted in the clinical environment enables possibilities to emerge for facilitating compliance with hospital and WHO guidelines.

We assembled a national sample of all 18 907 formal patient complaints filed against doctors with health service ombudsmen (‘Commissions’) in Australia over an 11-year period. We analysed the distribution of complaints among practicing doctors. We then used recurrent-event survival analysis to identify characteristics of doctors at high risk of recurrent complaints, and to estimate each individual doctor's risk of incurring future complaints.

The distribution of complaints among doctors was highly skewed: 3% of Australia's medical workforce accounted for 49% of complaints and 1% accounted for a quarter of complaints. Short-term risks of recurrence varied significantly among doctors: there was a strong dose-response relationship with number of previous complaints and significant differences by doctor specialty and sex. At the practitioner level, risks varied widely, from doctors with <10% risk of further complaints within 2 years to doctors with >80% risk.

A small group of doctors accounts for half of all patient complaints lodged with Australian Commissions. It is feasible to predict which doctors are at high risk of incurring more complaints in the near future. Widespread use of this approach to identify high-risk doctors and target quality improvement efforts coupled with effective interventions, could help reduce adverse events and patient dissatisfaction in health systems.

Parents (n=172) were surveyed about perceptions of hospital safety climate (14 items representing four domains—overall perceptions of safety, openness of staff and parent communication, and handoffs and transitions) and perceived need to watch over their child's care. Confirmatory factor analysis (CFA) was used to validate safety climate domain measures. Logistic regression was used to relate need to watch over care to safety climate domains.

CFA indices suggested good model fit for safety climate domains. Thirty-nine per cent of parents agreed or strongly agreed they needed to watch over care. In adjusted models, need to watch over care was significantly related to overall perceptions of safety (OR 0.20, 95% CI 0.11 to 0.37) and to handoffs and transitions (0.25, 0.14 to 0.46), but not to openness of staff (0.67, 0.40 to 1.12) or parent (0.83, 0.48 to 1.45) communication.

Findings suggest parents can provide valuable data on specific safety climate domains. Opportunities exist to improve our safety climate's impact on parent burden to watch over their child's care, such as targeting overall perceptions of safety as well as handoffs and transitions.

TeamGAINS was administered during simulation-based trainings for clinical and behavioural skills for anaesthesia staff. One of the four daily scenarios involved all trainees, whereas the remaining three scenarios each involved only two trainees with the others observing them. Training instructors were senior anaesthesiologists and psychologists. To determine debriefing quality, we used a post-test-only (debriefing quality) and a pre-post-test (psychological safety, leader inclusiveness), no-control-group design. After each debriefing all trainees completed a self-report debriefing quality scale which we developed based on the Debriefing Assessment for Simulation in Healthcare and the Observational Structured Assessment of Debriefing. Perceived psychological safety and leader inclusiveness were measured before trainees’ first (premeasure) and after their last debriefing (postmeasure) at which time trainees’ reactions to the overall training were measured as well.

Four senior anaesthetists, 29 residents and 28 nurses participated in a total of 40 debriefings resulting in 235 evaluations. Utility of debriefings was evaluated as highly positive. Pre-post comparisons revealed that psychological safety and leader inclusiveness significantly increased after the debriefings.

The results indicate that TeamGAINS could provide a useful debriefing tool for training anaesthesia staff on all levels of work experience. By combining state-of-the-art debriefing methods and integrating systemic-constructivist techniques, TeamGAINS has the potential to allow for a surfacing, reflecting on and changing of the dynamics of team interactions. Further research is necessary to systematically compare the effects of TeamGAINS’ components on the debriefing itself and on trainees’ changes in attitudes and behaviours.

To facilitate future ACT research by presenting a framework that provides a shared language of teamwork behaviours, allows for comparing previous and future ACT research and offers a measurement tool for ACT observation.

Based on teamwork theory and empirical evidence, we developed Co-ACT—the Framework for Observing Coordination Behaviour in ACT. Integrating two previous, extensive taxonomies into Co-ACT, we also suggested 12 behavioural codes for which we determined inter-rater reliability by analysing the teamwork of videotaped anaesthesia teams in the clinical setting.

The Co-ACT framework consists of four quadrants organised along two dimensions (explicit vs implicit coordination; action vs information coordination). Each quadrant provides three categories for which Cohen's overall value was substantial; but values for single categories varied considerably.

Co-ACT provides a framework for organising behaviour codes and offers respective categories for succinctly measuring teamwork in ACTs. Furthermore, it has the potential to allow for guiding and comparing ACTs study results. Future work using Co-ACT in different research and training settings will show how well it can generally be applied across ACTs.

From March to July 2010, data were collected during elective surgery at the Geneva University Hospitals, Switzerland. The main outcome was to assess whether each item of the Time Out and the Sign Out checklists have been checked, that is, ‘confirmed’ by at least one member of the team and ‘validated’ by at least one other member of the team. The secondary outcome was the number of validated items during the Time Out and the Sign Out.

Time Outs (N=80) and Sign Outs (N=81) were conducted quasi systematically (99%). Items were mostly confirmed during the Time Out (range 100–72%) but less often during the Sign Out (range 86–19%). Validation of the items was far from optimal: only 13% of Time Outs and 3% of Sign Outs were properly checked (all items validated). During the Time Out, the validation process was significantly improved among the highest risk interventions (29% validation vs 15% among interventions at lower risk). During the Sign Out, a similar effect was observed (19% and 8%, respectively). A small but significant benefit was observed when using a printed checklist as a memory tool during the Sign Out, the proportion of interventions with almost all validated items being higher compared with those without the memory tool (20% and 0%, respectively).

Training on the proper completion of the checklist must be provided to OR teams. The severity of the interventions influenced the number of items properly checked.

Senior doctors were recruited from across 20 hospitals in the North Western Deanery, England, UK. The intervention comprised a half-day course in patient safety theory, root cause analysis and small-group facilitation, following which participants were invited to sign up as faculty for a region-wide patient safety training programme for trainees ‘Lessons Learnt’. Course evaluation comprised a prospective longitudinal study conducted in 2010–2012. Patient safety knowledge, attitudes and skills were evaluated pre and post course and retention further evaluated 8 months post course.

216 senior doctors volunteered as faculty of whom 122 were appointed. Participants reported high levels of satisfaction with the course. Objective scores of patient safety knowledge significantly improved immediately post course (Median_{Pre course}=70%, Median_{Post course}=80%, p<0.001) and were sustained at 8 months (Median_{8 month post course}=90%). Similarly, measures of attitudes and self-reported safety skills also significantly improved post course and were sustained. Upon completion of the course, 88/122 (72%) participants facilitated 213 ‘Lessons Learnt’ sessions from January 2011 to July 2012 (mean 2, range 1–8 sessions per faculty member). Trainee satisfaction with faculty was high.

There is considerable appetite for senior doctors to engage with training in patient safety as teachers and learners. Training senior doctors in patient safety is feasible, acceptable and effective as a means of building capacity and capability for delivering training in this rapidly emerging field.

There were three stages in this study. In Stage 0, data collected from 2005–2006 was used to identify failures in the process, and during Stage 2 (2007) and Stage 3 (2008) the efficacy of intrahospital and interhospital strategies to reduce the D2B time were evaluated. This study enrolled 385 patients; 86 from 2005–2006; 80 in 2007; and 219 in 2008.

By making improvements in these steps, the median D2B time was reduced from 146 min to 32 min for all patients. The proportion of patients with a D2B time of <90 min significantly increased from Stage 0 to Stage 1 and from Stage 1 to Stage 2, for all patients as well as for the non-transferred and transferred subgroups of patients (all p values <0.0001). For non-transferred patients, only reinfarction-free survival showed significant difference among the three stages (p=0.0225), and for transferred patients, only overall survival showed significant difference among the three stages (p=0.0322). Cox's proportional hazards regression analysis showed Stage 2 was associated with a lower risk of reinfarction and mortality compared with Stage 0.

This study found that failure mode and effect analysis is a powerful method for identifying weaknesses in D2B processes and evaluating strategies to reduce the D2B time.

We conducted a cross-sectional survey of house officers in internal medicine at Johns Hopkins Hospital. Ethical reasoning was assessed using the Defining Issues Test, V.2 (DIT-2). Higher DIT-2 scores are associated with more sophisticated ethical reasoning, and scores tend to improve after formal ethical training. Respondents were then given a medical error scenario with multiple scripted disclosure options. Ethical reasoning scores were compared for different levels of disclosure.

A total of 29 out of 127 medicine residents completed the survey (response rate=22.8%). The 41% of residents who would explicitly tell a patient that, ‘an error happened’ had significantly higher DIT-2 scores than those who would not (51.1 vs 31.6, p<0.01). Higher DIT-2 scores were found in the 62% of residents who would give the most detail about the error (44.8 vs 31.2, p=0.04) and in the 71% of residents who would take personal responsibility (43.5 vs 27.9, p=0.03).

Ethical reasoning scores were associated with acknowledging an error, providing more detailed explanations and taking personal responsibility. The low response rate may limit generalisability. Nevertheless, taken together with prior studies showing that ethical reasoning can be taught and measured, our findings suggest that ethical training may help to improve disclosure of medical errors.

Thirty-three qualitative interviews with a representative sample of patients from six units in a teaching hospital in the north of England were carried out. Patients were asked either to describe their most recent/current hospital experience (unstructured) or were asked to describe their experience in relation to specific contributory factors (structured). Responses were coded using the YCFF. Face validity of the PMOS was tested with 12 patients and 12 health professionals, using a ‘think aloud’ approach, and appropriate revisions made. The research was supported by two patient representatives.

Patients were able to comment on/identify 13 of the 20 contributory factors contained within the YCFF domains. They identified contributory factors relating to communication and individual factors more frequently, and contributory factors relating to team factors, and support from central functions less frequently. In addition, they identified one theme not included in the YCFF: dignity and respect. The draft PMOS showed acceptable face validity.

Patients are able to identify factors which contribute to the safety of their care. The PMOS provides a way of systematically assessing these and has the potential to help health professionals and healthcare organisations understand and identify, safety concerns from the patients’ perspective, and, in doing so, make appropriate service improvements.

To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT.

Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland–Altman plot assessed measurement error and limits of agreement.

Only 31% of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland–Altman plot suggests little systematic error but large random error.

Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.

Using staff responses from the NHS Staff Survey 2009, and correlating these with hospital standardised mortality ratios (HSMR), correlation analyses were conducted at institutional level with further analyses of staff subgroups.

Over 60 000 respondents from 147 NHS trusts were included in the analysis. There was a weak negative correlation with HSMR where staff agreed that patient care was their trust's top priority (Kendall = –0.22, p<0.001), and where they would be happy with the care for a friend or relative (Kendall = –0.30, p<0.001). These correlations were identified across clinical and non-clinical groups, with nursing staff demonstrating the most robust correlation. There was no correlation between satisfaction with the quality of care delivered by oneself and institutional HSMR.

In the context of the continued debate about the relationship of HSMR to hospital performance, these findings of a weak correlation between staff satisfaction and HSMR are intriguing and warrant further investigation. Such measures in the future have the advantage of being intuitive for lay and specialist audiences alike, and may be useful in facilitating patient choice. Whether higher staff satisfaction drives quality or merely reflects it remains unclear.

In an iterative process we translated, tested and refined an existing checklist from the aviation industry, called self, human interaction, aircraft, procedures and environment, in the context of healthcare, notably emergency departments (ED) and intensive care units (ICU). The EPOC comprises six dimensions: assertiveness, working with others; task-oriented leadership; people-oriented leadership; situational awareness; planning and anticipation. Each dimension is specified into several concrete items reflecting verbal behaviours. The EPOC was evaluated in four ED and six ICU.

In the ED and ICU, respectively, 378 and 1144 individual and 51 and 68 contemporaneous observations of individual staff members were conducted. All EPOC dimensions occur frequently, apart from assertiveness, which was hardly observed. Intraclass correlations for the overall EPOC score ranged between 0.85 and 0.91 and for underlying EPOC dimensions between 0.53 and 0.95.

The EPOC is a new instrument for evaluating the use of non-technical skills in healthcare, which is reliable in two highly different settings. By quantifying professional behaviour the instrument facilitates measurement of behavioural change over time. The results suggest that EPOC can also be translated to other settings.

We used two sequential evaluative methods to study a QI effort to improve delivery of HIV preventive care in public health facilities in three districts in KwaZulu-Natal, South Africa, over a 3-year period. We initially used a cluster randomised controlled trial (RCT) design.

During the RCT study period, tensions arose between intervention implementation and evaluation design due to loss of integrity of the randomisation unit over time, pressure to implement changes across the randomisation unit boundaries, and use of administrative rather than functional structures for the randomisation. In response to this loss of design integrity, we switched to a more flexible intervention design and a mixed-methods quasiexperimental evaluation relying on both a qualitative analysis and an interrupted time series quantitative analysis.

Cluster RCT designs may not be optimal for evaluating complex interventions to improve implementation in uncontrolled ‘real world’ settings. More flexible, context-sensitive evaluation designs offer a better balance of the need to adjust the intervention during the evaluation to meet implementation challenges while providing the data required to evaluate effectiveness. Our case study involved HIV care in a resource-limited setting, but these issues likely apply to complex improvement interventions in other settings.