• Health policy
• Health professions education
• Health services research

Healthcare organisations have a mandate to provide the highest standard of care for patients and their families. While it may be difficult to empirically demonstrate that an organisation provides a high standard of care using basic outcomes such as mortality,1 the foundations of both measuring and improving healthcare quality include consideration of adequate structures and processes of care with proven relationships to better outcomes.2 ,3 Healthcare organisations rely on evidence-based processes of care (eg, proven medication for the treatment of acute myocardial infarction and appropriate prevention strategies for avoiding postoperative infections, such as the use of deep venous thrombosis prophylaxis in hospitalised patients) and structures (such as adequate nurse–patient ratios and sufficient patient volumes for complex surgical procedures) to support quality for several important reasons.4 First, outcomes are influenced by patient-level factors that are unrelated to quality of care, such as case-mix5; second, risk-adjustment models are unreliable in detecting true outliers, with a large degree of variation across different models of care6 and third, even with perfect risk adjustment unmeasured confounders, such as differences in the proportions of high-risk patients, can produce misleading measures of performance.7 ,8 And, finally, organisations frequently do not have enough volume of cases to distinguish high performers from low performers within a useful time frame.9 Thus, it is imperative to identify structures and processes of care that are associated with better outcomes.

Processes of care are frequently identified from randomised controlled trials. Once a new drug or intervention is shown to be superior to a control (either a placebo or another drug or intervention), it may become standard of care, and failure in providing it for eligible patients may be considered an error of omission when assessing the presence of preventable adverse events. Structures that facilitate better care are, however, more difficult to identify. Not unexpectedly, very few randomised trials compare different structures, mainly because these trials are extremely complex. It is hard to imagine, for instance, multiple hospitals agreeing to be randomised to have higher or lower numbers of patients cared for by each nurse. Thus, much of what we know about structures comes from observational studies.10

An analysis from a large cohort of inpatients in acute care hospitals in the UK suggests that outcomes after cardiac arrest are better during daytime working hours compared with night-time and weekends.11 This is important information that policy makers may use to shape future healthcare delivery by demanding increases in out-of-hours staffing in an attempt to improve these outcomes. Before this happens, the findings of this study and any recommendations that are based on these findings merit rigorous and careful evaluation, as they may be misleading, and acting on them will certainly incur increased costs in already strained healthcare systems.

Using data for 27 000 patients from April 2011 to September 2013 from the UK National Cardiac Arrest Audit, Robinson and colleagues analysed outcomes related to in-hospital cardiac arrest, including return of spontaneous circulation (ROSC) for at least 20 min and survival to hospital discharge. Taking daytime hours during the work week as the benchmark, they reported worse risk-adjusted outcomes at night and during the day on weekends. Specifically, resuscitation of patients who suffered cardiac arrest on weekends during daytime hours less often led to ROSC >20 min (OR 0.88; 95% CI 0.81 to 0.95) or survival to discharge (OR 0.72; 95% CI 0.64 to 0.80). Cardiac arrests at night showed a similar pattern, with lower rates of both ROSC (OR 0.72; 95% CI 0.68 to 0.76) and survival to discharge (OR 0.58; 95% CI 0.54 to 0.63) compared with weekday daytime.

There are several reasons to be careful in this situation. First, observational studies always suffer from unmeasured confounders. These are variables that can change the association between the proposed structure (decreased out-of-hours staffing) and outcomes (increased mortality from cardiac arrest), but that are not known or adjusted for. For example, earlier reports suggested that discharges from intensive care units (ICUs) at night were associated with worse outcomes,12 suggesting a need for a change in structure (increase in ICU or hospital beds to avoid night-time discharges). However, a more recent study demonstrates that this association was due to an unmeasured confounder, orders to limit life-supporting interventions13 and not due to lower quality of care.

One important confounder in the present study is that daytime arrests include events during surgeries and cardiac procedures. These patients will generally be less sick (they were well enough to go for a planned procedure), and the events are all witnessed with multiple staff present and usually occur in monitored settings, such as the operating theatre or postoperative recovery area. To address this issue, the authors conducted a subgroup analysis restricted to medical ward patients (presented in one of the supplementary appendices). Interestingly, ROSC during the daytime on weekends occurred at virtually the same rate as on weekdays, but night-time outcomes were still lower, as in the main analysis. And, hospital survival was still significantly worse for both weekend daytime and at night on any day. However, given the limited data with which to perform risk adjustment in this cohort, it remains entirely plausible that the differences in outcomes observed in the current study merely reflect unmeasured confounders, such as sicker patients suffering cardiac arrest out-of-hours.

Second, healthcare organisations are heterogeneous in their approach to organised care and structures that may be optimal in one setting could be either detrimental or unnecessary in different settings. For example, while there is wide acceptance of high-intensity staffing models (where patients in ICUs are seen by a specialist in intensive care medicine), data from a large cohort study suggest that ICUs without high-intensity staffing models may also perform well.14 It would be a mistake to enforce a high-intensity model in these units, as it would increase the costs of healthcare, with unclear benefits. Another example is the use of night-time intensivists. In a large cohort study, night-time intensivists conferred no benefit except for units with low-intensity staffing during the day.15 The current study would provide a more compelling case for change if it could demonstrate that the effects of out-of-hours cardiac arrest are modifiable by certain organisational characteristics, such as the intensity of staffing levels, the number of high-dependency beds and the size of the organisation.

Third, some observational studies describe associations that are more complex than they initially seem, so that changes to existing structures may fail to improve outcome (or may even make them worse). For example, the USA limited duty-hours for residents, with the assumption that less sleep-deprived residents would provide better care.16 However, the effects were not necessarily what was expected, as residents continued to sleep the same amount of time at night17 and there have been no improvements in patient outcomes until now.18–20 Several assumptions would need to hold true for a change in out-of-hours staffing to improve the outcomes of cardiac arrest. First, the extra providers would need to be immediately available and close to the ward where a cardiac arrest is happening. Second, they would need to be rested both physically and mentally. And third, they would also need to have the same expertise and engagement as providers available during regular working hours.

In conclusion, this report adds to the existing literature that suggests worse outcomes during out-of-hours periods.21 However, as often happens with observational studies, it remains unclear to what extent the relationship is causal. As with many important policy decisions, as opposed to specific clinical therapies, we are unlikely to see a randomised trial that settles the question. But, if healthcare systems decide to intensify staffing out-of-hours in response to these data, we strongly recommend the development of a rigorous implementation strategy, including a high-quality prospective evaluation of the implementation, service delivery and expected benefits, including the cost-effectiveness.