What makes any improvement (or its targeted problem) ‘problematic’?

Something can be difficult without being ‘problematic’. When we know what work needs to be done to achieve a goal, we knuckle down and do the work. That is not problematic. If we do not have the time or resources to invest in this work, we walk away (‘I can't afford to do that’). Or, if we regard the goal as worthwhile, we make the time, find the money, or invest whatever other resources are required to achieve it.

But, what if we make these investments of effort and resources, yet fail to achieve our goal? Then, we have a problem. If it happens just to us, we ask for advice from someone who has succeeded. When we find, however, that many people have failed to achieve the same goal despite efforts similar to ours, then we have a deeper, shared problem.

Problematic improvements

An improvement we expect to work can fail to deliver satisfactory results in a number of ways.

1. The intervention may commonly be implemented inadequately: the correct way is known, but not widely appreciated. The surgical checklist comes to mind in this regard. Non-experts often do not recognise that achieving improvement can require changes to teamwork, culture or workflow that make the overall effort far from simple or easy,1 ,2 and perhaps not even effective at all,3 ,4 when unaccompanied by these other more labour-intensive changes.

2. The intervention requires much more effort or expertise than generally recognised. Medication reconciliation presents an example of this type. Studies that report meaningful improvements in medication safety almost all involve medication reconciliation carried out by pharmacists.5 Yet, the vast majority of hospitals cannot afford to employ enough pharmacists to carry out this activity. Instead, most hospitals create new forms to support doctors or nurses performing medication reconciliation as part of their workflow. Without any deeper efforts to create the time for staff to do anything differently, ‘the best possible medication history’ supposedly generated by this medication reconciliation process may well consist of the same old error-ridden medication history as before.

3. The intervention is commonly misused—it achieves one goal well, but is more commonly misapplied to another goal. Incident reporting systems provide an example of this type. Incident reporting is meant to capture ‘critical incidents’ that shed light on system problems. The rate at which such events occur does not matter. What matters is why they happened and if careful characterisation of the contributing factors reveals changes that could lower the risk of similar incidents in the future. In practice, however, incident reporting systems serve to generate pie charts and bar graphs. ‘We had X medication incidents this year and Y incidents last year’. Or, ‘As you can see, the most common type of incident is falls, followed by medication problems’. Incident reporting systems do not quantify the frequency or rate at which a problem occurs. Variations in the number of incidents and the relative frequencies of different event categories say more about changes in reporting behaviour than they do about the rate at which harm occurs.6

4. The solution can work, but frequently fails to deliver in practice. Electronic health records and computerised order entry easily come to mind in this category. But other examples might include root cause analysis, standardisation, and efforts to reduce alarm fatigue, to name a few.

Problematic problems

Problems can require substantial effort to solve without being ‘problematic’. When an elegant solution to a mathematical problem is not apparent, mathematicians sometimes say that it can be solved through ‘brute force’. Application of well-known methods will produce a solution eventually. But sometimes applying known methods and substantial effort fails to deliver any useful solution. In such cases, the problem sometimes lies not with our solutions per se, but rather with our formulation of the problem itself.

Conceptually, one might apply the framework of Glouberman and Zimmerman7 and say that problematic problems arise when we misconstrue which type of problem we face—simple (cooking from a recipe), complicated (building a rocket to the moon), and complex (raising a child). Many persistent health policy problems (eg, reducing overcrowding in emergency departments and lowering rates of readmission to hospital) probably reflect a failure to distinguish complicated from complex problems.7 In the case of preventing readmissions, the hospital side of the equation has usually been characterised as simple (albeit effortful), involving bundles of strategies, many of which appear fairly simple in themselves—patient education about discharge plans, medication reconciliation, having follow-up appointments in place prior to discharge, checking in with patients by phone to ensure no problems with medications or follow-up appointments, and various other strategies. Where it becomes much more complicated, possibly even truly complex, is determining how care delivery needs to change in the outpatient setting, possibly including policy and funding decisions affecting all sectors of the health system, not just hospitals, but clinics, home care services, nursing homes and rehabilitation facilities.

Less conceptually interesting but still practically important instances of problematic problems include:

1. Focusing on a problem out of proportion to its importance.

2. Faced with any persistent problem, one can rationally ask the question: ‘Is this problem really so worth solving after all?’ Maybe it is important after all and we just need to remind ourselves why before returning to the fray. Alternatively, maybe our efforts to articulate the importance of the problem leads us to shift our focus to something else. Inpatient falls strikes us a commonly targeted quality problem that would benefit from consideration in this way.

3. Or course, it is unfortunate when a patient has a serious fall. But, the rate of injury is very low, interventions that reduce fall-related injuries remain elusive, and focusing on falls reduction risks interfering with improving mobility, which is probably a more important goal for hospitalised elders.8 Our point is not that falls should be abandoned entirely as a target of improvement efforts, just that the field would benefit from an article clearly articulating the pros and cons of focusing on this problem.

4. Failing to appreciate the complexity of a problem.

5. Not all interruptions are bad and simply counting how frequently they occur in various clinical settings serves little purpose. The desirability of preventing an interruption depends on a complex interplay between the content of the interruption, the task being performed, and the potential for recovery strategies, among others.9 ,10

6. Strategies that sound like solutions but still haven't been worked out, so are really unsolved problems. For instance, it is fine to say we should engage patients or the public in designing improvement strategies or playing roles on committees, but how to do so as more than a token exercise remains unclear. How does one patient (or even several) stand in for all patients? How do we assess the degree to which the intended benefits are occurring or even something as basic as the degree to which patients have been usefully involved in the process? Improving culture might provide another such example. We've all heard the line that ‘culture eats strategy for breakfast’, so it seems quite reasonable to say we have to improve culture. But, we know so little about how to achieve this goal11 that ‘improving culture’ is currently a problematic problem as much as it is an intervention.

Open for debate

Of course, we recognise that many of these topics will generate some controversy. Unnecessary controversy can perhaps be avoided if we clarify that any given ‘The problem with X’ article in the series might just as easily be called ‘Beware of X’ or ‘Pitfalls to avoid when undertaking X’. In other words, ‘The problem with incident reporting’ does not equate with ‘Stop bothering with incident reporting altogether’. But, we would want readers to have in one place a concise summary of the myriad barriers that prevent accomplishing anything useful with incident reporting systems.

That said, advances in human understanding benefit from controversy and debate. So, we welcome debate on the topics included in this series. We hope to publish most responses to articles in the series on the journal's website. And, we will undoubtedly publish some counterarguments as formal letters to the editor. In cases where we anticipate strong arguments for and against calling something problematic, we will invite a pair of authors for a point–counterpoint debate. One recent example (submitted and accepted for publication before we decided to launch the series) involved a debate by two experts in patient safety for and against the ‘no blame’ approach to improving patient safety.12

Venting versus advancing the field

We all have pet peeves or topics that make us wince due to disastrous experiences. Some of us are wont to vent over external regulations, performance measures, or various others topics. And, sometimes working in quality improvement full-time can make one sick of hearing about certain recurring strategies or problems. We hope to avoid choosing topics that speak only to personal peeves, but rather represent compelling cases of needing to broaden our thinking in the field on these topics. We frequently find in complex socio-technical systems that our assumptions about causes and solutions are wrong. Contention and debate can challenge such assumptions and prompt reconceptualising the problems we target or how we tackle them.

Even if you know you cannot really wave goodbye to incident reporting systems, a narrow focus on hand hygiene compliance, standardised mortality ratios, or hospital readmission rates among other topics potentially covered in the series, reading why you might be justified in doing so may at least provide an enjoyable diversion from your efforts. More substantively, though, we hope the material covered by articles in the series will arm you with new ideas and insights that will make future efforts with these deceptively difficult solutions and problematic problems more likely to succeed.