• qualitative research
• quantitative research

In considering the place of incident monitoring in the overall scheme of things, one is reminded of Peter Ustinov's anecdote about his father who is reported to have enjoyed entertaining the fairer sex; he was “ . . . always galloping, like a daring scout, in the no-man's-land between wit and poor taste”.¹ Promoting incident monitoring to one's scientific colleagues is an analogous activity. Like telling a risque story, it can be enjoyable and may yield unexpected, interesting outcomes; however it is, at least at the moment, only marginally respectable.

Despite widespread reservations about its pedigree, incident monitoring is classical qualitative research, with attributes and limitations which are familiar to social scientists. Many biomedical scientists dismiss activities which cannot capture a numerator and a denominator, but in doing so, may constrain their horizons and limit the scope of their research. Indeed, some of our colleagues simply sit back and enjoy the status quo, comfortable with conventional dogma and the knowledge that it cannot easily be challenged using conventional quantitative techniques.

Quantitative methods have been the mainstay of traditional biomedical research. There is no doubt that the “gold standard” for establishing the efficacy or applicability of a treatment or technique is a randomized, prospective, double blind study; ideally, all new forms of medical treatment and, indeed, all existing forms, should be subjected to such scrutiny. However, there are frequently great difficulties pursuing this course—logistic, political, financial and ethical. Studies may be carried out using quantitative methods of less rigour, but the possibility of erroneous conclusions increases the further one moves from the classical prospective study. All too often, nothing is done at all, with the result that much of our professional activity continues to be empirically based. There are some constraints which are peculiar to quantitative research. An a priori hypothesis is required; this may limit the chance of a truly new finding. Indeed, Claude Bernard taught that new information usually lay in “outliers” of data sets, not in the body of information substantiating a hypothesis.² Another disadvantage is that values must often be reduced to numbers using measurement techniques which may only capture one facet of a multi-faceted phenomenon. However, the main constraint of quantitative research may be that studies of adequate design may be so expensive to set up and difficult to run, that nothing at all is done.

Qualitative research has its own set of applications and limitations. It may be particularly useful where problems are “complex, contextual and influenced by the interaction of physical, psychological and social factors”²; it would thus seem well suited to probing the complex factors behind human error and system failure.

Unconstrained by the need to reduce the data to a set of numbers, qualitative research may allow phenomena to be studied from more perspectives and in greater depth; it may also allow studies to be more easily carried out in a normal environment and during routine work. In this sense, qualitative research lends itself to a naturalistic approach. Data collection methods include observation, interviews, focus groups, questionnaires, narratives and video- and audio-tape recordings.³

The incident reporting study described in this symposium exploited several of the advantages inherent to qualitative research. Its overall structure was relatively unconstrained as anaesthetists were asked to report “any unintended incident which did, or could have reduced the safety margin for a patient”. It used a questionnaire with both unstructured (free narrative) and structured components (to reliably find out, for example, which monitors were in use). The data were then classified according to the task in hand (e.g. defining the role of monitors, testing the validity of an algorithm). Established methods were used to test reliability and validity, e.g. inter-observer reliability, concurrent validity (where one method of research yields similar results to another) and construct validity (such that observations are consistent with current theory).^3–5

“Good qualitative research should give answers which are plausible, fit other evidence that we are aware of, be convincing, and should have the power to change practice.”³ The incident monitoring study reported in this symposium provides a comprehensive qualitative picture of current clinical anaesthetic practice, and is a powerful tool for “continuous quality improvement” at “grass-roots” level. Its use is entirely consistent with the philosophy of “kaizen”—“continuous improvement involving the entire system and everyone in it”. Indeed, it has been suggested that attention to continuous improvements in process rather than a preoccupation with objective evidence of improvements in outcome may be the main difference between the successful Japanese model and the less successful “Western” one.⁶ There are many improvements which can be made to the “process” of the anaesthesia “system” which eliminate potentially dangerous situations at nominal cost. Examples include changing the size of connectors to prevent tubes being joined in dangerous configurations, validating crisis management algorithms, refining check-lists, detecting and correcting deficiencies in practice, and adding additional sequences to the tasks that should be carried out during equipment maintenance.⁷

However, incident reporting cannot provide information with which to compare one individual or one institution with another; indeed, if it could it is likely that the quantity and quality of reporting would be adversely effected. At the moment, most of the “quantitative” systems in place cannot do this either. The variability in referral patterns and casemix will require expensive and potentially distorting “correction factors” to allow valid comparisons. What to do with those who end up in the fifth percentile (some of whom would be there by chance) does not appear to have been addressed. It would seem more suitable to direct one's attention to the entire system, allowing a qualitative rather than a quantitative approach.

Qualitative research usually starts with observations which, when categorised, may suggest the formulation of theories and hypotheses, whereas quantitative research uses measurements to prove or disprove existing hypotheses. The two approaches are complementary; good qualitative research may be necessary before a prospective study can be designed which has a high probability of having adequate statistical power.

Let us address the relative merits of quantitative and qualitative research by examining the contribution of each approach to the difficult question of whether one can justify the use of pulse oximetry for every patient undergoing anaesthesia.

The use of pulse oximetry was prospectively randomized for 20 802 patients.^8,9 Because no significant differences were shown between the groups with and without oximeters for certain “outcomes”, it was concluded by some observers that either pulse oximeters lacked efficacy or that the sample size was too small¹⁰; however, significant differences were shown for the detection of hypoxaemia, hypoventilation, endobronchial intubation and myocardial ischaemia, with a trend towards fewer cardiac arrests.⁹ Had the information published in this symposium been available at the time of designing this study, considerable time and effort could have been saved. Firstly, the “outcomes” chosen would not intuitively be expected to have been influenced by the use of pulse oximetry, with the possible exceptions of cardiac arrest, postoperative coma and myocardial infarction, for which it was acknowledged the sample size was too small.⁹ Secondly, incident monitoring yielded identical conclusions: with oximetry, significantly more cases were detected with hypoxaemia, endobronchial intubation, myocardial ischaemia and hypoventilation (when other “disconnect” monitors were not used) and there was a strong trend towards fewer full cardiac arrests under general anaesthesia (p=0.018).⁴

Both studies have virtually identical messages for the practising anaesthetist, but neither provides outcome figures justifying oximetry that would satisfy the doctrinaire quantitative biomedical scientist. However, I would argue that sufficient evidence has been provided to justify the strong recommendation for all anaesthetists in Australia and New Zealand to use oximetry for every case. Problems with the airway, ventilation and endotracheal tubes which lead to hypoxaemia and hypoventilation have been responsible for at least one third of preventative deaths and cases of brain damage over the last few decades^11–13; oximetry indisputably detects these far earlier.^9–14 Oximetry could have prevented one third of the deaths attributable to anaesthesia in our incident monitoring study (excluding surgical deaths due to uncontrolled bleeding).¹⁵ If we accept the preoximetry figure of one death solely attributable to anaesthesia for each 26 000 cases as being representative of Australian practice,¹⁶ then oximetry should prevent one anaesthetic death for each 78 000 cases. Pulse oximetry seems to have reliably picked up desaturation in well over 90% of the cases in which it was used,¹⁴ so let us assume it would prevent one death for each 100 000 cases. To provide an oximeter from before induction until discharge from the recovery ward would cost no more than $2 per patient; each life saved would thus cost about $200 000. Taxpayer funded road improvements are generally considered profitable up to a cost of $1.6 million per life saved¹⁷; this translates to purchasing “quality-adjusted life years” for an amount equivalent to many of the more expensive treatments in our hospitals ($30 000 to $60 000).¹⁸ Oximetry would seem to save lives at one eighth of this cost. The cost of brain damage, which results in greater insurance payouts than death,¹³ has not been addressed. A payout of $2.3 million was awarded recently in Australia for a case in which hypoxic brain damage followed a ventilator problem.¹⁹

When one considers the many other advantages of pulse oximetry and the fact that no value has been placed in these calculations on the peace of mind of the anaesthetist, on the quality of practice, on its utility as a teaching tool and on sequelae other than death, then a strong recommendation for the routine use of pulse oximetry in Australia and New Zealand would seem to be thoroughly justified (particularly at less than $2 per case).¹⁴

Quantitative and qualitative research are complementary; each provided the same information in the example chosen above. The quantitative study was a bold inititiative which also provided the incidence of various events and outcomes, but at a cost far greater than the qualitative study reported in this symposium. Incident monitoring is easy and cheap to implement and provides a wealth of information not only about oximetry, but the entire system. It can also provide a continuous monitor of how the system is changing in response to the implementation of strategies for improvement. I would argue that it represents good value for money when trying to balance cost, yield and feasibility—considerations of vital importance in an era of finite budgets and ever-expanding possibilities.^20,21