HISTORY OF THE POCA REGISTRY

The POCA Registry is an offshoot of the American Society of Anesthesiologists (ASA) Closed Claims Project and the work of the Committee on Quality Assurance of the American Academy of Pediatrics Section on Anesthesiology. The ASA Closed Claims Project is a systematic review of closed anesthesia malpractice claims that began in 1985 and is ongoing, with a current national database of 5480 cases. It is sponsored and directed by the ASA Committee on Professional Liability, with day to day management coordinated by a steering committee at the University of Washington in Seattle. The Closed Claims Project database provides detailed information on adverse anesthesia events that resulted in malpractice claims, and has proved to be an effective mechanism for describing patterns of relatively rare events and possible causative mechanisms.¹ These findings have contributed to the development of practice guidelines as well as stimulating clinical and laboratory research to investigate mechanisms of anesthetic related injury.

Analysis of closed pediatric anesthesia malpractice claims² suggested differing patterns of injury from adult claims. In particular, pediatric claims had a higher mortality rate (50% v 35%) and more respiratory events (43% v 30%) than adult claims. While hypoxemia was suggested as a common mechanism of injury in pediatric claims, there was insufficient information in the Closed Claims Project database to explain the underlying causes of hypoxemia. The Closed Claims Project collected detailed data on each claim (the data form is 10 pages), yet because there were so many different kinds of anesthetics and adverse events, data could still be insufficient to explain a particular event such as cardiac arrest among pediatric patients. For example, the Closed Claims Project collected information on anesthetic agents and techniques, but not drug doses and inhalation agent concentrations. Cardiac arrest and resuscitation details were also lacking.

A different strategy was needed to collect sufficiently detailed data to explain causes of severe adverse events among pediatric anesthesia patients. Dr Jeffrey Morray, who was Director of Anesthesia at Children's Hospital and Regional Medical Center in Seattle, WA and lead author of the closed pediatric claims report, and Dr Alvin Hackel, then Chair of the Quality Assurance Committee of the American Academy of Pediatrics Section on Anesthesiology, discussed the possibility of a joint project. Pediatric anesthesiologists from over 50 hospitals throughout the United States and Canada expressed interest in contributing data to a registry to investigate causes of cardiac arrest among pediatric anesthesia patients. With leadership and broad participation in place, the additional elements needed to implement such a project were financing, staffing, and a steering committee (table 1). Since organizational and analytical expertise were already available within the established infrastructure of the Closed Claims Project, the ASA Committee on Professional Liability agreed to sponsor the extra costs associated with its implementation. Using the pre-existing staff and resources of the Closed Claims Project, with an overlay of a Steering Committee of pediatric anesthesiologists (table 1), the POCA Registry was officially established in April 1994 to investigate the incidence and causes of cardiac arrest associated with administration of anesthesia to children.

ORGANIZATION OF THE POCA REGISTRY

The two primary objectives of the POCA Registry were to determine the incidence and investigate the causes of perioperative cardiac arrest among patients aged ≤18 years. While case reports were the primary interest because they would shed light on the etiology of the arrest, estimation of incidence required a structure that would also provide a denominator from which these cases were drawn. The Steering Committee therefore determined that cases would be collected from a group of participating institutions that would provide case volume statistics in addition to detailed reports of perioperative cardiac arrest. Data would be collected at a central coordinating center at the Department of Anesthesiology of the University of Washington in Seattle for inclusion in a North American database (fig 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Organization of the POCA Registry Panel. Panel members are hospitals, anesthesia departments or services (group practices) that submit case volume and case report data to the coordinating center. Case volume represents anesthetics administered to patients aged 0–18 years, excluding neonatal and emergency room resuscitations. Case report eligibility includes cardiac arrest or death in the operating room or postoperative recovery area (excluding ICU). Case volume data are used to generate estimates of incidence and risk factors for cardiac arrest or death. Anonymous case reports provide data to investigate causes, associated factors, and outcomes of cardiac arrests.

The organization of the POCA Registry can be characterized as a “panel”—that is, a set of participants who submit data periodically to a central data collection facility. Panel members, in this case called “participating institutions”, were hospitals, anesthesia departments, or anesthesia group practices. Participation required that each panel member be able to report overall statistics on the total number of anesthetics administered each year as well as detailed data on every case of perioperative cardiac arrest or death among their pediatric anesthesia patients. Most participating institutions were hospital anesthesia departments, although some were anesthesia group practices (fig 1). A few were anesthesia groups that provided only partial anesthesia services at their hospital or provided anesthesia services at multiple locations. However, they conducted their own quality assurance review and were therefore able to report on any relevant cases as well as provide annual practice statistics. Each participating institution assigned an individual to be their POCA Registry representative. This individual was responsible for data collection and reporting at that institution and was the POCA Registry contact for newsletters and other communications. Some institutional representatives did all reporting of statistics and cases personally, while others distributed the workload within their organization—for example, assigning the attending anesthesiologist or quality assurance committee chair to complete case reports.

Panel membership was initially recruited by the Quality Assurance Committee of the Section on Anesthesiology of the American Academy of Pediatrics. This initial core set of panel members was enlarged by open recruitment through the American Society of Anesthesiologists and the Anesthesia Patient Safety Foundation newsletters. Presentations at annual meetings of the American Society of Anesthesiologists and the American Academy of Pediatrics were also used to publicize the project and recruit new panel members. In addition, the POCA Registry was added to the ASA Closed Claims Project website (www.asaclosedclaims.org) for educational and recruitment purposes.

The initial panel was mainly drawn from pediatric hospitals with academic affiliations. In order to broaden the panel to better represent pediatric anesthesia throughout North America, panel members were asked to recruit new members from community hospitals in their geographical region. This resulted in a number of new panel members from community hospitals without formal academic affiliations. The initial panel of 50 hospitals was expanded to over 70 institutions within the first year. Over the life of the POCA Registry some panel members have withdrawn while new panel members have been added. The panel has remained relatively stable at about 60 institutions during the last few years.

The incidence of cardiac arrest was calculated from annual case volume statistics provided by participating institutions (fig 1). It was soon realized that mid year enrolment created difficulties in conducting accurate incidence calculations. This problem was resolved by instituting annual enrolment. Institutions wishing to enrol as panel members were enrolled at the beginning of the calendar year. Institutions enrolling in the middle of the year would need either to report retrospectively any cases occurring during that year or to wait until the beginning of the next calendar year to begin participation. Because some institutional representatives would fail to communicate with the POCA Registry when they left their institutions without assigning a replacement, an annual re-enrolment policy was implemented. Each participating institution was required to submit a simple re-enrolment form at the beginning of the new calendar year. Failure to re-enrol after two reminders (and follow up telephone calls) resulted in the institution being dropped from the panel and their case volumes being dropped from incidence calculations.

ANONYMOUS CASE REPORTING

POCA Registry case reports were designed to provide a detailed account of perioperative cardiac arrest or death, including possible causative factors and medical errors that may have contributed to the event for patients 18 years of age or younger. Cardiac arrest was defined as the need for cardiopulmonary resuscitation (including closed chest massage) or death (regardless of cause). In order to investigate arrest associated with administration of anesthesia, only arrests in the immediate perioperative period (from induction through recovery room discharge or transfer to the intensive care unit) were eligible for report. Arrests occurring later in the postoperative course were excluded, as were neonatal and emergency room resuscitations.

Given the US malpractice litigation system, there was concern that institutions would be reluctant to provide case details to an outside organization when a claim for malpractice could be pending. There was additional concern that the POCA Registry Coordinating Center did not have a mechanism to protect identifiable data from legal discovery. It was therefore decided that case reports would need to be anonymous, without a link between the case report data and the submitting institution. Aggregation of case report data submitted by institutions throughout North America would provide extra assurance of anonymity for case reporters.

Anonymous case reporting presented logistical problems for research. One problem concerned reporting compliance: we did not know how many eligible cases were not reported to the POCA Registry. It was safe to assume that fewer than 100% of eligible cases were reported, but we had no mechanism to quantify missing data and their characteristics. In other words, we could not calculate a “response rate” and could not compare responders with non-responders to analyze possible response bias in the reports we did receive. Our calculation of the incidence of perioperative cardiac arrest and death was therefore biased downward. We could estimate a minimum incidence, but the true incidence was probably higher than our estimate.

Another problem arose from incomplete or missing data in the cases that were reported. The case report form was quite long and included a narrative section for elaborating on the sequence of events and possible causes of arrest. At times, a report raised questions in the Steering Committee that required additional details that were not included in the initial case report. At the suggestion of Dr William Runciman, Director of the Australian Incident Monitoring System,^3,4 we implemented a missing data request system. POCA Registry institutional representatives were asked to assign their own unique identification code (“submission number”) to each case report. This code had no meaning to the POCA Registry Coordinating Center. If the Coordinating Center received a case report for which we required additional details, we sent a “Missing Data Alert” to all members of the POCA Registry panel. This Missing Data Alert described a few key aspects of the case and requested specific additional details. The “submission number” assigned by the institutional representative provided the key for that representative to recognize the case and anonymously respond to the Missing Data Alert. Other institutions that may have had a case that sounded similar would know that the alert did not concern their case because the submission number did not match any cases they reported.

Another problem of anonymous case submission was the possible submission of cases by institutions that were not enrolled in the POCA Registry panel. It was possible for anyone gaining access to a case report form to submit a case without formally enrolling in the POCA Registry. With an event as rare as perioperative cardiac arrest, submission of cases from non-enrolees could seriously bias estimates of incidence. Sample case report forms included in the enrolment pack were clearly labelled as samples to prevent their use by institutions who did not complete the enrolment process. After a few years of operation and the withdrawal or dis-enrolment of a number of panel members, we realized that case report forms could be erroneously submitted by institutions whose enrolment had lapsed or had otherwise been discontinued. To address this potential problem we created year-specific case report forms. Institutions were provided with a supply of new case report forms upon re-enrolment at the beginning of each calendar year. The “year of event” on the case report form needed to match the year preprinted at the top of the form or the case would not be accepted. For example, a cardiac arrest occurring in 2001 that was submitted on a 2000 form would not be included in the POCA Registry database. Fortunately, this did not result in loss of cases and provided assurance to the Coordinating Center that all cases submitted were coming from enrolled members of the panel.

DATA COLLECTION

Case volume data for incidence calculation

Each participating institution was asked to provide case volume data at enrolment and annually at re-enrolment (box 1). These data provided a denominator for estimating the incidence of cardiac arrest and death in the international panel. They were reported in January for the previous year of enrolment. If an institution did not re-enrol (and therefore did not provide case volume data for their last year of participation), their previous year's data were used to estimate case volume for their final year of enrolment and case reporting.

When the POCA Registry had collected sufficient cases for preliminary data analysis, the Steering Committee realized that it would be valuable to have case volumes reported by patient age and ASA physical status. ASA physical status is a five point scale ranging from generally healthy to moribund and not expected to survive 24 hours with or without surgery.⁵ Since ASA physical status classifications have been found to be correlated with surgical mortality,⁶ these data would enable estimates of the incidence of cardiac arrest or death for different patient risk groups. Because such data were not readily available at all participating institutions, they were optional rather than required for enrolment. Some institutions estimated distributions of patients among age and ASA physical status groups rather than obtaining data from their clinical information systems. This difficulty in obtaining comprehensive and accurate age and ASA physical status data hindered attempts by the POCA Registry to provide risk estimates of cardiac arrest and death for different patient groups.

Case report data

The experience of the ASA Closed Claims Project was invaluable in developing the POCA Registry case report data form. The data collection instrument from the ASA Closed Claims Project provided a model for the POCA Registry. Since the POCA Registry was not collecting malpractice claims data, litigation details from the Closed Claims instrument were eliminated. Clinical details were added so that causes and factors associated with cardiac arrest and death could be better assessed. The data form included sections on the patient (age, sex, ASA physical status), the surgical procedure and team members, anesthetic agents and techniques, clinical events and physiological clues of impending arrest, outcome, and assessment of causative factors and potential for prevention (table 2). An important portion of the data form was the narrative section in which the sequence of events, differential diagnoses, and any details not recorded elsewhere were provided. The Closed Claims Project investigators had discovered that such narrative descriptions were often instrumental in analysis of adverse events. Narratives also provided data on theories of causation and outcome associations that were not evident when the data collection form was developed, but could be investigated retrospectively by analysis of information included in the narrative description of events.

Initial analysis of POCA Registry case reports suggested that details about resuscitation activities might prove valuable in explaining outcomes. The “Uttstein style” for reviewing and reporting in-hospital resuscitation⁷ was used as a basis for collection of detailed resuscitation data. These data included the status of the patient before the cardiac arrest (including cardiac rhythm and volume status) and the sequence and timing of resuscitation activities (pharmacological and other interventions). It was hoped that these data might prove useful in addressing the effectiveness of various resuscitation strategies.

Case reports were submitted intermittently by participating institutions. We asked that case reports be submitted as soon as practicable after the occurrence of an eligible cardiac arrest or death. However, the Coordinating Center received batches of forms at the end of each year, suggesting that some participating institutions collected case reports for annual submission. Each case report was carefully reviewed by POCA Registry staff and Steering Committee members for data consistency, coding, and completion. Assessments of the cause of cardiac arrest were independently reviewed by at least two members of the Steering Committee. Disagreements in differential diagnosis were resolved by discussion among committee members. This process, adapted from the ASA Closed Claims Project, has been shown to result in reliable assessments.⁸ Case report data were entered into a computer database for aggregate analysis. Original case report documents were stored in waterproof and fireproof cabinets at the Coordinating Center.

INITIAL FINDINGS OF THE POCA REGISTRY

The first report from the POCA Registry appeared in July 2000 and included cases collected between 1994 and 1997.⁹ Because many cases of cardiac arrest and death among very sick children resulted from the surgical or underlying disease process rather than anesthetic factors, an attempt was made to separate these cases from events that appeared to be related to anesthesia care. Among the initial 289 cases reported, 150 (52%) were assessed as being causally related to anesthesia care rather than surgery or underlying disease. This assessment does not imply that anesthesia care was substandard or that errors were made. Rather, the assessment was based on the Steering Committee's evaluation of the probability that the anesthetic agents or procedures were responsible for the arrest combined with the absence of surgical or disease factors to explain the adverse events. Cardiac arrest related to anesthesia was estimated to occur in a mean (SD) of 1.4 (0.45) per 10 000 pediatric anesthetic patients. If case reporting by the panel of participating institutions was incomplete, this estimate was low and the actual incidence of anesthesia related cardiac arrest was higher.

The majority (55%) of anesthesia related cardiac arrests occurred in infants <1 year of age. Medication issues were associated with 37% of all anesthesia related cardiac arrests and 64% of arrests in previously healthy children (ASA physical status 1 and 2). Most of these cases involved acceptable doses and administration of common anesthetic medications. Cardiovascular system problems were associated with another 32% of arrests. The overall mortality rate following anesthesia related cardiac arrest was 26%, with severe underlying patient disease and emergency surgery strongly associated with poor outcome.

LIMITATIONS AND FUTURE DIRECTIONS

Some obvious limitations of the POCA Registry method of investigating rare adverse events have been suggested during the course of describing the project:

• under-reporting of cases;

• lack of detailed patient data in case volumes for risk assessment;

• over-representation of academically affiliated institutions in the panel; and

• reliance on voluntary participation resulting in a non-random, possibly biased sample that may not reflect the North American pediatric anesthesia population.

When the data were analyzed and factors that appeared to be associated with cardiac arrests and their outcomes were delineated, other limitations also came to light.¹⁰ For example, many of the medication related cardiac arrests were associated with administration of halothane alone or in combination with other anesthetic agents. While it has been hypothesized that some children may be excessively sensitive to the known tendency of halothane to depress the myocardium and subsequently experience cardiac arrest when exposed to normal doses, the POCA Registry had no denominator data on halothane usage among the panel of participating institutions. We could not investigate whether the incidence of cardiac arrest differed among children exposed to halothane compared with other anesthetic agents. Because of the anonymity of case reporting, we could not investigate whether certain types of hospitals (children's hospitals, hospitals with academic affiliations) had a different profile of events and outcomes than other institutions.

There are many risk factors for cardiac arrest and its outcome that cannot be explored by POCA Registry data. It has been suggested that a multi-institutional prospective case/control study collecting detailed data from pediatric anesthetic patients that did not result in cardiac arrest or death (controls) as well as data on cases would overcome some of the major limitations of the registry approach to studying rare adverse events and outcomes.¹⁰ This would be a major endeavor, requiring additional resources and coordination beyond the current infrastructure and funding, but would represent an important and valuable next step in understanding anesthesia related cardiac arrest in children. In the meantime, the POCA Registry contains a wealth of data from cases collected since the initial report, in addition to the resuscitation details added to the case report forms, that may provide further insights into the incidence, causes, and outcomes of cardiac arrest in children undergoing anesthesia.