APPROACH

Critical review of the literature and design of a compendium of methods and tools

Descriptions of methods and tools in quality improvement and highly illustrative examples of their application were sought using search engines on the internet and from the ANAES library catalogue, institution reports (Institut de Recherche et Développement de la Qualité (IRDQ), Institut Qualité et Management (IQM),¹⁵ Mouvement Français pour la Qualité (MFQ)) and educational aids from consulting agencies. We searched three electronic databases from 1993 to September 1999 using a wide range of English and French text words; 364 references were retrieved from MEDLINE, 284 from HealthSTAR, and 470 from PASCAL. Relevant citations were identified in retrieved documents to generate a “snowball” effect.

Over 200 different terms relating to quality control, quality assurance, quality improvement, or total quality management were identified from this literature search and these were set out in the form of a matrix of methods/tools (rows) by their descriptors (columns). This procedure gave us a clear overview of the field and highlighted synonymous terms. It also enabled us to define a method and a tool (they are not always clearly distinguished in the literature). We defined a method as a set of principles governing the measures that are implemented and the techniques that are used to obtain a result. A tool was defined as a means, with a specified mode of operation, of carrying out an action implicated in a method. Tools that comprised several of the following steps were defined as methods: choosing the project topic, making a diagnosis of current status, defining a new way of working or building a new process, implementing remedial actions, checking results or measuring improvement, and adjusting the process or undertaking new actions to meet the objectives that were set.

When drawing up our matrix, we identified 16 well documented methods (to which we added two promoted by ANAES) and 122 specific or generic tools of which 22 were implicated in several methods and often cited. A standard form was completed for each of these 16 methods and 22 tools. The main items were: definition of the method or tool; field of application; objective(s); synonymous or variant tools and methods; historical background; resources needed; conditions for successful implementation; and brief description (in particular, steps to be carried out). Each method/tool was illustrated by a published example of its use in health care. The descriptions and examples were combined in a draft loose leaf compendium.

Evaluation of the compendium

Because most quality improvement methods have their roots in industry, we first submitted our draft compendium to three quality experts from the industrial sector during individual face-to-face interviews. Two of the experts were in charge of Masters’ degree courses at leading French business schools, the third was in charge of a “methods and development” unit in the aviation industry. Their opinion was that our selection had not omitted any essential method or tool. More advanced methods and tools were available but they did not recommend their use by “beginners” or “intermediates” in the healthcare sector.

At this time we also enquired whether HCO staff were already familiar with these methods and tools in order to gauge the potential usefulness of our compendium. According to information provided by the ANAES network of correspondents, staff in HCOs were aware of only six methods: clinical audit,¹⁶, basic PDCA, FOCUS-PDCA, failure mode effects and criticality analysis, hazard analysis critical control point (HACCP), and problem solving.

To confirm the relevance and applicability of the methods and tools to the healthcare sector we submitted the draft compendium to a panel of 13 healthcare professionals selected for their expertise in quality, safety and/or legal issues, risk management, hospital accreditation and/or statistics. The panel included five doctors, two nurses, three hospital managers, two quality managers, and a legal expert. They were affiliated either to public or private French HCOs, the French standards institute (AFNOR), the national blood transfusion institute (INTS), the French federation of private hospitals (FEHAP), or an establishment providing private hospitals with training and advice in quality issues. One member of the panel was from the Agency for Healthcare Research and Quality in the United States. They were convened to two meetings at which they had to arrive at a consensus on the methods and tools best suited to quality initiatives in French HCOs.

The panel selected 13 methods (table 1) and 20 tools (table 2) from our list and added one method (“morbidity and mortality conferences”) specific to the healthcare sector. They rejected five methods and two tools (table 3). They also drew up a list of factors they considered conducive to successful project implementation, irrespective of method (box 2).

The compendium was revised in the light of their comments and, to make sure it met the needs of potential users, it was submitted to a review group (five further experts in quality in healthcare including an expert from Quebec and the ANAES network of over 40 correspondents who had prompted our study). The process of the design and evaluation of the compendium is illustrated in fig 1.

CLASSIFICATION OF METHODS AND TOOLS

The panel of 13 healthcare professionals did not rank the methods and tools because performance depends largely on objectives, context of use, and resource availability.

Methods were classified according to their complexity, main thrust, and nature (table 1). Three levels of complexity were distinguished. Level 1 methods are the simplest, best known, most used, or mandatory methods in HCOs (e.g. HACCP is compulsory in catering). Level 2 methods require staff who already have some experience in continuous quality improvement. Level 3 methods often require specialist help and advice. The main thrust of the method could be the study of a process, a comparison with a standard or with other institutions with similar activity, a problem to be solved, a special concern for the expectations and needs of customers, a focus on costs or on leadership by the management. Methods could be based on general principles and involve a subdivision into steps during which other methods are applied; they could be focused on safety and prevention.

The taxonomy of tools drawn up by the panel of experts is given in table 2:

• Problem solving tools (tools 1–6) which correspond fairly closely to “the seven QC tools” elaborated in Japan⁶ (the group rejected control charts which are one of the seven Japanese tools).

• Quality management tools for expressing vague problems in clear factual terms so that group members can reach a consensus (tools 7–12). These correspond to the seven management and planning tools of total quality management outlined by Plsek,⁷ except for factorial analysis which was rejected by the group. Moreover, Plsek grouped together the related program evaluation and review technique (PERT) and critical path method (CPM) developed independently in the 1950s.

• Miscellaneous tools.

All tools were characterised according to their main actions—collect data, analyse, visualise, measure, monitor, or select.

EVIDENCE FOR INTEREST IN THE COMPENDIUM

The compendium met with considerable success when it was published in September 2000. Its publication was announced in a press release sent to all HCO managers, and a four page summary was sent to the presidents of hospital medical committees and to the heads of nursing. The contents were presented at meetings attended by members of professional societies, hospital federations, academic institutions, etc. The compendium was posted on the ANAES website (www.anaes.fr) and ranked among the top three ANAES publications. It was downloaded an average of 900 times per month during the first 6 months.

A telephone survey of 900 managers, doctors and carers in 321 HCOs carried out in July 2001 indicated that eight out of 10 HCOs had a quality policy, 63% of the interviewees could name at least two ANAES publications relating to quality improvement or health technology assessment, 61% said that work by ANAES had prompted quality initiatives in their HCO, and 74% considered that the HCO paid heed to ANAES recommendations.

DISCUSSION

Several reasons may explain interest in the compendium. Although staff in HCOs aspire towards quality improvement, they do not always know how to set about it. We wanted to provide them with a practical guide that would enable them to enter a process of active learning and would offer them a variety of methods and tools for reviewing, assessing and adapting their own practice. This was, in our mind, a pragmatic way of inciting HCOs to adopt a project orientated approach towards quality improvement and of helping them to choose the approach(es) that suited them best at a local level.

The compendium was designed to best satisfy their needs. Its contents were validated at several levels by quality experts in industry/business/healthcare and by potential users. Its classification of methods and tools, examples of applications, and loose leaf format mean that it can be easily used by professionals with different levels of background knowledge in quality issues. Above all, the publication of the compendium was timely. All HCOs in France have to undergo accreditation. The accreditation reports already received at ANAES indicate that HCOs are increasingly aware of the need for information on quality improvement methods and for appropriate technical expertise. Many have had to recruit qualified quality experts, train their staff in the use of quality tools and methods, and learn to employ consultants to assist them in their projects. The compendium has proved to be a valuable aid.

In conclusion, the ANAES initiative to design a compendium of methods and tools has highlighted three important factors in the promotion of quality improvement: (1) the need to meet a demand expressed by potential users, (2) the need to be user friendly, and (3) the need to provide a timely response. However, quality depends not only on technical expertise but also on department organisation, human resources, staff morale, risk management, and public input. Particular attention should also be paid to these dimensions.