The Malcolm Baldrige National Quality Award provides a set of criteria for organisational quality assessment and improvement and has been used by thousands of business, healthcare and educational organisations for more than a decade. The criteria can be used as a tool for self-evaluation, and are widely recognised as a robust framework for design and evaluation of healthcare systems. As described by Batalden, Nelson and colleagues, a clinical microsystem can be defined as “the small, functional front-line unit that provides health care.”^1,2 In 2005, the Malcolm Baldrige Award took a specific interest in the clinical microsystem concepts, as they could operationalise the Baldrige criteria and bring them into mainstream efforts to improve the performance of large and small healthcare organisations. Towards this end, a focus on clinical microsystem thinking was included in the case study written for the 2006 annual examiner training. Following examiner training, the case study was made available to the public as a resource that is illustrative of an award application (available online at http://baldrige.nist.gov/Arroyo.htm).

The purpose of this paper is to explore the insights that can be gained from considering the Malcolm Baldrige criteria for healthcare in the context of high-performing clinical microsystems. Specifically, we discuss whether the microsystem success characteristics, identified in prior research,¹ cover the same range of issues addressed by the Baldrige criteria; examine whether this comparison might better inform our understanding of either framework; and investigate what we might learn about high-performing microsystems by looking at them through “Baldrige lenses”. Finally, we discuss the tools that are available for those wishing to engage in self-assessment based on the Baldrige Award criteria and microsystem characteristics. These are summarised and presented as recommendations for organisations ready to accept the challenge of using a system-based approach to improve the quality and safety of care provided to their patient populations.

OVERVIEW OF THE MALCOLM BALDRIGE NATIONAL QUALITY AWARD

The Malcolm Baldrige National Quality Improvement Act, signed into US law in August 1987, led to the creation of the Malcolm Baldrige National Quality Award in 1988. The purpose of this competitive award programme is to improve quality and productivity in the USA by establishing guidelines and criteria that can be used by organisations to evaluate their own quality improvement efforts.³ The Baldrige Award is given by the US President to businesses (manufacturing as well as service) and to education and healthcare organisations that apply and are judged to be outstanding in seven areas: (1) leadership, (2) strategic planning, (3) customer and market focus, (4) measurement, analysis and knowledge management, (5) human resource focus, (6) process management and (7) results.⁴

The Baldrige Award is designed and managed by the National Institute of Standards and Technology (NIST), a non-regulatory agency of the Commerce Department’s Technology Administration. NIST develops and promotes measurements, standards and technology to enhance productivity, facilitate trade and improve the quality of life. NIST was selected by the US Congress to design and manage the award programme because of its role in helping American organisations compete, its world-renowned expertise in quality control and assurance, and its reputation as an impartial third party. The American Society for Quality assists NIST with the application review process, preparation of award documents, publicity and information transfer.⁴ Awards were made to business applicants beginning in 1989. Criteria for healthcare and education were available beginning in 1999 and the first healthcare award was made in 2002. By the conclusion of the 2005 application cycle, a total of 116 healthcare organisations had submitted applications and five healthcare organisations had received the prestigious award, recognising their commitment to the values and concepts espoused by the programme.⁵ The Baldrige criteria have been successfully used by the Veterans Health Administration as a framework for the its Kizer Quality Achievement Recognition Grant. Similarly, 37 states in the USA offer quality awards based on Baldrige criteria.⁵

Table 1 summarises the Malcolm Baldrige Criteria for Performance Excellence and the specific items for healthcare that are based on those criteria. Similar to the criteria designed specifically for business and education, the healthcare criteria reflect some of the unique issues facing healthcare organisations. Figure 1 illustrates the relationship of the seven healthcare criteria, which are built on the following set of inter-related core values and concepts:

• Visionary leadership

• Patient-focused excellence

• Organisational and personal learning

• Valuing staff and partners

• Agility

• Focus on the future

• Managing for innovation

• Management by fact

• Social responsibility and community health

• Focus on results and creating value

• Systems perspective

OVERVIEW OF THE CLINICAL MICROSYSTEM CONCEPT

The conceptual underpinnings of our clinical microsystem work are based on ideas developed by Deming, Senge, Wheatley and others, who have applied systems thinking to organisational development, leadership and improvement.^6–8 The seminal idea for the clinical microsystem stems from the work of James Brian Quinn.⁹ Quinn’s work is based on analysing the world’s best-of-best service organisations, such as FedEx, Mary Kay Cosmetics, McDonald’s, Scandinavian Airlines and Nordstroms. He focused on determining what these extraordinary organisations were doing to achieve high quality, explosive growth, high margins and wonderful reputations with customers. He found that they organised around, and continually engineered, the frontline relationships that connected the needs of customers with the organisation’s core competency. Quinn called this frontline activity that embedded the service delivery process the “smallest replicable unit” or the “minimum replicable unit”. This smallest replicable unit, what we call the microsystem, is the key to implementing effective strategy, information technology and other key aspects of intelligent enterprise.

A healthcare clinical microsystem can be defined as the combination of a small group of people who work together in a defined setting on a regular basis—or as needed—to provide care and the individuals who receive that care (who can also be recognised as part of a discrete subpopulation ofpatients). As a functioning unit, it has clinical and business aims, linked processes, a shared information and technology environment and produces services and care which can be measured as performance outcomes. These systems evolve over time and are (often) embedded in larger systems/organisations.

As any living complex adaptive system, the microsystem must: (1) do the work, (2) meet member needs and (3) maintain itself as a functioning clinical unit. As we continue to move beyond conceptual theory and research to application in clinical settings, the emerging fields of chaos theory, complexity science, complex adaptive systems and lean production have influenced how these concepts have been applied to improving microsystems.^10–13 This is evident in the work to bring together microsystems from around the world to learn and share best practices (updates on these efforts are available at http://clinicalmicrosystem.org¹⁴).

STUDY OF HIGH-PERFORMING MICROSYSTEMS

In the late 1990s, Mohr and Donaldson conducted a national study of high-performing clinical microsystems. They identified 43 clinical units by using a theoretical sampling methodology. Semistructured interviews were conducted with leaders from each of the microsystems.^15,16 Additional research, described below, built on the Mohr and Donaldson study and conducted 20 case studies of high performing microsystems. This study included on-site interviews with representative members of the microsystems and analysis of individual microsystem performance data.^1,17–24

In the 2000–2002 study, funded by the Robert Wood Johnson Foundation, Nelson et al sought to identify success characteristics present in microsystems that provide high-quality, cost-efficient care. Using site visits, detailed interviews, direct observations, and reviews of medical records and financial information, a common set of 10 success characteristics was identified.^1,17–21 These characteristics have been described in detail and are summarised in table 2. As we considered these high-performing microsystems and the success characteristics, we felt it would be important to compare them to the Baldrige criteria, which is a widely recognised and established framework.

Methods

We created a database of fully transcribed interviews with members of 20 high-performing microsystems. This consisted of 223 documents linked to qualitative data analysis software, ATLAS.ti (version 4.1 for Windows). Two researchers coded the database using success characteristics, as described in detail elsewhere.¹

Of the 20 high-performing microsystems in the Robert Wood Johnson Foundation database, we selected 10 for additional analysis using the Baldrige framework. These 10 were felt to be the best performers in terms of quality and cost, based on reviews of the information gathered at site visits and on overall impressions of the team leaders. The transcripts from these 10 sites (a total of 110 documents) were then coded by two researchers using the Baldrige criteria in effect at that time (table 1).

Note that the transcribed interviews do not constitute a Baldrige assessment. While some questions posed by the interviewers were similar to those asked by Baldrige, many areas were not explicitly addressed in the conversations. Sites did not undertake the extensive self-assessment that Baldrige requires, and data to support results were not formally presented in the interviews. However, we felt that given the wide-ranging nature of the interviews and the focus on high performance, many areas covered in Baldrige would probably also be addressed during the site visits. The coders thus reviewed the transcripts for “answers” to the Baldrige “questions” in categories 1–6. In effect, researchers coded the transcripts based on which, if any, Baldrige criteria (“codes”) were invoked by each individual statement. Category 7 (results) was not coded, as actual results data were not presented in the transcript. Only material pertaining to the microsystem itself was coded with the Baldrige codes; comments about the larger organisation, or its relation to the microsystem, were coded as “macro”.

For all of the documents in the Baldrige sample, one researcher coded the even numbered documents and the other coded odd numbered documents. Together, the researchers read interviews from all 10 sites, for a total of 110 documents. Forty-three of these documents were detailed verbatim notes taken in the field; the remaining 67 were direct transcriptions of recorded interviews. A total of 3404 text quotations were selected for coding, with the Baldrige “codes” applied 4043 times, as some statements were responsive to more than one of the Baldrige codes. A grand total of 4359 codes were applied when the “macro” code is included.

To ensure that researchers were coding the interviews in a similar fashion, we checked 10% of the documents for reliability, based on a consultation with Research Talk, Inc. (Bohemia, NY, USA), a consulting company specialising in qualitative analysis. This meant that both researchers coded 10% of the documents. These documents were selected at regular intervals throughout the coding process. The other 90% of the documents were coded by one or the other researcher alone.

Analysis

In addition to analysis of the Baldrige coding, we also merged the Baldrige-coded database with the success characteristics-coded database. Multiple methods of analysis were used, including discussion among researchers about their thoughts and experiences coding, and discussion about the definitions of the Baldrige and success characteristics codes. The ATLAS.ti query and supercode tools were used to examine co-occurrences of the success characteristics and Baldrige codes.

For comparison of the success characteristics and Baldrige codes, co-occurrences were tabulated. For example, to compare the Baldrige criterion 1.1 (organisational leadership) with the success characteristics, all instances that had been coded as 1.1 were identified, and the database queried regarding which success characteristics occurred in those same segments of text. The reverse comparison was also performed, taking a given success characteristic and seeing which Baldrige criteria had been coded for the same material. This highlighted areas of correspondence and difference between the success characteristics and Baldrige codes. The overall distribution of the Baldrige codes was also tabulated.

Results

All 14 possible Baldrige codes (one for each of the healthcare specific items in categories 1–6 shown in table 1) were used; co-occurrences with all of the success characteristics were identified. This suggests that the two different frameworks cover similar conceptual territory, and supports the idea that the microsystem success characteristics that emerged from the original analysis are comprehensive. One area specifically addressed by the success characteristics is organisational support, which allows investigation of the relationship between the microsystem and the larger organisations that encompasses it, which we refer to as the mesosystem and the macrosystem which “surround” the microsystem. This relationship is not necessarily called out by the Baldrige system, which is designed more to look at an entire organisation than at the discrete and complementary subsystems that come together to form the whole enterprise. In our coding, this issue regarding the relationship of the microsystem to the larger organisation, was captured using the “macro” code (short for macrosystem or the larger organisation in which the microsystem was embedded), which occurred frequently (316 times). Both positive and negative comments about the larger organisation were coded as “macro”.

Analysis of which success characteristics co-occur with the Baldrige codes and which Baldrige codes co-occur with the success characteristics is informative. Table 3 presents the most common co-occurrences for each of the 14 criteria. Looking from the standpoint of the microsystem success characteristics, leadership, staff focus, patient focus, performance results, process improvement, and information and information technology have the greatest number of co-occurrences. In many cases, the relationships seem quite obvious: 1.1 (organisational leadership) with leadership, 5.3 (staff wellbeing and satisfaction) with staff focus, and so on. In other cases, examination of the success characteristics co-occurrences indicates ways in which the Baldrige criteria might be better understood and worked with by a microsystem. For example, 5.1 (work systems) had a number of success characteristics that frequently co-occurred. Although the most common was staff focus, the presence of interdependence, information and information technology, process improvement, leadership and patient focus suggests that all these aspects of the microsystem must be considered when designing the work that takes place there. Similarly, 6.1 (healthcare service processes) has as its major co-occurrences process improvement, patient focus, information and information technology, and performance results; the close links between process improvement, analysis of performance, use of information and maintaining a clear focus on patients are shown.

When all the Baldrige codes are considered, note that process improvement was among the most frequent co-occurrences with 12 of the 14; performance results co-occured with 9 of the 14. This confirms the importance of process literacy and measurement in these high-performing microsystems. Reading across the rows, the greatest number of co-occurrences are with 1.1 (organisational leadership), 5.1 (work systems) and 6.1 (healthcare service processes); 5.1 (work systems) was also the most commonly coded Baldrige item, followed by 6.1 (healthcare service processes) and 1.1 (organisational leadership). This suggests that the design of work systems and a deep understanding of processes are major contributors to success. Indeed, the high degree of process awareness often drives the design of the work, and the coders’ experience was that the two were often intertwined in the transcripts. Category 2 (strategy) was coded infrequently, but this may not reflect a lack of strategic thinking in microsystems. Often, discussions about strategy invoked the larger organisation, and these were deliberately not coded using the Baldrige criteria. We observed that many comments regarding strategy development and deployment currently reside in the “macro” code used to demarcate material relating to the larger organisation. In addition, there were relatively few questions specifically about strategy that were posed during the course of the interviews. 3.1 (Patient/customer and healthcare market knowledge) was also not commonly coded. This also probably relates to the questions asked during the interviews, rather than reflecting a lack of customer knowledge in these microsystems.

DISCUSSION

Our use of the Baldrige criteria to code transcripts from 10 high-performing microsystems was informative on several levels. First, the Baldrige system is robust; this is supported by the fact that, even in conversations not specifically designed to address the criteria, all were invoked. The frequency with which certain criteria (work systems, healthcare service processes, and organisational leadership) were coded underscores the importance of these areas in the work of the microsystem. The design of work and a deep understanding of process are crucial to outstanding performance, as is leadership.

The co-occurrences of the success characteristics and Baldrige codes can provide insights for microsystems seeking to improve their performance (or to understand their current level of performance). For example, an organisation seeking to better understand work systems (5.1) in the Baldrige system might find it helpful to know that while the most common co-occurrence with the success characteristics was with staff focus, there were also frequent co-occurrences with interdependence, information and information technology, process improvement, leadership and patient focus. This helps to clarify aspects of the microsystem that must be addressed when designing its work. For measurement and analysis of organisational performance (4.1), performance results was the most frequent co-occurrence as one might expect, but process improvement and information and information technology were also frequently coded, suggesting the importance of technology in measurement and analysis, as well as the need to understand processes to measure and analyse performance effectively. Interestingly, certain Baldrige codes frequently co-occurred with almost all of the success characteristics. That 5.1 (work systems) is among the most frequent co-occurrences for 9 of the 10 categories highlights that collaborative work models and an understanding of what motivates staff are important in all domains. An understanding of how work is designed, effective leadership and a focus on processes are vital to optimal performance; improvement in any domain must be informed by work in these areas.

Our study had several limitations. This analysis involved only 10 high-performing microsystems. The selection criteria were in part subjective, and although attempts were made to represent a variety of domains of high performance, others domains (such as community service) might have been relatively under-represented. Qualitative research is inherently subjective; indeed, the coders’ immersion in the data and personal understanding is a cornerstone of the process. The Baldrige coding was completed by two team members who frequently discussed their approaches and experiences, and a formal process to assess reproducibility was used, but even so there might have been underlying systematic differences in the way they coded, or even understood, the Baldrige criteria. As noted earlier, these transcripts in no way approximate a Baldrige self-evaluation. Many areas that might be explored were not addressed by the questions asked of respondents during the interviews. Similarly, the category which in Baldrige carries the greatest number of “points” (results) was not coded. Although the sites had provided information on clinical outcomes, quality and financial performance, this was considered only in choosing the 10 sites for the Baldrige analysis. Thus, our findings are limited because, unlike a completed Baldrige evaluation, we have no “hard evidence” of performance. Although results were not the focus of this analysis, note that they were assumed to reflect the high performance that led to the initial selection.

RECOMMENDATIONS: TOOLS AND RESOURCES FOR APPLYING BALDRIGE CRITERIA AND MICROSYSTEM CONCEPTS

Several tools and techniques are available for microsystems that wish to engage in self-assessment based on the Baldrige Award criteria and microsystem characteristics. First, the Microsystem Assessment Tool (MAT) can be used for a microsystem’s self-assessment.^25,26 The success characteristics emerged from our analysis of the coded transcripts; they reflect what people working in high-performing microsystems say about their work and how they do it. Consequently, they provide the framework for a microsystem-specific analysis of performance, and have been the basis of the MAT (shown in table 4) which is designed to help microsystems understand how they can improve their performance. The MAT addresses the nature of the interaction between the microsystem and the parent organisation, which is not specifically called out by Baldrige. MAT is designed to be used quickly and easily by microsystem members to evaluate their own frontline unit. Although it clearly does not have the depth of a comprehensive Baldrige assessment, it offers considerable insight into the functioning of a microsystem. Understanding how the success characteristics and the Baldrige criteria relate to each other can help in doing Baldrige self-assessment and in understanding challenges to high performance. The success characteristics, including principles and specific processes, could be helpful to a microsystem working to better understand the Baldrige criteria or improve performance in a specific area.

Second, we have developed a series of “toolkits” and “workbooks” to provide a path forward for assessing one’s microsystem. Workbooks are available for different types of clinical microsystem, including:

• Primary care practices

• Specialty practices

• Cystic fibrosis programmes

• Brain trauma programmes

• Inpatient care units

• Emergency departments

Each of these workbooks uses a standard approach to conduct a full assessment of a microsystem based on the “5 P” method, which includes assessments of the different “parts” of a clinical microsystem—purpose, patients, professionals and staff, processes and patterns of performance (outcomes, values, beliefs and practices). The workbooks, which are available electronically (http://www.clinicalmicrosystem.org), include a variety of methods and tools to evaluate each respective aspect of a microsystem. Tools available from Malcolm Baldrige include “are we making progress?” and “are we making progress as leaders?”. These questionnaires were designed to assess how an organisation is performing and to learn what can be improved. Based on the Baldrige criteria, the questionnaires help focus improvement and communication efforts on areas needing the most attention from the perspective of the organisation’s employees. These tools are available electronically (http://baldrige.nist.gov).

CONCLUSION

The convergence of microsystems thinking and Baldrige assessment offers a powerful new way to think about performance at the frontlines of healthcare. We undertook an analysis of high-performing microsystems using a Baldrige “perspective” to better understand the success characteristics we had previously described. The analysis described here demonstrates that both Baldrige criteria and microsystem success characteristics cover a wide range of areas crucial to high performance. A second reason for performing this analysis was to validate the microsystem success characteristics by comparing them to Baldrige criteria. We wanted to ensure that the microsystem success characteristics that were identified based on our field research, were consistent with a well-recognised “gold standard” method (the Baldrige criteria) for assessing the success of an organisation.

Microsystems that operate within the context of a larger organisation face many challenges. In the ideal world, organisational alignment would be clear and consistent at all levels; the transcripts reveal that even these outstanding performers do not live in such a world. While Baldrige assessment can make those gaps clear, organisational leadership must be committed to closing them. In the case of the microsystem, organisational leadership may not be accessible or amenable to the changes required to achieve alignment. Thus, a different tool may be needed, which recognises the context and suggests ways that the microsystem can work to improve relations with the parent organisation. The success characteristics explicitly consider this aspect of microsystem functioning.