MEASURES

Patients' views have different dimensions (table 1). This papers focuses on patients' views of health care—specifically on their preferences, evaluations, and reports.

Preferences

Preferences are ideas about what should occur in healthcare systems.² Related concepts are expectations, perceived needs, desires, wants, requests and priorities. Expectations have two distinct meanings: beliefs about what should occur or what people want of care (“normative expectations”), and beliefs about what will actually happen, irrespective of whether this is wanted (“predicted expectations”).² The term “preferences”, which has its origins in cognitive psychology and economics, is most often used to refer to individual patients' views about their clinical treatment. The term “priorities” is more often used to describe preferences for healthcare services in a population of patients or citizens.³ Qualitative as well as quantitative methods can be used to study preferences.

Qualitative research methods such as individual interviews and focus groups can be used to elicit preferences. These methods often use open ended approaches such as topic lists rather than structured questionnaires. It is often difficult for patients to decide what is important in general terms, given the limited experience of any one individual. Focus groups generate interaction among participants which may lead to shared views that transcend individual experiences. It may be helpful to present realistic but hypothetical situations as a trigger for discussions. Facilitation skills may be needed to test whether the group views are well considered and stable.

Quantitative methods for eliciting preferences include surveys and consensus methods such as the Delphi and nominal group techniques. Different types of data can be collected including scale responses that range from “not important” to “extremely important”; rankings—for example, preferences expressed in paired comparisons of alternatives; choice of alternatives—for example, a vote for the most desirable alternatives. Individuals can be asked to rate, rank, or vote for different care providers (GP or hospital) or attributes of care providers—for example, short waiting list, adequate information delivery. In a study of patient priorities in different countries high correlations were found between different methods of rating, ranking of, and voting for aspects of general practice care.⁴ A study of different methods for eliciting treatment preferences, however, found significant differences.⁵ A range of methods has been developed to collect preference data such as the expectancy value model, multi-attribute utility models, and conjoint analysis models (discrete choice experiments).⁶

A number of methodological issues related to the use of methods for priority setting in health care have been described.³ A first problem is how “options” are generated: limiting the choices will limit the preference frame. Patients should contribute to the development of a preference framework but they usually lack the expertise to generate a model completely on their own. Decisions in prioritisation issues in healthcare systems inevitably involve a wide array of factors, so methods have to be able to incorporate multidimensional influences. The most realistic methods involve presenting constrained choices where trade offs have to be made between different attributes or alternatives. It is important to be explicit about the methods used for the aggregation of individual preferences because different procedures will lead to different results. Researchers should understand that the choice of methods will influence results and that the assessment processes are at least transparent if methods have the potential to over-represent the views of some population sectors over others. An example of the use of patient preferences is given in box 1.

Evaluations

Patient evaluations are “health care recipients' reactions to salient aspects of the context, process, and result of their service experience”.⁸ Related concepts are “satisfaction”, “unmet needs”, “judgements”, “complaints”, and “comments”. The term “evaluation” suggests a cognitive process in which specific aspects of care are assessed, while “satisfaction” refers to an emotional response to the whole experience in health care. The term “patient satisfaction” is probably most often used in the literature.

Many studies have used written questionnaires that comprise structured questions with some sort of rating scale. The overall satisfaction with the healthcare experience is usually very high and this often masks less positive evaluations when aspects that are more specific are explored. A literature review showed that questionnaires that asked for evaluations in terms of “satisfaction/dissatisfaction” showed less discrimination than questionnaires that used terms such as “good/bad” or “agree/disagree” with very concrete aspects of care.⁹ Some questionnaires measure both preferences and experiences and derive evaluations from these two factors by calculating difference or ratio scores.¹⁰ There is some evidence that patients distinguish between the two concepts,¹¹ but there is no validated framework for deriving evaluations from preferences and experiences.¹²

Patients have evaluative responses to experiences in health care which are not necessarily translated into satisfaction,¹³ and qualitative methods can be used to examine these in more depth. An example is shown in box 2. Qualitative approaches are particularly useful for exploring patients' views in areas that have not been fully elaborated. Thorough data analysis of qualitative material is time consuming. Pragmatic approaches such as logging key themes without undertaking full transcription analyses may be used but, as far as we are aware, the reliability and validity of such approaches have not been assessed.

Reports

Patient reports represent objective observations of organisation or process of care by patients, regardless of their preferences or evaluations.¹⁵ Patients' experiences and their perceptions of professional performance are similar concepts. Patients can, for instance, register how long they had to wait in the waiting room, irrespective of whether this was too long or not. Although reports reflect patients' observations, they do not necessarily imply a patient's perspective on the quality of care. Nevertheless, patient reports can be used for quality improvement. In some situations patients' reports are the most accurate observation method if, for instance, the data are required about a patient's pathway through different healthcare institutions.

Validity of instruments

Instruments for assessment of patient views of health care should be validated to ensure that the tools measure what they are supposed to measure. A review of 195 studies of patient satisfaction published in 1994 showed that only 89 (46%) reported some validity or reliability data and only 11 (6% of 181 quantitative studies) reported content validity and criterion validity or construct validity and reliability.¹⁶

Ideally, the instrument should be compared with a gold standard or a criterion measure (an instrument with established validity). For instance, patient reports on the care received can be compared with the medical records or clinicians' reports on the care delivered (box 3). This approach is comparable with the validation of a diagnostic test. A criterion measure for preferences or evaluations is not often available, however, so other approaches are needed.

The validity of most instruments for patients' views should be based on conceptual frameworks that describe a specific domain (the relevant aspects of health care) and it is preferable if patients have been consulted regarding the selection and description of the relevant aspects. Qualitative studies are particularly suitable for that purpose. For instance, the Europep instrument for patients' evaluations of general practice care covers medical care, interpersonal relationships, information and support, and organisation of care. The aspects were selected on the basis of literature studies and qualitative and quantitative studies (box 4).

Sometimes it is possible to verify whether patients' views are associated with other factors or whether the measure meets criteria set by the theory from which the measure has been derived (construct validity). For instance, most patient satisfaction studies have shown that older patients have more positive evaluations of health care than younger patients. It can therefore be predicted that a new measure for patient satisfaction will show similar associations.

Psychometrics

Quantitative instruments should have adequate psychometric features.¹⁸ High item response rates will indicate the presence of questions that are more likely to be relevant and understandable. Some instruments, however, are intended to identify rare events such as medical errors (complaint procedures) or side effects of medications (surveys among drug users). High item response rates are not relevant in such cases. Instruments designed to measure aspects of quality should also show good variation across patients (discrimination) and variation between measurements at different points in time (responsiveness). If indicators are supposed to be clustered within dimensions, validity is supported by proven unidimensionality and high internal consistency. However, not all instruments assume clustering between indicators because indicators may not be seen as repeated measurements. This will be the case for many instruments that measure reports of concrete aspects of care. Ideally, instruments will also show good test-retest reliability.

The most often used reliability coefficients refer to the internal consistency of items within a dimension per patient—for example, Cronbach's alpha. In the context of quality improvement, however, aggregated scores per care provider are often needed—that is, aggregation over many individuals. These figures are based on a number of indicators and a number of patients or events. An example is the percentage of patients with positive evaluations of the accessibility of care in a specific hospital based on a survey of 100 patients who answered 10 different questions on accessibility. Generalisability analyses can be used to calculate reliability coefficients for the aggregated scores.¹⁹ It appears that an increase in the number of patients often has more influence on the reliability of the aggregated scores than an increase in the number of indicators.

Sampling

The inclusion criteria for the study population determine the generalisability or external validity of a study or audit. For instance, measurements among patients who attend a clinician are not generalisable to the general patient population registered at a practice or a population of Internet users who visit a site on a health problem. Qualitative studies use theoretical sampling to achieve a specific sample which may be heterogeneous or homogenous, depending on the overall purpose of the work. Quantitative studies use many different forms of sampling methods to achieve a representative sample (random, stratified, etc). It is important to achieve high response rates and low dropouts in order to avoid selection bias (except if this was sought). Non-responders are more likely to be represented by those who are ill, less satisfied with care provided, and less frequent users of health care than responders.^20,21 Surveys of interview methods need to consider the impact in case these groups are excluded or drop out.

Response rates in surveys among patients vary considerably. A literature review reported a mean of 60% in response rates and a standard deviation of 21%.²² Many factors may influence the response rate of a survey, such as the motivation of the clinician to recruit patients, the attractiveness of the lay out of a questionnaire, the method of administering the questionnaire to patients, the use of monetary incentives, and possibly the use of information technology for administering questionnaires. Insight into which factors are most relevant is limited. A comparison of handing out questionnaires to visitors to the general practice and mailing questionnaires to patients at home gave response rates of 72% and 63%, respectively; the content of the answers of both sample populations was, however, largely similar.²³ A randomised trial showed that written reminders could improve the response rates unless the rate was already above 80%.²⁴

USE OF PATIENTS' VIEWS FOR QUALITY IMPROVEMENT

Table 2 outlines the potential use of patients' views in healthcare delivery and quality improvement.²⁵ One approach focuses on those who want to make choices about their utilisation of healthcare services. Health education materials may include information on patients' views based, for instance, on qualitative research of their experiences in health care. Public reports on the performance of different care providers may include information about patients' evaluations of care.²⁶ A comparison with other care providers requires adequate adjustment for case mix, which is difficult because insight into predictors of patient evaluations of care (and most other indicators) is limited.²⁶

Another approach focuses on patients in episodes of care using, for instance, shared decision making strategies or patient-held records. Identification of patient preferences is part of most of these approaches. For instance, shared decision making implies that the care provider gives information on relevant options, assesses patient preferences regarding these options, and takes a decision with or checks approval of the patient.²⁷

A third approach provides different types of patient feedback on healthcare received derived from surveys, patient groups, or complaint procedures. These views can be used for continuing education and service improvements. Patient views can be compared with ethical or clinical guidelines for good practice but, in many cases, such standards are difficult to define. An exception is a lawsuit where an ethical or legal assessment is explicitly sought. Comparison with other care providers can help to prioritise issues that need attention.

A fourth approach focuses on the involvement of patients and the public in the design and planning of healthcare systems. This requires information on patients' views such as studies of patient priorities or the assessment of local needs for health care. Patient organisations express the views of patients who coalesce around issues or conditions and these may differ from the aggregated views of individuals in wider populations. In these situations, patients' views are only one of a number of inputs into a wider policy making process.

RELEVANT OUTCOMES

Methods to identify and use patients' views for the improvement of health care can be seen as a technology which should be evaluated in terms of effectiveness and efficiency. The choice of relevant outcomes for the use of patients' views for quality improvement requires further attention. It appears logical to derive outcome measures for the evaluation from the underlying objectives of this effort (table 3).

It is an ethical and legal rule that patients should be informed and involved in their health care, at least to minimal standards. Many patients wish to be involved in the decision processes, at least to some extent.²⁸ In line with this aim, the process of involvement rather than its outcome is crucial and so it is the ethical principles and patient preferences that define the criteria for effectiveness. For instance, shared decision making can be evaluated in terms of information delivered on treatment options, checking of understanding and preferences, and making a shared decision.²⁹

Patient involvement may also result in better processes and outcomes of care. It could, for instance, make clinicians more responsive to patient preferences, contribute to a better implementation of clinical guidelines, and result in better adherence to treatment, health status and satisfaction with care. Patients can be seen as co-producers of health care because their decisions and behaviour influence healthcare provision and its outcomes. Outcome measures should reflect the effects on process or outcomes of care that are expected.

Integration of patients' views may be driven by political and strategic motivations such as protection of a position in a competitive healthcare market, the wish to have democratic control in the healthcare organisation, or the need to do something for underserved populations. Such aims may be difficult to assess, but measurable outcome measures can be found in some cases—for instance, position on the healthcare market can be evaluated in terms of attendance rates and turnover of patients.

Finally, evaluations should consider possible negative consequences such as unrealistic patient expectations of what health care can deliver; defensive behaviour of care providers, resulting in higher numbers of unnecessary clinical procedures; undermining of professional morale; and increased costs. Such consequences are not imaginary. A randomised trial on low back pain showed that 80% would have chosen radiography if available, but that patients who received radiography often had more pain at 3 months than the control group and were nevertheless more satisfied with the care provided.³⁰

Not only should the effects of specific methods be studied, but also their actual uptake in health care. Clinicians and patients may lack competence or skills to use specific instruments or have negative attitudes regarding specific approaches. Organisational structures may limit the application of specific methods. Such barriers need to be identified and addressed by means of targeted strategies which should be evaluated in terms of success of uptake of the methods.

CONCLUSIONS

A range of approaches is available to integrate patients' views into healthcare delivery systems and their improvement. The methods to measure and use patients' views should be studied in the context of their intended application. Quantitative as well as qualitative approaches can be used to measure patients' views, and the validity and reliability of the methods should be examined. The effectiveness and efficiency of the methods should be studied in terms of their consequences for process and outcomes of health care. Increased patient participation in health care can be seen as desirable in itself, but this should not inhibit evaluation of the methods used to achieve this aim.