Article Text
PDF
Abstract
Objective: To explore the potential for using seven prescribing indicators, individually and in combination, to measure prescribing quality for hospitalised psychiatric patients.
Design and setting: The dataset included full details of all psychotropic medication prescribed over a 24 hour period to 4192 inpatients in 49 British mental health services in 1998.
Results: Despite the large size of the dataset, for three of the indicators 20 services had fewer than 10 eligible patients. There was great variation between services in indicator scores. Correlations between standardised indicator scores and total score (which omitted the indicator concerned) were above 0.3 for all but one of the indicators. Cronbach’s alpha was 0.73 when this outlying indicator was removed.
Conclusions: There are no routinely collected prescribing data that allow for the quality of prescribing for psychiatric patients to be monitored. Six of the seven indicators measured during this census survey appear to reflect a common attribute of the services, and the analysis suggests that they might be combined to give an overall measure of service performance. There was, however, no relationship between performance on the seventh indicator and performance on the other six. This raises questions about case mix and service level factors that might influence indicator scores independent of prescriber decision making. The psychometric properties of prescribing indicators (occurrence rates, consistency over time) are unknown.
- prescribing
- quality of care
- psychiatry
- hospital patients
- performance monitoring
Statistics from Altmetric.com
Almost all patients admitted to hospital receive prescribed medication at some point during their stay and, for some, it is the only treatment intervention. Medicines are effective if used correctly, but can be ineffective or even toxic if they are not. Prescribing, dispensing, and administration errors are major clinical risks. This paper focuses on indicators of prescribing and their potential for use as a measure of the quality of prescribing decisions. We also consider whether such indicators are, in fact, a proxy measure of the overall quality of clinical care provided by a service.
An ideal indicator of the quality of prescribing practice of a doctor or group of doctors would have a number of qualities:
-
The prescribing behaviour that is measured by the indicator should have a close association with a good or bad outcome for the patient—that is, there should be strong research evidence that prescribing within the parameter leads to a good clinical outcome at minimal cost, particularly in terms of adverse effects for the patient, and/or that the converse is true for prescribing outside the parameter.
-
The indicator should be robust and stable so that the indicator score truly reflects differences that are due solely to prescriber decision making. There should not be many permissible exceptions to the “rule”. This is particularly true for exceptions that might cause the indicator score to be influenced by differences in case mix (such as those related to age, sex, severity of the disorder, comorbid conditions, or behavioural disturbance) or by systematic differences in the quality of the care environment—for example, differences in nurse staffing levels that might lead to prescribing as an alternative to intensive nursing care.
-
There should be consistency between indicator scores relating to the same prescriber or group of prescribers—that is, each should relate to a common factor that reflects a general difference between prescribers in knowledge of therapeutics.
-
The prescribing behaviour should occur sufficiently frequently to permit meaningful comparisons across individual prescribers or groups of prescribers.
-
The indicator score should be possible to measure, either from routinely collected prescribing data or by an ad hoc approach.
Although most of the work that has been done in developing and using prescribing indicators focuses on general medical services, there is growing interest in the use of prescribing indicators as measures of performance of mental health services.
The National Institute for Clinical Excellence (box 1) technology appraisal guidance on atypical antipsychotics1 suggests that the number of occasions when an atypical and typical antipsychotic are prescribed concurrently should be audited. The NHS National Service Framework for Mental Health2 proposes using the ratio of benzodiazepine to antidepressant prescriptions as an indicator of the quality of prescribing for depression. A recent UK Department of Health consultation document listed a set of potential performance indicators which included prescribing rates for benzodiazepines, antidepressants and atypical antipsychotics and the proportion of individuals for whom these classes of drugs are prescribed in the recommended dose range and for the correct length of time.3 Other proposed markers of prescribing quality are the proportion of prescriptions for carbamazepine and valproate for a therapeutic dose4 and the proportion of patients who are prescribed high dose antipsychotics or more than one antipsychotic concurrently.5
Box 1 National Institute for Clinical Excellence (NICE)
NICE was established in 1999 as a special health authority for England and Wales. Its explicit purpose is to provide authoritative, robust, and reliable guidance regarding healthcare interventions to clinicians, patients and the public.
Two main types of guidance are issued:
-
Health Technology Appraisals (HTAs) of new or controversial health technologies—for example, expensive new drugs and electroconvulsive therapy (ECT), respectively.
-
Clinical guidelines for the management of individual conditions—for example, schizophrenia, deliberate self-harm.
Prescribing indicators are attractive to those responsible for monitoring performance because much is known about the efficacy and effects—both wanted and unwanted—of psychotropic drugs, and there is a large and growing body of authoritative advice about what constitutes good practice under specified circumstances. This includes both positive guidance about what is likely to be beneficial and warnings about which drugs or combinations of drugs are likely to cause harm. Also, unlike psychological therapies or service level interventions where “fidelity to the model” is an issue, the presence of a written prescription provides an accurate record of the precise nature of the intervention, although not necessarily of whether the patient adhered to the prescribed regime.
In the UK most non-hospitalised patients with severe mental illness are seen at community based facilities. Here, prescriptions written by psychiatrists are usually dispensed through community pharmacies. The current system for recording these prescriptions does not allow for aggregation in a way that would enable the practice of individual psychiatrists, or even of small groups of psychiatrists, to be evaluated. Also, it is common practice for a psychiatrist to recommend a medication regime to the patient’s general practitioner who actually writes the prescription. This means that, at the present time, routine monitoring of prescribing would only be possible for inpatients.
In 1998 the Royal College of Psychiatrists’ Research Unit conducted a large national audit of antipsychotic prescribing against standards drawn from national guidance5 (box 2). The data provide an opportunity to explore the potential for using seven prescribing indicators, individually and in combination, to measure prescribing quality for hospitalised psychiatric patients. We hypothesised that mental health services that fared poorly on one indicator of “prescribing quality” would fare poorly on others, giving an overall measure of the quality of prescribing in that service.
Box 2 Royal College of Psychiatrists’ Research Unit’s national audit of antipsychotic prescribing
-
Forty nine UK mental health trusts chose to take part in the audit which had the express aims of quantifying the prevalence of high dose antipsychotic prescribing and measuring the quality of associated monitoring. The number of patients included from each trust was at the discretion of the trust staff who collected the data. For each patient, data were collected about age, sex, ethnicity, diagnosis, and all psychotropic drugs prescribed on a regular or as required basis. Prescribing data were obtained from prescription cards.
-
Data were submitted for 4192 patients who were in hospital on 21 July 1998.
-
20% of patients overall were prescribed a total dose of antipsychotic that was above recommended guidelines, and there was huge variation between services with 0–50% of patients in each service falling into this category. Demographic and clinical variables such as being young, male, having a diagnosis of schizophrenia, or being detained under the Mental Health Act accounted for only 18% of the variance between services.
-
Documentation and monitoring of high dose regimens was poor.
There are clear proposals to measure prescribing as part of a basket of performance indicators for English mental health services. The objectives of this study are to examine some of the implications and practicalities of doing this.
METHODS
Data set
Full details of how the data from which the indicator scores are derived were collected are given elsewhere.5 In brief, the dataset includes full details of all psychotropic medication prescribed over a 24 hour period to 4192 inpatients in 49 British mental health services (box 2).
Indicators
Seven potential indicators of prescribing quality can be derived from the dataset. Each has been given a short title for easy reference:
-
“High dose antipsychotics”: percentage of patients prescribed any antipsychotic whose total daily dose of antipsychotic drugs is above the maximum recommended by the British National Formulary (BNF).
-
“Antipsychotic polypharmacy”: percentage of patients prescribed any antipsychotic drug in whom more than one antipsychotic drug is being prescribed concurrently.
-
“Atypical polypharmacy”: percentage of patients prescribed any atypical antipsychotic in whom another antipsychotic drug is being concurrently prescribed.
-
“Multiple PRN”: proportion of all patients being prescribed three or more psychotropic drugs on an as required (PRN) basis.
-
“Subtherapeutic doses of mood stabilisers”: percentage of patients prescribed either valproate or carbamazepine for whom the dose prescribed is below the therapeutic level (at least 1 g/day or 600 mg/day, respectively).
-
“Hypnotic prescribing”: percentage of all patients prescribed a hypnotic drug, including benzodiazepines normally prescribed as hypnotics.
-
“Benzodiazepines:antidepressants”: ratio of total number of prescriptions for benzodiazepines (including all benzodiazepines normally prescribed as anxiolytics) to total number of prescriptions for antidepressants.
For all indicators, lower scores are desirable. For the first six indicators a score of 0% would be ideal.
Data analysis
Variation in “performance” was examined by plotting the distribution of scores for the 49 mental health services for the seven indicators. The interrelationship between the indicators was then tested. The aim was to determine whether (1) there is evidence of consistency to support the hypothesis that some or all of the indicators relate to some common attribute, and (2) there is any justification or benefit in combining indicators. Firstly, the relationships between raw scores on indicators were examined using Pearson correlation coefficients. For subsequent analyses, performance against each indicator was standardised by ranking mental health services for each indicator according to their level of adherence to the standard stated where 1 = best performing service and 49 = worst. Indicator-total correlations were calculated using these standardised indicator scores. Cronbach’s alpha was computed including all seven standardised indicator scores and then separately after the exclusion of each individual indicator.6
RESULTS
The number of patients who formed the denominator varied between indicators. Table 1 shows the total number, mean number per service, and range between services in the number of patients who were considered in the calculation of performance against each indicator. The number of eligible patients varied between services; for indicators 3 and 5, five and 20 services, respectively, had fewer than 10 eligible patients.
Numbers of patients eligible for inclusion in each indicator and performance against indicators
Overall performance and variation between services
Table 1 shows the mean level of adherence to each indicator and the extent of variation between the 49 services. The nature of the distribution of scores is also illustrated in fig 1 which shows the large variation in performance for each of the seven suggested indicators of prescribing quality.
Distribution of scores on the seven indicators across the 49 mental health services.
Relationship between indicators
The Pearson correlation coefficients between the raw scores of different prescribing indicators across all services are shown in table 2.
Interrelationship between the prescribing indicators (Pearson correlation coefficients)
Table 3 further examines the relationship between indicators using the standardised scores created by ranking services (from 1 to 49) on the basis of their performance on each indicator. The “indicator-total correlation” is the correlation between the individual indicator and a total score omitting that indicator. Thus, the first row of table 3 shows that the correlation between standardised scores on indicator 1 and the sum of standardised scores on indicators 2, 3, 4, 5, 6 and 7 is 0.46. Item-total correlations above 0.2 are usually considered to justify the inclusion of an item in a multi-item scale designed to measure a common attribute.6 All indicators except for hypnotic prescribing had indicator-total correlations above that threshold.
Indicator-total correlations and Cronbach’s alphas for standardised indicator scores
When all seven standardised indicators are included, the Cronbach’s alpha is 0.66. As shown in table 3, this increases to 0.73 when the indicator relating to hypnotic prescribing is omitted. A Cronbach’s alpha above 0.7 is generally considered to demonstrate acceptable homogeneity between items in a measurement scale.6
DISCUSSION
Six of the potential indicators reflect a categorical concept—that is, whether prescribing for individual patients is within or outside a stated parameter. The assumption is that prescribing within the parameter represents good practice and prescribing outside the parameter represents poor practice. Here, the indicator score is the percentage of patients whose prescriptions are considered poor practice. The seventh indicator (ratio of benzodiazepines to antidepressants) reflects prescribing practice for a group of patients and the indicator score is a ratio.
Perhaps the most striking finding is the extent of variation between services for all seven indicator scores. This is consistent with findings in primary care where prescribing analyses and cost data (PACT) are used to monitor the cost and quality of prescribing.7 Adherence to quality indicators derived from PACT has been shown to vary widely between practices.8
Evidence base for these indicators
Some research evidence is available which suggests that robust indicators relating to high dose antipsychotic drugs and antipsychotic polypharmacy (indicators 1, 2 and 3) might be of value. This is consistent with recommendations made in a number of evidence and consensus based guidelines.1,5 There is also evidence to support the use of indicator 5 because doses of carbamazepine and valproate above these thresholds have been shown to be associated with a response while doses below the thresholds have not.4
Although not strictly evidence based, there is a consensus that hypnotic drugs should only be used for a short period of time if sleep hygiene measures have failed.9 This provides some justification for indicator 6. Also, indicator 4 (multiple prescriptions of psychotropic drugs on a “PRN” basis) may be considered to be poor prescribing practice because it gives too much discretion to the psychiatric nurses who administer the drugs and who might have only limited knowledge of their effects.10
The recommendation in the English National Service Framework2 about using the ratio of benzodiazepines to antidepressants (indicator 7) is presumably based on the assumption that this indicates that a diagnosis of depression is being missed and that patients prescribed benzodiazepines may be more appropriately treated with antidepressants. This might not be the case for inpatients where many benzodiazepines are likely to be prescribed to control agitation (see below).
Is there consistency between indicator scores?
Some of the significant correlations between individual indicators (table 2) are expected because of an overlap between them. Thus, it is known that there is a close association between “high dose antipsychotics” (indicator 1) and “antipsychotic polypharmacy” (indicator 2).11 Also, antipsychotic drugs and benzodiazepines are often prescribed PRN so indicators 2 (“antipsychotic polypharmacy”) and 7 (“benzodiazepine:antidepressant” ratio) would be expected to correlate with indicator 4 (“multiple PRN”). However, some of the individual correlations cannot be so easily explained and suggest that the various indicators might reflect some common attribute of the services concerned. The indicator-total correlations support this and the high value of Cronbach’s alpha, after omission of the hypnotic prescribing indicator, suggest that the scores on the other six indicators might be combined.
What are these indicators measuring?
If these indicators are measuring some attribute of the services, the question remains—what is this? It is probably not simply the quality of the clinical knowledge and prescribing skills of the psychiatrists working on these wards. More likely, the “common attribute” is a complex set of interacting factors relating to case mix and the quality of ward environment and service organisation as well as the decision making skills of the prescriber.
For this dataset, simple case mix factors such as sex, diagnosis, and Mental Health Act status explain 26% and 40% of the variance between the 49 mental health services in indicator scores for high dose antipsychotics and antipsychotic polypharmacy, respectively.12
It is probable that case mix variables influence performance on other prescribing indicators. For example, benzodiazepines are a standard treatment for acute behavioural disturbance, so a service that admits primarily behaviourally disturbed psychotic patients and few patients with depression will fare poorly on this indicator.
There is a clear relationship between qualities of the ward environment—such as layout, staffing, and overcrowding—and levels of disturbance and aggression among the patient group. This is likely to influence the perceived need for prescribing of high doses of antipsychotics, the writing of PRN medication, and the prescription of hypnotics. Also, bed shortages and the resulting pressure for early discharge might lead to a rapid escalation of drug doses and polypharmacy.13
Is measurement feasible?
Table 1 indicates that, for all indicators, the number of eligible patients varies widely between services. In some cases the numbers were too small to be meaningful. This is despite the fact that our prescribing dataset is the largest ever collected for psychiatric inpatients in the UK and the prescribing indicators chosen cover common prescribing practice where there is at least a perceived evidence base. Also, each service will have many prescribers and the contribution of each to the overall service score is unknown. Using individual prescribers as the denominator would magnify the problems described above.
Key messages
-
There are no routinely collected prescribing data that allow for the quality of prescribing for psychiatric patients to be monitored. Such data have to be collected ad hoc and this is time consuming.
-
Several indicators of prescribing quality in psychiatry have been proposed.
-
Performance on indicator scores varies widely between services.
-
An association was found between relative performance on six of the seven indicators measured, and these could usefully be combined to give an overall measure of prescribing quality. There was, however, no relationship between performance on the seventh indicator and the other six.
-
Prescribing indicators need to be interpreted in the context of case mix and service level factors that may influence them independently of prescriber decision making. Those responsible for using prescribing indicators must be aware of the limitations of these measures.
In the absence of routinely collected national prescribing data for mental health services, indicator scores can only be measured by ad hoc data collection of the type used here. The collection of census data of the type used in this analysis is a demanding task, requiring significant cooperation by the services concerned. One way of securing this cooperation might be to foster the collection of the data as part of the local clinical audit and to create incentives for benchmarking between trusts.
It is not known how stable the indicator scores are over time within services because the data from which the scores are derived were collected in a single one-day census. It is essential that further work is undertaken to improve our understanding in this area before prescribing indicators are incorporated into routine assessment processes.
Conclusions
It seems likely that prescribing will be monitored using indicators of this type. If this is the case, those responsible for using prescribing indicator scores must understand how difficult they are to interpret. When differences between prescribers or groups of prescribers are found, a number of questions should be posed before asking them to review their practice. These questions relate to the psychometric properties of the indicators (occurrence rates, consistency over time) and case mix and service level factors that might influence indicator scores independent of prescriber decision making. Systematic research is required to identify and quantify the impact of factors other than the quality of decision making on prescribing indicators such as those described here.
Acknowledgments
Thanks are due to the 49 mental health services that participated in the national audit from which these data are drawn. The views expressed do not necessarily reflect those of the Royal College of Psychiatrists.
Footnotes
-
See editorial commentary, p 9
Linked Articles
- Quality lines
- Commentary