Article Text
PDF
Abstract
Introduction The use of patient reported outcome measures (PROMs) for four elective operations is mandatory in the English NHS from April 2009. In view of some scepticism by some clinicians as to the validity of PROMs, our aim was to explore the relationship between patients' and clinicians' reports of health status before and after knee arthroplasty.
Methods A secondary analysis of linked data from the Knee Arthroplasty Trial (patients' reports using the Oxford Knee Score) and the Tayside Arthroplasty Audit (clinicians' reports using the American Knee Society Score—Knee Score and Functional Score) was carried out. Correlations of scores were obtained for 284 patients before and 226 patients after surgery.
Results There was a moderately strong correlation between patients' and clinicians' views 1 year after surgery: Oxford Knee Score (OKS) versus American Knee Society Scores (AKSS) Knee Score r=−0.64; OKS versus AKSS Functional Score r=−0.44. Before surgery, the correlation between the OKS and the AKSS Functional Score was also moderate (r=−0.55) but was weak with the Knee Score (r=−0.23). There was no systematic direction to the differences between patients' and clinicians' assessments; patients were just as likely to report better health than their clinician as to report worse health.
Discussion Patients' postoperative assessments following knee arthroplasty, as regards their symptoms and disability, are practical to collect and can make a meaningful and useful contribution in routine use. In view of the advantages of collecting data on symptoms and disability directly from patients—lower cost, higher response rates, avoidance of systematic biases—confirmation of a moderately strong association with clinicians' views offers further reassurance for the routine use of PROMs, at least with knee arthroplasty.
- Outcome assessment (healthcare)
- arthroplasty
- replacement
- knee
- questionnaires
Statistics from Altmetric.com
Introduction
There is increasing interest in patients' reports of the outcome of surgery to supplement or even replace clinicians' reports. The Department of Health in England requires all providers of elective surgery for NHS patients to use patient reported outcome measures (PROMs) from April 2009 for four common procedures.1 While generally welcomed by clinicians, some are concerned as to the validity and reliability of patients' reports and are uncertain about the relationship between patients' and clinicians' reports.
There is evidence of a moderate degree of association between patients' and clinicians' reports of health gain from surgery when the same dimensions of health are being assessed.2 Thus, when both report on disability, there is a moderately strong correlation (0.5–0.6), whereas the association between patients' assessments of disability and surgeons' assessment of impairment is, not surprisingly, poor (correlation 0.3).
Rigorous evidence about the degree of association between patients' and clinicians' assessments of the four operations for which the use of PROMs is mandatory in England from 2009 is limited. For example, there have been only two such studies of knee-replacement surgery using the Oxford Knee Score (OKS), the instrument that has been chosen by the English Department of Health to obtain patients' views. The first study, conducted by the developers in the 1990s, compared the OKS (a measure of symptoms and disability) with clinicians' views obtained using American Knee Society Scores (AKSS).3 In 117 preoperative patients, moderate associations were found with the AKSS Knee Score (symptoms and impairment) and the Functional Score (disability) (correlations −0.47 and −0.54 respectively). Recently, a second single centre study of 175 preoperative patients has reported similar correlations of −0.42 and −0.58, though stronger correlations were obtained in a different group of 312 patients 2 years after surgery (−0.64 and −0.65).4
Given the level of concern among some clinicians about the exclusive use of PROMs for assessing the outcome of knee arthroplasty and the limited evidence available, our aim was to determine the association between the OKS and clinician reported outcomes in a large sample of patients, drawn from several hospitals and assessed both preoperatively and nearer the postoperative time point that PROMs will be routinely used from April 2009.
Methods
This was an opportunistic study based on secondary analysis of linked data from two existing databases. Data were available on 333 patients who underwent total knee replacement in Tayside, Scotland between July 1999 and November 2002 as part of a multicentre randomised trial, the Knee Arthroplasty Trial (KAT).5 Information was collected from patients before and 1 year after surgery about their symptoms and disability using the OKS. This is a 12-item questionnaire that generates an index score of 0–48, with higher scores indicating greater symptom severity and disability (at the time of the study, the score used was 12–60). The development study for this questionnaire cites good reproducibility (r=0.92 between the total scores for a test–retest sample, coefficient of reliability=6.45 using the Bland–Altman method), good construct validity (moderate correlation with AKSS, significant agreement with parts of the SF-36 and the HAQ) and sensitivity to change greater than for the SF-36 (effect size=2.19).
Data were also available on these patients from the Tayside Arthroplasty Audit (TAA). This included surgeons' reports of patients' symptoms, impairment (clinical examination findings) and disability measured using the AKSS before and 1 year after surgery.6 The AKSS generates a Knee Score (0–100), based on symptoms and impairment, and a Function Score (0–100) calculated from questions on disability. Higher scores indicate lesser symptom severity, impairment or disability.
Data were also obtained on patient's age, sex, hospital and date of operation, and postoperative assessment date. Dates of preoperative assessments were not available, but all took place no more than a few weeks before surgery, though the KAT and TAA data were not necessarily collected at the same time. Of the 333 patients, 49 (14.7%) with some missing preoperative data were excluded from the preoperative analysis. Of the remaining 284, 34 (12.0%) were excluded from the postoperative analysis either because of missing data or because their KAT and TAA assessments were more than 3 months apart. Of the 250 with complete data, 226 (90.4%) had both their postoperative assessments within a 3-month period, of which 172 (68.8%) were within 1 month.
Data from the two sources were supplied in an anonymised format and then linked using a unique identifier, and so ethical approval was not sought. Analyses were undertaken using SPSS version 16 (SPSS, Chicago, Illinois) and Stata version 10 (Stata, College Station, Texas). Pearson correlation coefficients were used to describe the association between OKS and AKSS scores. Pre- and postoperative OKS were divided into quartiles, as there is no recognised or validated system for categorising the scores.7 Mean AKSS Knee and Function Scores were then compared across the four OKS quartiles using analysis of variance. For the association between two similar items (questions) about function relating to walking and climbing stairs, the Spearman rank correlation was used.
Results
Patients for whom data were available preoperatively (n=284) and postoperatively (n=226) were similar to all patients undergoing surgery (n=333) as regards mean age, sex ratio and hospital of treatment. For postoperative assessments, there was no significant difference in correlations by the time interval between the clinical (AKSS) and patient-reported (OKS) assessments. Results are therefore presented for all patients for whom the two assessments were made within a 3-month period (n=226).
The mean (SD) preoperative scores for the OKS, AKSS Knee and AKSS Function were 40.9 (7.5), 27.6 (14.2) and 49.7 (16.9). Corresponding postoperative scores were 24.6 (9.7), 84.5 (15.7) and 69.5 (22.2).
The association between the OKS and AKSS Knee Score was moderately strong after surgery (table 1). In contrast, there was only a poor correlation before surgery (−0.23; p<0.01). The strength of association between the OKS and AKSS Function Score was moderate both before and after surgery. Scatter plots (not shown) revealed that discordant assessments were evenly distributed—patients who viewed their health as poor were equally likely to be assessed by clinicians as good or poor.
Association between Oxford Knee Score (OKS) and American Knee Society Scores (AKSS) preoperatively and postoperatively (Pearson correlation coefficients (95% CI))
The relationship between OKS and AKSS scores was investigated further by considering mean AKSS scores for quartiles of OKS scores (table 2). There was a statistically significant difference in means (p<0.01) across the quartiles for both knee and function score both pre- and postoperatively (figures 1, 2).
American Knee Society Scores (AKSS) Knee and Function scores (mean, 95% CIs) for Oxford Knee Score quartiles
Relationship between OKS quartile and AKSS Function Score (mean and 95% confidence interval) pre-operatively and post-operatively.
Relationship between OKS quartile and AKSS Knee Score (mean and 95% confidence interval) pre-operatively and post-operatively.
For the only two items that appeared in both instruments, questions about walking and stair climbing, associations though statistically significant (p<0.01) were only moderate both before and after surgery (table 3).
Association between corresponding questions in Oxford Knee Score and American Knee Society Scores pre- and postoperatively (Spearman correlation coefficients (95% CI))
Discussion
There was a moderately strong correlation between patients' and clinicians' views 1 year after surgery. This was also true before surgery when OKS and AKSS function score were compared. In contrast, there was only a poor correlation before surgery between OKS and AKSS knee score. There was no systematic direction to the differences between patients' and clinicians' assessments; patients were just as likely to report better health than their clinician as to report worse health.
The strength of this study is that it was based on a large, representative sample of patients undergoing knee arthroplasty in three centres. The only potential limitation was that postoperative data collection from patients and from clinicians took place at different times (24% were 1–3 months apart). However, analyses by length of time difference found no significant impact.
Our results are mostly consistent with those previously reported. Before surgery, the moderate correlation found between the OKS and the AKSS Functional Score (−0.55) was the same as the two smaller, previous studies.3 ,4 However, we found only a poor correlation (−0.23) with the AKSS Knee Score compared with previous reports of −0.42 and −0.47. The reason for this is unclear, given that the same clinicians were involved in recording AKSS Scores both before and after surgery.
The implications of our results are that patients' postoperative assessments of knee arthroplasty, as regards their symptoms and disability, are practical to collect and can make a meaningful and useful contribution in routine use. Given the difference both in dimensions included and in the perspective of patients and clinicians, a stronger correlation between the OKS and AKSS Scores would not be expected. Each party's view is equally valid; notions of ‘gold standards’ are not appropriate. Instead, the two views should be considered as complementary. In view of the advantages of collecting data on symptoms and disability directly from patients—lower cost, higher response rates, avoidance of systematic biases—confirmation of a moderately strong association with clinicians' views offers further reassurance for the routine use of PROMs, at least with knee arthroplasty.
Acknowledgments
We thank the Tayside Arthroplasty Audit Group and the Knee Arthroplasty Trial Steering Group, for allowing us to use their data, and L Johnston, K McCormack and D Ridley, for extracting and advising on the data.
Footnotes
Funding EB was funded by NHS Scotland. The Knee Arthroplasty Trial was funded by the NIHR Health Technology Assessment Programme, and the Tayside Arthroplasty Audit is funded by NHS Tayside and the University of Dundee.
Competing interests NB has advised the Department of Health on the use of PROMs in surgery, and the OKS in particular, in the NHS. DM was one of the original developers of the OKS.
Provenance and peer review Not commissioned; externally peer reviewed.