Article Text
PDF
Abstract
Background Guidance for appropriate utilisation of transthoracic echocardiograms (TTEs) can be incorporated into ordering prompts, potentially affecting the number of requests.
Methods We incorporated data from the 2011 Appropriate Use Criteria for Echocardiography, the 2010 National Institute for Clinical Excellence Guideline on Chronic Heart Failure, and American College of Cardiology Choosing Wisely list on TTE use for dyspnoea, oedema and valvular disease into electronic ordering systems at Durham Veterans Affairs Medical Center. Our primary outcome was TTE orders per month. Secondary outcomes included rates of outpatient TTE ordering per 100 visits and frequency of brain natriuretic peptide (BNP) ordering prior to TTE. Outcomes were measured for 20 months before and 12 months after the intervention.
Results The number of TTEs ordered did not decrease (338±32 TTEs/month prior vs 320±33 afterwards, p=0.12). Rates of outpatient TTE ordering decreased minimally post intervention (2.28 per 100 primary care/cardiology visits prior vs 1.99 afterwards, p<0.01). Effects on TTE ordering and ordering rate significantly interacted with time from intervention (p<0.02 for both), as the small initial effects waned after 6 months. The percentage of TTE orders with preceding BNP increased (36.5% prior vs 42.2% after for inpatients, p=0.01; 10.8% prior vs 14.5% after for outpatients, p<0.01).
Conclusions Ordering prompts for TTEs initially minimally reduced the number of TTEs ordered and increased BNP measurement at a single institution, but the effect on TTEs ordered was likely insignificant from a utilisation standpoint and decayed over time.
- Quality improvement
- Control charts, run charts
- Decision support, computerized
- Statistical process control
Statistics from Altmetric.com
- Quality improvement
- Control charts, run charts
- Decision support, computerized
- Statistical process control
Background
The American College of Cardiology included echocardiography for asymptomatic native valve disease within its Choosing Wisely selected measures, specifically noting providers should ‘not perform echocardiography as routine follow-up for mild, asymptomatic native valve disease in adult patients with no change in signs or symptoms’ as noted by the Appropriate Use Criteria (AUC) for echocardiography.1 ,2 However, there are additional transthoracic echocardiograms (TTEs) ordered for other indications that may be of limited clinical significance. Reviewing 1 month of TTEs at an academic medical centre, Matulevicius et al determined >20% of TTEs resulted in no change in clinical care, regardless of whether the patient had previously undergone TTE or the study was considered appropriate by AUC.2 ,3
Overall, the utilisation of TTEs among Medicare beneficiaries almost doubled between 2000 and 2008, and more than half of those who receive a TTE will have another within 3 years.4–6 Increasing TTE use has led to efforts to optimise TTE utilisation. The UK's National Health Service's National Institute for Clinical Excellence recommends first using serum natriuretic peptide levels in patients without a history of myocardial infarction but with suspected chronic congestive heart failure (CHF).7 Using this diagnostic approach, the negative predictive value against CHF for a normal natriuretic peptide (eg, serum B-type natriuretic peptide (BNP) <100 pg/mL) is around 90%, and therefore a TTE could be avoided in most patients with normal levels.7–9 Previous short-term educational quality improvement (QI) efforts with education and feedback focused on the AUC reduced the absolute number and inappropriate fraction of TTEs ordered by housestaff in an inpatient setting and a cardiology fellow clinic.10 ,11
At the Durham Veterans Affairs Medical Center (DVAMC), approximately 330 TTEs are ordered per month from inpatient and outpatient services. In a retrospective review, we found that 37% were ordered for evaluating possible CHF and 11% for valvular indications, with arrhythmias the only other indications comprising >5%. As part of a QI initiative, we incorporated information from the AUC for echocardiography and National Institute of Health and Care Excellence CHF guidelines into our electronic ordering system for TTEs for the indications of dyspnoea, oedema and murmur/valvular disease. As these indications were responsible for up to 50% of echoes ordered at our facility, we hypothesised that increasing the information available for these indications at the point of order would lead to a reduction in TTEs ordered.
Methods
Setting and patient population
DVAMC is a 151-bed tertiary-care hospital within VA Integrated Services Network Region 6 (VISN 6) serving as the primary VA hospital and primary care access facility for >54 000 unique patients and as a referral hospital for subspecialty care and tertiary care for VISN 6.
Intervention
Prior to the intervention, the electronic ordering system allowed for TTE ordering when the physician addressed three questions: (1) a free-text response to ‘What is the clinical question?’, (2) a check box indicating whether the patient underwent TTE within the preceding year, and (3) study indication (with structured answer options as radio buttons).
For our intervention, we altered the ordering prompt such that the third portion with indication check boxes was replaced with a question about whether the TTE was being ordered for clinical concerns of dyspnoea, oedema or murmur/valvular disease. Each of these indications was accessible with a radio button, which then presented information to the provider about when a TTE would be indicated for these indications. If one of these was chosen, additional information was provided (see figure 1). For the concerns of dyspnoea and oedema, the text reviewed the need to obtain a BNP level prior to ordering the TTE. Beneath this, text then noted that a serum BNP level that is normal for our facility (<100 pg/mL) has a 98% negative predictive value for underlying chronic CHF. In a following bulleted list, case scenarios in which a TTE would or would not be indicated based on AUC were presented. In the case of valvular disease, the text regarding BNP was omitted and scenarios based on the AUC in which a TTE would not be indicated were shown (figure 2).
Ordering prompt for transthoracic echoes ordered for clinical concerns of ‘oedema’.
Ordering prompt for transthoracic echoes ordered for clinical concerns of ‘murmur’ or ‘valvular disease’.
The prompt text was reviewed and developed in a Delphi process by cardiologists, hospitalists and general internists at DVAMC prior to implementation. The order system was changed at our facility in September 2013. No concurrent educational interventions for attendings or housestaff accompanied this ordering process change.
Outcome assessment
The primary outcome was the mean number of TTEs ordered per month. To account for increasing patient volume over time leading to increasing order volume, secondary outcomes included the rate of outpatient TTE ordering per 100 primary care and cardiology visits. In addition, we evaluated the percentage of TTE orders in which a BNP was ordered in the 30 days prior to the TTE request.
Subgroup analysis was performed using (1) only inpatient TTE orders and (2) only outpatient TTE orders. A second subgroup used only the months from the ‘preintervention’ phase that matched the same month of the calendar year in the postintervention group (ie, for every ‘March’ in the postintervention group, there was one and only one ‘March’ in the preintervention used in this subgroup analysis).
All outcomes were compared before and after the intervention. TTE orders and BNP orders and values from 1 January 2012 through 9 January 2014 were obtained using the electronic health record.
Statistical analysis
We performed negative binomial regression modelling for the monthly order count before and after the intervention, as Poisson models had evidence of overdispersion. To estimate the interaction between the effect of the intervention and the time from the intervention, we created a second variable to represent the time difference in months from the date of the intervention.
We compared the fraction of TTE orders that had a predetermined BNP level before and after the intervention using χ2 testing. We compared rates of outpatient TTE ordering per 100 primary care and clinic visits using negative binomial regression modelling. All analyses were considered significant at a p value <0.05. Statistical analyses were performed in R and confirmed with STATA V. 9.2 (College Station, Texas, USA).12
Statistical process control c-charts were created for order rates by month using QI Macros (KnowWare International, Denver, Colorado) within Microsoft Excel (Redmond, Washington). Upper and lower control limits were specified as ±three SDs from the mean, respectively.13 Special cause variation was detected using the Montgomery Supplementary rules.14 Charts were divided into preintervention and postintervention periods alongside the remainder of the analysis.
IRB statement
As this work was an evaluation of a systematic clinical change applicable to all patients, it was deemed exempt from review by the Durham Veterans Affairs Medical Center Institutional Review Board.
Results
From January 2012 through August 2013, the mean number of TTEs ordered monthly was 338±32. After the intervention in September 2013, the mean number of TTEs ordered monthly decreased 5% to 320±33, although this change was non-statistically significant over the full postintervention study period (p=0.13 for negative binomial regression, monthly ordering rates shown in figure 3).
Transthoracic echoes (TTEs) ordered, by month*. UCL=upper control limit, equal to +3 SDs from the mean. CL, centre line, or mean; LCL, lower control limit, equal to −3 SDs from the mean. *Fiscal year 2012 corresponds to October 2011 through September 2012, fiscal year 2013 corresponds to October 2012 through September 2013, and fiscal year 2014 corresponds to October 2013 through September 2014. The separating space in the graph indicates the time of the intervention.
We assessed the rates of TTE ordering to adjust for the increase in outpatient primary care and cardiology patients. Rates of outpatient TTE ordering decreased significantly by 12.5%, from 2.28/100 visits before the intervention to 1.99 afterwards (HR 0.87, 95% CI 0.80 to 0.95, p<0.01). Outpatient primary care and cardiology clinic visits at DVAMC increased by 10.7% from 102 059 in fiscal year 2012 to 112 955 in fiscal year 2014. Thus, in fiscal year 2014, the reduced rate means that >300 TTEs were avoided.
When TTE ordering was analysed relative to the time from the intervention, a significant interaction was found (p<0.02 for interaction of time and intervention). The reduction in ordering was highly significant immediately after the intervention, but ordering frequencies later returned to levels more closely approximating the levels prior to the intervention (see figure 3). We found nearly identical results when we examined postintervention changes in the rate of echocardiogram ordering per 100 outpatient clinic-visits, although this difference remained statistically significant over the full study period (see online supplementary appendix).
BNP measurement prior to ordering TTEs increased modestly after the intervention (21.8% of all patients referred for echocardiography prior to intervention vs 26.1% after, p<0.01). This was true for TTEs ordered from the inpatient and outpatient settings (36.5% prior vs 42.2% after in the inpatient setting, p=0.01; 10.8% prior vs 14.5% after in the outpatient setting, p<0.01) and did not wane over time. The fraction of BNPs ordered prior to a TTE that was normal did not differ by study period (22.3% preintervention vs 23.5%, p=0.45).
Discussion
Incorporating historical data and trends into decision support structures at the point of care to aid providers in real-time decisions has the potential to increase the appropriate utilisation of limited resources.15 ,16 In this single-centre QI project, we added information from published guidelines on TTE ordering into our ordering system. To our knowledge, this is the first study focusing on improving TTE ordering simultaneously within the outpatient and inpatient settings. Overall, we found an initial small reduction in TTE orders with inclusion of these data, which returned near the previous baseline of use after a few months of implementation. Additionally, we noted a significant increase in the proportion of TTEs ordered that had a preceding BNP checked.
While previous studies have reduced TTE orders over short periods in focused ward and clinic settings, our study indicates that the effect of these interventions may wane over time.10 ,11 The rate of TTE orders initially decreased immediately after our intervention, but these effects began decaying approximately 6 months afterwards. Other studies have shown this phenomenon of reduced TTE ordering immediately after an education intervention and then regression back to preintervention levels after discontinuation of the intervention.10 ,17 Unlike these previous interventions, however, ours required no additional provider time as it involved no educational outreach associated with presentations or conferences nor periodic audit and feedback methods. Short of providing periodic intervention ‘boosters’, durable responses may require a duration of intervention beyond a certain threshold in order to be sustainably successful. Other QI methods that may be more effective in optimising utilisation may require providers to electronically engage with appropriate use standards at the point of ordering or review previous TTE results and document potential therapeutic changes that would be based on different TTE findings.
An understanding that positive QI interventions may decline in effect over time highlights the importance of ongoing monitoring with statistical process control charts. This ongoing prospective data collection in real time, rather than a retrospective analysis, should help identify an early decaying effect. This early warning would allow opportunities for retraining or could indicate the need for a change in process to prevent further decay, as opposed to evaluation of data at the ‘end’ of the QI intervention when the decay may have already happened.
Although our results show that efforts focused on a handful of TTE indications can have an initial impact, two specific possibilities for the waning effect deserve comment. One potential reason for the increase in orders over time may be ‘prompt fatigue’. Over time, providers may no longer continue to match the clinical scenario with the prompt scenarios and may be more likely to order the study without fully absorbing prompt information. Importantly, we did not prevent any provider from ordering a study, even if it clearly would not have been indicated under the AUC or prompt scenarios. Alternatively, the increase in TTE ordering in more recent months may be due to the continued increase in the veteran population cared for at our institution, as previous analysis has shown much of the change in TTE ordering with the VA system over a several-year period was due to an increased number of patients rather than per-patient use.18 In fact, our small reduction in TTE ordering may have been larger if the veteran population for which DVAMC is responsible and the number of visits associated with that population had remained flat. As geographical and temporal patterns of veteran healthcare benefit use change, continued study of resource utilisation within the Veterans Healthcare Administration will be important to designate optimal resources to facilities.
The rate of BNP ordering prior to TTE ordering increased after the intervention. Using a sequential diagnostic testing strategy incorporating natriuretic peptides prior to echocardiography has been estimated to be cost-effective in the British and Swedish populations without a history of prior myocardial infarction.8 ,19 Although we do not know the number of patients evaluated in our outpatient setting with suspected CHF without a prior diagnosis of myocardial infarction, the language within our prompt targets this population specifically. As such, we feel this increase in BNP ordering within the outpatient setting is directly related to the ordering change. Within our inpatient setting, most patients are cared for by housestaff, and most BNP levels may be used to establish or compare with a patient's baseline level in those known to have a previous history of CHF. However, there may be diagnostic uncertainty regarding CHF symptoms in some patients, leading to the increased use of BNP patients in the admitted population.
This study was a non-random before/after evaluation of a QI intervention. This design is most pragmatic for QI work but creates the potential for confounders to affect the results. A potential confounder is that the physicians changed over time. While there could be temporal trends that affected housestaff ordering across a year of training, many of the months with the highest ordering levels immediately preceded the intervention and there was no concurrent educational intervention that would have changed practices of which we are aware. Furthermore, most outpatient TTEs are ordered from settings where housestaff are the minority of providers. With the change in our ordering system we were unable to track specific indications for TTE, preventing us from attributing the reduction in TTE orders or increase in BNP orders to specific indications or assessing the appropriateness for these studies. We were unable to determine whether appropriate TTEs were not performed after the intervention since the appropriateness criteria suggest when an echocardiogram would be acceptable as opposed to necessary, making a retrospective review for the appropriateness of not ordering the test impossible. As even appropriate TTEs may not be clinically meaningful, future studies may be improved by focusing on other outcomes, such as CHF hospitalisations, as a better balancing measure.3 Finally, although the fraction of patients for whom a TTE was ordered with a normal BNP did not change with the intervention, some of these patients may have had other reasons to undergo TTE. A better understanding of these factors could allow continued targeted interventions for other ordering indications.
In conclusion, a revision in a TTE ordering prompt throughout a VA hospital and its associated clinics led to a brief but likely unmeaningful reduction in ordering frequency and a significant increase in the frequency of TTEs linked to a prior BNP test. As the number of TTE orders increased over time after the intervention, more sophisticated QI strategies may be necessary to optimise utilisation of TTEs.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Footnotes
Twitter Follow Joel Boggan at @joelboggan
Contributors JCB, RDS, MD and DLS: substantial contributions to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolve. JCB, RDS, MD and DLS: drafting the work or revising it critically for important intellectual content
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.