Methodologic Standards for the Development of Clinical Decision Rules in Emergency Medicine☆,☆☆,★,★★
Section snippets
INTRODUCTION
Reports of clinical decision rules are becoming increasingly common throughout the medical literature and particularly within emergency medicine journals. Clinical decision rules (prediction rules) are designed to help physicians with diagnostic and therapeutic decisions at the bedside. We define a clinical decision rule as a decisionmaking tool that is derived from original research (as opposed to a consensus-based clinical practice guideline) and incorporates 3 or more variables from the
1. IS THERE A NEED FOR THE DECISION RULE?
Clinicians should ask themselves whether there really appears to be a need for a particular decision rule, or whether the rule appears only to represent the analysis of a convenient set of data. Is there a demonstrated inefficiency or variation in current medical practice, and does there appear to be the potential for improved efficiency through guidelines or a decision rule?
2. WAS THE RULE DERIVED ACCORDING TO METHODOLOGIC STANDARDS?
Research methodology standards for the derivation of a clinical decision rule were first reviewed in 1985 in a landmark paper by Wasson et al.9 Feinstein,10 one of the fathers of clinical epidemiology, later added to the literature of evidence-based patient assessment in his book Clinimetrics . More recently, our research group at the University of Ottawa Clinical Epidemiology Unit assessed clinical decision rule articles in 4 major medical journals and proposed modifications to Wasson et al’s
3. HAS THE RULE BEEN PROSPECTIVELY VALIDATED AND REFINED?
Unfortunately, many clinical decision rules are not prospectively assessed to determine, in a new patient population, their accuracy, reproducibility, acceptance to clinicians, or potential effect on practice. This validation process is very important because many statistically derived rules or guidelines fail to perform well when tested in a new population.53, 54, 55 The reason for this poor performance may be statistical—overfitting or instability in the original derived model56—or may be
4. HAS THE RULE BEEN SUCCESSFULLY IMPLEMENTED INTO CLINICAL PRACTICE?
Very few clinical decision rules have been implemented and shown to alter clinical practice in what has been termed “the next painful step” for evaluating decision aids.69 A decision rule that has been shown to be valid and reliable still faces significant barriers to implementation in terms of both patient and physician acceptance.48, 49, 70 Emergency physicians are concerned about the medicolegal consequences of missing a diagnosis. Patients must place their trust in a physician whom they
5. WOULD USE OF THE RULE BE COST-EFFECTIVE?
If an implementation trial does, in fact, show that the decision rule alters clinical behavior, then a formal economic evaluation conducted by a health economist might be conducted.74, 75 The objective of such a study would be to clearly demonstrate the health care savings that might be associated with widespread use of the decision rule. Economic assessment is concerned with choosing between alternative uses of resources. Resources are limited and choices must be made.
Three basic concepts are
6. HOW WILL THE RULE BE DISSEMINATED AND IMPLEMENTED?
For a clinical guideline to have widespread effect on health care delivery, there must be an active plan for dissemination and implementation. We are well aware that the simple passive diffusion of original study results (through publication in medical journals or presentation at scientific meetings) is unlikely to significantly alter clinical practice in emergency medicine or in any other discipline.76, 77 Strategies to ensure the dissemination and implementation of clinical research are
References (88)
- et al.
A study to develop clinical decision rules for the use of radiography in acute ankle injuries
Ann Emerg Med
(1992) - et al.
Variation in ED use of computed tomography for patients with minor head injury
Ann Emerg Med
(1997) Predictors of intracranial injury in patients with mild head trauma
Ann Emerg Med
(1995)- et al.
Interobserver agreement in the examination of acute ankle injury patients
Am J Emerg Med
(1992) - et al.
Stratification by stepwise regression, correspondence analysis and recursive partition: A comparison of three methods of analysis for survival data with covariates
Comput Stat Data Anal
(1986) - et al.
RECPAM: A computer program for recursive partition and amalgamation for censored survival data and other situations frequently occurring in biostatistics. I. Methods and program features
Comput Meth Prog Biomed
(1988) - et al.
Accuracy of clinical assessment of deep-vein thrombosis
Lancet
(1995) - et al.
Value of assessment of pretest probability of deep-vein thrombosis in clinical management
Lancet
(1997) Limited assurances
Am J Cardiol
(1989)Radiographic decision-making by the emergency physician
Emerg Med Clin North Am
(1985)
Need for radiographs in the acutely injured ankle
Lancet
Evaluation of the Ottawa clinical decision rules for the use of radiography in acute ankle and midfoot injuries in the emergency department: An independent site assessment
Ann Emerg Med
Sensitivity of the Ottawa Rules
Ann Emerg Med
Cost-effectiveness analysis of the Ottawa ankle rules
Ann Emerg Med
Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations
Lancet
Clinical prediction rules: A review and suggested modifications of methodological standards
JAMA
A prediction rule to identify low-risk patients with community-acquired pneumonia
N Engl J Med
Decision rules for the use of radiography in acute ankle injuries: Refinement and prospective validation
JAMA
Guidelines you can follow and can trust: An ideal and an example
JAMA
Implementation of the Ottawa ankle rules
JAMA
Clinical predition rules: Have they come of age?
JAMA
A multicentre trial to introduce clinical decision rules for the use of radiography in acute ankle injuries
BMJ
Clinical prediction rules: Application and methodological standards
N Engl J Med
Clinimetrics
Cranial computed tomography for nontrauma patients: Critical appraisal of a decision rule
Acad Emerg Med
National Hospital Ambulatory Medical Care Survey: 1992 emergency department summary
Advance Data
National Hospital Ambulatory Medical Care Survey 1992
Use of radiography in acute ankle injuries: Physicians’ attitudes and practice
Can Med Assoc J
Use of radiography in acute knee injures: Need for clinical decision rules
Acad Emerg Med
Variation in emergency department use of cervical spine radiography for alert, stable trauma patients
Can Med Assoc J
Emergency physicians’ attitudes toward the use of clinical decison rules for radiography
Acad Emerg Med
High-yield selection criteria for cranial computed tomography after acute trauma
Acad Emerg Med
Clinical biostatistics: LIV. The biostatistics of concordance
Clin Pharmacol Ther
The measurement of observer agreement for categorical data
Biometrics
Statistical Methods for Rates and Proportions
A matrix of kappa-type coefficients to assess the reliability of nominal scales
Stat Med
The design and analysis of chemical experiments
Sample size determinations for the two rater kappa statistics
Psychometrika
Sample size and optimal designs for reliability studies
Stat Med
Sample size requirements for the comparison of two or more coefficients of interobserver agreement
Stat Med
Testing the homogeneity of kappa statistics
Biometrics
Sample size determinations using logistic regression with pilot data
Stat Med
Sample size tables for logistic regression
Stat Med
A recursive partitioning decision rule for nonparametric classification
IEEE Trans Comput
Cited by (451)
Clinical Policy: Critical Issues in the Management of Adult Patients Presenting to the Emergency Department With Acute Ischemic Stroke
2023, Annals of Emergency MedicinePediatric trauma smackdown: PTS vs SIPA
2023, InjuryComposite Outcomes in Clinical Prediction Modeling: Are We Trying to Predict Apples and Oranges?
2022, Annals of Emergency Medicine
- ☆
Dr Stiell is a career scientist of the Medical Research Council of Canada.
- ☆☆
Address for reprints:Ian G Stiell, MD, MSc, FRCPC, Clinical Epidemiology Unit, Ottawa Hospital Loeb Health Research Institute, 1053 Carling Avenue, Ottawa, Ontario, Canada, K1Y 4E9.
- ★
0196-0644/99/$8.00 + 0
- ★★
47/1/94993