Reading the Literature to Change Your Clinical Practice

Kurt P. Spindler, MD
Vanderbilt Sports Medicine Ctr
2601 Jess Neely Dr
Nashville, TN 37212-2039, USA
Tel: +1 615 343 1685
Fax: +1 615 322 7126

Evidence-based medicine, or EBM, is the application of the scientific method to clinical practice.  In other disciplines of medicine, particularly noninvasive or drug related, EBM is becoming the cornerstone of decision-making.  Yet in surgical fields, the application of randomized controlled trials (RCTs) to determine most efficacious treatment is more difficult.  Surgeons are uncomfortable with and misunderstand the value that EBM approach can bring to their practice.

Being an advocate of EBM approach in decision-making does not mean the exclusion of previous observational case control studies, training, or experience.  Rather, it is raising the standard or bar by which treatment decisions are made.  Since RCTs are the most accurate way to judge treatment decision-making, but are extremely costly, they are usually restricted to major principles or fundamentals.

In fact, based on both cost and time to complete, the majority of clinical practices cannot be EBM.  For example, a group of fellowship-trained orthopaedic sports medicine physicians jokingly wrote, "Patient requests treatment for diagnosis but the EBM physician replies 'I'm sorry, I can't treat you because we have only two RCTs, both of which are unrelated to your diagnosis'."  Even though these three know that EBM approach does not preclude treatment decisions, but should be considered in the first review of literature and be most important in decision-making,they reflect a general misunderstanding about benefits.  Just as the American Journal of Sports Medicine (AJSM) and the Journal of Bone and Joint Surgery-American (JBJS-Am) have either required authors to state design (AJSM), or classify based on EBM hierarchy (JBJS-Am), readers should prioritize the evidence of the literature or article to reach the most scientific, and therefore most efficacious, treatments for their patients.

The goal of this article, and EBM in general, is to provide the best care for patients.  It is our opinion that, if following a simple algorithm can provide a necessary tool to review articles, literature, or presentations, each physician will make better decisions to change their clinical practice accordingly, and insist on better controlled clinical treatment trials in the future.

The two-page algorithm is one approach.  It is developed from Trisha Greenhalgh's book, How to Read a Paper: An Evidence-Based Approach, as well as Thomas A. Lang and Michelle Secic's How to Report Statistics in Medicine, and has been used in systematic reviews of literature by the current author.  This is not the only approach or necessarily the best, but it is a clear advance in our scientific or EBM review.  This algorithm has also served as a guide to evaluate peer-review publications and clinical grants.  It is used as part of an instructional course lecture at the Annual Meeting of the American Orthopaedic Society for Sports Medicine.

The first step is to record the basics:  title, author, reference, and funding source(s).

Next, one will need to identify the hypotheses, both primary and secondary.  For example, what is the difference in KT-1000 knee stability after hamstring vs. patellar tendon ACL reconstruction, or difference in reoperation for failed meniscus repair between all-inside vs. inside-out meniscus repair.

The type of study being attempted will determine the most appropriate study design.  Five major types of study include treatment, diagnosis, screening, prognosis, and causation.  If the primary hypothesis or topic of research is treatment (drug, prevention strategy, surgical procedure, or rehabilitation), the preferred research design is a RCT (others discussed below). To determine prognosis of disease, injury or surgical treatment, the preferred design is a prospective longitudinal cohort.  Likewise, to establish causation (exposure to X), a cohort or case-control is preferred.  A cross-sectional survey is recommended to establish diagnosis (diagnostic test) or screening to determine the value of a test.

The traditional hierarchy of clinical treatment studies is: 1) RCT; 2) cohort (two or more groups selected basis of different exposure to "agent" and follow-up); 3) case control where patients with particular on disease or condition are identified and "matched" with controls; 4) cross-sectional where data are collected at a single time point; and 5) case reports or series where medical histories on one or more patients with a condition/treatment (injury, surgery, etc.) are reported.

Unfortunately, the predominant orthopaedic literature is on the case series, which are uncontrolled (no comparison or control group).

In the methods section of the algorithm, is there a control or comparison group in the treatment study?  Did the authors eliminate intervention or performance bias?  (This refers to the identification and analysis of major variables that could influence result other than treatment.  For example, if intervention (meniscus allograft) improves function postoperatively but most had ACL reconstruction and/or high tibial osteotomy, both of the other procedures (intervention) clearly have been shown to improve function.)

Next, is a gold standard evaluated in diagnostic study?

Finally, is the data collection and/or study prospective or retrospective?  A prospective design allows better control of confounding variables (like ACL reconstruction or high tibial osteotomy in the above example) because data collection is planned in advance.  A retrospective study is a review of "normally" collected information like chart reviews.

Identifying potential forms of bias is the next critique of methods of study.  Table 1 lists four forms of bias (selection, performance, exclusion or transfer, and detection) and where they occur in study design.

 HOW TO IDENTIFY BIAS

The definition of selection or susceptibility bias is the difference in comparison groups, secondary to incomplete randomization.  An example is all males in one group, or self-selection by patients to treatment group.  Performance bias is difference in care or treatment provided apart from intervention being studied (see allograft meniscus complications).  Differences in withdrawal or follow-up <70-80% is exclusion or transfer bias.  If a study reports only 30% follow-up for knee stability after two treatments, the inclusion of lost to follow-up could alter results.  Finally, detection bias is different evaluation for objective or subjective outcome measure.  This is best performed by independent or blinded examiners or validated outcome questions.

Continuing in completing the worksheet, did the author demonstrate demographics (age, gender) are equal between groups?  Next, what is follow-up - both minimum and average?  (Though a statistician best performs statistical scrutiny, several basics are worth noting.)  If measure is continuous like age, height, KT-1000, strength (if distribution passes normality content), the parametric tests like t-test or ANOVA can be performed.  If data is frequency or discrete (yes or no), then nonparametric tests can be performed. (Refer to the Greenhalgh book.)  Finally, one should simply say if stats are acceptable, yes, no, or unknown, and if stats consultation is needed.

List the desired outcome and, importantly, the difference with p value.  If results are not significant, then a power calculation should be available with a minimum 80% to detect a clinically meaningful difference.  The final evaluation determines if a significant result is clinically relevant.  For example, if the results from 3 studies of KT-1000 show difference of 1, 2, or 3 mm side-to-side difference, then it is likely that, as the difference increases, a higher percentage of surgeons will change technique.

To summarize, the best EBM designs should be first in the choice of treatments; i.e. choose based RCTs first, then cohort, etc.

When finished with the worksheet, the reader needs to answer the following questions:

1. If no comparison group or control group in TREATMENT STUDY® READ FOR INFORMATION ONLY - no evidence-based reason or data to change practice pattern.
2. If no comparison of groups that equal PreTREATMENT® RESULTS MAY NOT BE CAUSED BY TREATMENT BUT BY DIFFERENCES IN GROUPS.
3. If INTERVENTION contains additional proven or suspected variables other than 1° hypothesis indicating Performance Bias, results NOT SPECIFIC TO INTERVENTION® RESULTS UNCLEAR SIGNIFICANCE TO TREATMENT.
4. Inadequate follow-up (<70%) indicates EXCLUSION or TRANSFER BIAS® RESULTS COULD CHANGE IF ADEQUATE FOLLOW-UP.
5. Are statistical evaluations acceptable?  If not, either consult a statistician or request if editor evaluated for.
6. Finally, are absolute values that are statistically significant also clinically relevant?  A suspect few people would alter their preferred ACL reconstruction technique for 1 mm KT-1000 side-to-side difference, but a large percentage would be for 3 mm KT-1000.  Both could be statistically significant.

If A®F are acceptable, your patient population is similar to the study population, and you are trained or comfortably performing technique or treatment, change your practice accordingly.  Caveat:  This is one approach to simplifying an application of EBM to reading the literature. 

We welcome your comments and constructive ideas.

Click here to view the two-page algorithm.