Summary

There is significant statistical fragility in published clinical trials reporting on autograft choice in ACL reconstruction surgery. This indicates that the reported outcomes should be interpreted with caution. Thus, we recommend co-reporting of fragility indices and p-values to provide a more comprehensive representation of a study’s conclusions when conducting comparative clinical trials.

Abstract

Introduction

The fragility index (FI) and reverse fragility index (rFI) are statistical metrics that address some of the limitations of the P-value. They represent the number of outcome event reversals required to reverse statistical significance. When co-reported, FI and rFI provide a more comprehensive representation of clinical trial data.

Objective

To determine the statistical fragility of clinical trials comparing bone-patellar tendon-bone (BPTB) and hamstring tendon (HT) autografts in primary anterior cruciate ligament (ACL) reconstruction surgery.

Methods

PubMed, EMBASE, and MEDLINE were queried for clinical trials published since 2010 that compared outcomes in ACL reconstruction surgery where experimental groups were stratified by autograft type (i.e. BPTB vs HT). Statistical significance was defined as a P-value <0.05. The FI and rFI were determined – for significant and non-significant outcomes, respectively – by determining the number of outcome reversals required to change statistical significance. The fragility quotient (FQ) was calculated by dividing the FI or rFI by the sample size. Values were reported with interquartile ranges (IQR).

Results

Our review identified 26 clinical trials, 21 of which were randomized controlled trials (RCTs), from 13 orthopaedic journals reporting 136 total dichotomous outcomes. The median FI of the 136 total outcomes was 2 (IQR 2-4) with an associated FQ of 0.0957 (IQR 0.0513-0.1631). 11 outcomes were reported as statistically significant with a median FI of 2 (IQR 1.5-3) and a median FQ of 0.0198 (IQR 0.0185-0.0428). 65 outcomes were reported as non-significant with a median rFI of 8 (IQR 6-10) and a median FQ of 0.1064 (IQR 0.0638-0.1596). Additionally, a subgroup analysis was performed on the most frequent reported outcomes, such as Lachman scores, pivot shift testing, and pain scores. Notably, the median FI was less than the number of patients lost to follow-up in 86.7% of the outcomes.

Conclusion

There is significant statistical fragility in current published clinical trials reporting on autograft choice in ACL reconstruction surgery. This indicates that the reported outcomes should be interpreted with caution. Thus, we recommend co-reporting of fragility indices and p-values to provide a more comprehensive representation of a study’s conclusions when conducting comparative clinical trials.