2015 ISAKOS Biennial Congress Paper #128

Results from the Arthroscopic Association of North America (AANA) Copernicus Initiative; A Multicentre Prospective, Randomized, Blinded Trial of Proficiency-Based Progression Training Employing Simulation for an Arthroscopic Bankhart Procedure

Richard L. Angelo, MD, PhD, Woodinville, WA UNITED STATES
Richard K. Ryu, MD, Santa Barbara, CA UNITED STATES
Robert A. Pedowitz, MD, PhD, Santa Monica, CA UNITED STATES
Anthony Gallagher, PhD, DSc, Cork IRELAND
AANA Orthopedic Learning Center, Rosemont, Illinois, USA

FDA Status Not Applicable

Summary: This multicentre prospective, randomized, blinded trial of proficiency-based progression (PBP) training with simulation, unequivocally demonstrates the superiority of this training strategy; residents randomized to the PBP training arm were 5.5 times more likely to achieve the final proficiency benchmark in performing an arthroscopic Bankart repair when compared to a traditionally trained group.

Abstract:

Purpose

To determine in a national, multicentre, prospective, randomized and blinded study the effectiveness of proficiency based progression (PBP) training using simulation in comparison to both the exact same training without proficiency requirements as well as to a traditional AANA Resident Course approach for learning to perform an arthroscopic Bankart repair.

Methods

44 fourth or fifth year orthopaedic residents from 21 approved US residency programs were randomly assigned to one of three different skills training protocols: 1) A traditional AANA weekend Resident Training Course (Group A, n = 14); 2) A modified curriculum which added the use of a dry shoulder model simulator (Group B, n = 14); or 3) A proficiency-based progression protocol also employing the model simulator (Group C, n = 16) for learning an arthroscopic Bankart repair. At the completion of training all subjects from each of the three study groups performed a 3-suture anchor arthroscopic Bankart repair on a cadaver shoulder. The procedure was videotaped in its entirety and independently scored in blinded fashion employing previously validated metrics for procedure steps and intra-operative errors.

Results

The inter-rater reliability (IRR) of all scored video recorded procedures was high (IRR ave. = 0.93). The PBP trained group (C) performed significantly more Bankart procedure steps (p < 0.000) and made significantly fewer objectively assessed intra-operative errors: ~60% fewer than the traditionally trained group (A) (p < 0.001) and ~46% fewer than Group B (p < 0.007) which received the exact same curriculum without the requirement to demonstrate proficiency during training. 75% of those in the PBP trained Group C who met all of the intermediate benchmarks, also achieved the proficiency benchmark on their final Bankart repair in comparison to 42.8% of Group B and 28.6% of Group A. Compared to Group A (control), Group B participants were 1.8 times more likely and Group C subjects were 5.5 times more likely to achieve the final proficiency benchmark. Those in Group C who met all of the intermediate proficiency benchmarks were 7.5 times more likely than Group A to achieve the final benchmark.

Conclusion

A PBP training protocol coupled with the use of a dry shoulder model simulator and previously validated Bankart procedure metrics produce a superior skill set when compared to both traditional resident training as well as an identical curriculum without the requirement to demonstrate proficiency to progress. This is the largest prospective, randomized and blinded evaluation of PBP simulation training for a complete surgical procedure and was led by a professional surgical organization (AANA). This study outlines the methodology derived from first principles of performance metrics by which other surgical training organizations might develop and validate performance metrics and use them for effective training on existing simulation platforms.

CLINICAL RELEVANCE
Metric-based PBP simulation training produces a superior skill set when compared to traditional training methods and will play an essential role in delivering efficient, safe, and effective surgical skill training curriculums.