The purpose of this study is to identify the construct validity of a hip arthroscopy simulator when the Arthroscopic Surgery Skill Evaluation Tool (ASSET), a validated 8 domain scoring tool used to assess global arthroscopic technical skill, is used to assess technical ability of surgeons with different levels of experience.
Validation of a Hip Arthroscopy Simulator Using The Arthroscopic Surgery Skill Evaluation Tool (ASSET)
Meghan E. Bishop, MD; Stephan Zmugg, MD; Brenda Chang, MPH; Ryan Rauck, MD; Daniel Hurwit, MD; Anil Ranawat, MD
Presenter and corresponding author: Meghan Bishop, MD; 215-620-3848
Objectives: Arthroscopic simulation training is becoming increasingly popular. Studies of efficacy in the shoulder and knee have shown arthroscopic simulation may lead to improvement in arthroscopic skills. However, there is minimal data concerning the efficacy of arthroscopic hip simulation. The purpose of this study is to identify the construct validity of a hip arthroscopy simulator when the Arthroscopic Surgery Skill Evaluation Tool (ASSET), a validated 8 domain scoring tool used to assess global arthroscopic technical skill, is used to assess technical ability of surgeons with different levels of experience. We hypothesize that advanced experience surgeons will perform better than the novice experience residents/students.
Medical students and orthopaedic residents and fellows from a single institution completed a simulated diagnostic arthroscopy of the hip and a therapeutic loose body removal exercise. Those with no prior training in arthroscopy (medical students through post graduate year [PGY] 2s) were designated to the novice group and those with prior training in arthroscopy (PGY3s through fellows) were designated to the experienced group. Using the supine position through the anterolateral portal for viewing, participants identified 9 structures in the central compartment and 6 structures in the peripheral compartment followed by a task requiring the removal of two loose bodies. Participants completed the tests on two separate occasions. Simulator metric scores (total operation time, camera/grasper path length, scratching of the femoral/acetabular cartilage, loose bodies retrieved) were compared with case experience and experience level. Simulator total score was correlated with the ASSET score as graded by two sports medicine fellows.
A total of 27 participants (20 male/7 female; average age 30 years old SD +/-5) completed the hip arthroscopy simulator module. There were 12 participants in the novice group (8 medical students, 3 PGY1s, 1 PGY2) and 15 participants in the experienced group (4 PGY3s, 1 PGY4, 2 PGY5s, 7 fellows, 1 attending). Number of arthroscopy cases performed correlated significantly with higher total diagnostic score (p=0.040) and approached but did not reach significance with ASSET score (p=0.055). Participants in the experienced group completed the diagnostic arthroscopy module faster (387.9 seconds +/-202.5 vs 581.1 +/-129.7, p=0.008) and removed more loose bodies (p=0.004) than those in the novice group. Experience in training (p=0.017) as well as prior simulation experience (p=0.042) significantly correlated with higher ASSET score.
There was no significant difference in overall simulator metric scores for the diagnostic and therapeutic tests between repeat tests (p=0.410; p=0.159). However, the diagnostic test procedure time for the repeat exam took significantly less time to complete (p=0.008). Females improved significantly more than males between repeat tests in change in total score (p=0.019) and change in procedure time (p=0.003). For ASSET scoring participants performed significantly better on repeat testing (p=0.006). There was very good inter-rater reliability with ASSET scoring on original (ICC=0.940 [0.916-0.961]) and repeat testing (ICC=0.948 [0.923-0.969]).
Increased experience in training as well as arthroscopy cases performed was associated with improved performance over a number of simulated tasks. There were significant correlations between number of arthroscopy cases performed with total diagnostic test score as well as training experience with diagnostic test procedure time and total loose bodies removed. Training experience significantly correlated with ASSET score but not with number of arthroscopy cases performed.