Summary

Online patient education materials related to rotator cuff surgery scored poorly with respect to readability, understandability, and actionability.

Abstract

Introduction

The American Medical Association (AMA) and National Institutes of Health (NIH) recommend online health information to be written at a 6th grade or lower reading level in order to be fully understood by the average adult in the United States. While syntax reading grade-level has previously been utilized, these analyses do not measure whether text is written such that readers can process key information (understandability) or identify available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT-P) is a valid and reliable method to measure the understandability and actionability of online patient education materials. The purpose of this study was to evaluate online resources regarding rotator cuff repair utilizing measures of readability, understandability, and actionability. It was hypothesized that current online resources score poorly on these objective measures. Additionally, it was hypothesized that these measures would not correlate with the order of the listed search results (rank).

Methods

Using trend analysis (trends.google.com), the most common searched term “rotator cuff surgery” was determined. Subsequently two independent online searches (Google.com) were performed. From the top 50 search results, websites were included if directed at educating patients. News articles, non-text material (video), articles (news/research/industry), and unrelated websites were excluded. The readability of included resources was quantified using valid objective algorithms: Flesch-Kincaid Grade-Level (FKGL), Simple Measure of Gobbledygook (SMOG) grade, Coleman-Liau Index (CLI), and Gunning-Fog Index (GFI). The PEMAT-P form was used to assess actionability and understandability. PEMAT score of 70% or below is considered poorly understandable or poorly actionable. The relationship between a website’s average search rank and its readability, understandability, and actionability was calculated.

Results

A total of 58 unique websites were identified and 49 sites (84.5%) met inclusion criteria. The mean FKGL graded materials at approximately a 10th grade reading level, with only two websites offering materials at a 6th grade reading level or below. The remaining readability studies graded the mean reading level at high school or greater, with the Gunning Fog Index scoring at a collegiate reading level. Mean understandability and actionability scores were 64.6% and 29.5% respectively, falling below the 70% PEMAT score threshold for both scales. Fourteen (28.6%) websites were above the threshold for understandability, while no website (0%) scored above the 70% threshold for actionability. When comparing source categories, commercial health publishers provided websites that scored higher in understandability, while private practice materials scored higher in actionability (p=0.013). Resources published by academic institutions or organizations scored lower in both understandability and actionability than private practice and commercial health publishers. No readability, understandability, or actionability score was significantly associated with search result rank.

Conclusion

Overall, online patient education materials related to rotator cuff surgery scored poorly with respect to readability, understandability, and actionability. Only two patient education websites scored at the AMA and NIH recommended reading level. Fourteen scored above the 70% PEMAT score on understandability, while no single website met the threshold for actionability. Online resources distributed by academic institutions and organizations scored lower in both PEMAT measures compared to commercial health publishers and private practices. If online resources are misunderstood due to poor readability, understandability, or actionability, these resources may lead to poor decision making by patients. Future efforts should be made to improve online resources in order to optimize patient knowledge and facilitate informed decision-making.