Currently there are no studies evaluating web-based resources providing patient information about irritable bowel syndrome (IBS), a common condition with complex pathophysiology.
Aims: to assess the reliance of patients on the internet for medical issues and the quality of commonly viewed websites about IBS.
Methods: 198 patients were surveyed about using the internet for medical information and the most popular websites identified by their web searches for “irritable bowel syndrome” were evaluated for validity, readability, and content.
Results: although a large portion of respondents rely on the internet for medical information, the majority of online resources for IBS are not easily readable. Four of 14 are written at an eighth grade reading level or below. Of these, only two have a Content Score of over 60 out of 93. This analysis supports clinicians in taking an active role to assist patients in finding online resources to improve their understanding.
Irritable bowel syndrome (IBS) is a common functional gastrointestinal disorder with an overall prevalence of up to 20% in the United States. Most patients with IBS are managed by primary care providers. However, the syndrome remains a challenge to diagnose due to both the
heterogeneity of clinical presentations and a lack of definitive diagnostic testing. The complex pathophysiology of IBS and the limited time that health care providers have to spend with patients in a single clinic encounter make it difficult to fully educate all patients in the outpatient primary care setting. These factors may result in many patients not fully understanding the condition, making them more likely to turn to alternate sources, including the internet.
With increased awareness of IBS by the public, patients are also more commonly self-diagnosing IBS based on their own symptoms.2 In today’s technological age, more patients access and rely on the internet as a major resource when seeking medical information about themselves or others.3 Some studies also suggest that females use the internet more frequently than men for the purpose of obtaining health information.4-7 IBS is much more commonly diagnosed in women8, making it especially important for online information about IBS to be of high quality.
While recent studies have evaluated internetbased resources for other common medical conditions,9-12 there are no current studies that assess the quality of online information pertaining to IBS specifically. The purpose of our study was to identify the most commonly viewed online websites providing information about IBS and to assess their validity, readability, and relevance of content. We hypothesize that there are many online patient education resources on the topic of IBS and that these are of variable quality. With a thorough evaluation of current online information on IBS, providers taking care of patients with this syndrome can both be aware of and recommend the highest quality patient education materials to all patients using the internet as a major resource for improving their understanding of IBS.
Patients visiting an outpatient gastroenterology clinic at an academic medical center were provided instructions to search “irritable bowel syndrome” on their smartphones or tablets and to voluntarily complete a survey to report their search results and answer a set of questions pertaining to demographics and internet use for medical information. Patients were asked to search using their own devices because online search engines customize search results based on an individual’s location and previous browsing history. All adult patients visiting the clinic were eligible for participation in the study. Specific exclusion criteria included age less than 18, lack of an Apple or Android device, or inability to properly complete the survey. The survey was cross-sectional in design and patients were recruited over a two-week period. Informed consent was obtained from each participant prior to beginning the survey. Study data were collected and managed using REDCap electronic data capture tools hosted at Penn State Health Milton S. Hershey Medical Center and Penn State College of Medicine.13 REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing an intuitive interface for validated data entry, audit trails for tracking data manipulation and export procedures, automated export procedures for seamless data downloads to common statistical packages, and procedures for importing data from external sources. The study was approved by the researchers’ institutional review board.
The survey included five questions: age (organized by generation category; i.e. Millennial/Gen Y, Gen X, Baby Boomer, Mature), gender, highest level of education, and two questions about the use of and reliance on the internet for obtaining medical information. Finally, participants were asked to paste the link to their individual online search results for IBS. The complete survey instrument can be viewed in Figure 1. Results from the survey were tabulated and analyzed for trends.
Validity, Readability, and Content of Web-based Resources
Websites appearing on the first page of patient search results were noted. The 14 most frequently appearing websites from patient search results were individually scored by three researchers (L.L., N.A., L.G.) for validity, readability, and relevance of content. Results from each evaluator were averaged. Validity, as defined as the ability to trust an online resource’s information, was measured with the DISCERN (maximum score: 5) and Health on the Net (HON) tools. DISCERN is an online instrument supported by the National Health Services Executive Research and Development Programme that measures the reliability of medical information, including treatment options for conditions.14 It has three sections with 16 questions total. Though originally designed for print materials, DISCERN is viewed as an effective tool to measure the reliability of online medical information.15 The HON resource, a non-profit organization affiliated with the World Health Organization, is a measure of the quality, confidentiality, and transparency of websites.16 Online patient education websites can request certification by HON and, if specific criteria are met, this status can be indicated by an emblem displayed at the bottom of the website. A standardized tool was used to measure the readability of each online resource. The Flesch Reading Ease Score scale was originated by Rudolf Flesch and estimates the difficulty of reading passages in English.17 The Flesch-Kincaid Grade Level scale was adapted from Flesch’s original tool, initially used by the United States Navy.18 These scales measure readability based on word length, number of syllables, and sentence length. Flesch Reading Ease Scores typically span from 0 to 100, with lower scores indicating a greater degree of difficulty in reading by persons with a basic reading level. The Flesch-Kincaid Grade Level estimates the average school grade level matched to the Flesch Reading Ease Score. These tools are conveniently found in Microsoft Word; however, the calculable formulas for each of these scores is shown: Flesch Reading Ease Score = 206.835 – 1.015 x (words/sentences) – 84.6 x (syllables/words) Flesch-Kincaid Grade Level = 0.39 x (words/ sentences) + 11.8 x (syllables/words) – 15.59 Relevance of content was measured using a rubric developed by three of the researchers who specialize in the care of patients with functional gastrointestinal disorders (E.T., S.R., A.O.). The content scoring rubric was developed with reference to the Rome IV criteria.19 It contains a list of terms and phrases that are believed to be important for patients to know about IBS. The rubric includes pertinent IBS symptoms, disease mechanisms, differential diagnoses, components of the work-up, and management options. Items that were considered important received a score of 3, those that were felt to be useful but not critical information received a score of 2, and those that were factual but probably not important to patient understanding of the condition received a score of 1. The maximum score was 93. Each page’s patient education content about IBS was scored using this rubric. A sample of the Content Score rubric with point breakdown can be viewed in Figure 2
Trends in Internet Usage Among Survey Respondents 200 patients successfully completed the survey. Two were excluded due to age less than 18. Results of the survey, including demographic information, are displayed in Table 1 (N=198). 65% of the patients completing the survey were female. Nearly 40% of respondents did not have any educational degree beyond high school. When patients have questions regarding medical problems, 83% reported that they “always”, “often”, or “sometimes” use the internet, and 43% said that they use the internet as their main resource for obtaining medical information. A univariate logistic regression was performed (Table 2). This analysis reveals that there was a statistically significant (p <0.05) difference in the frequency of internet usage for medical information based on both age and gender. Compared to the 18-40 age group (Millennial/Gen Y), the age >70 group (Mature) relies on the internet for medical information significantly less frequently. Compared to males, females in the survey rely on the internet for medical information more frequently. Based on the regression results, there was no consistent statistically significant difference in frequency of internet use based on level of education.
Evaluation of Internet Resources
The 14 most commonly viewed websites on a search for IBS are shown in Table 3, along with their scores for validity, readability, and content for each website, sorted by the Content Score. Despite their presence on many patient search results, MedlinePlus and The Atlantic were excluded from evaluation because they were not formatted appropriately for analysis using the validated tools.
Results of Validity
7 of the 14 websites (50%) were certified by the Health on the Net (HON) foundation, as indicated by a visible emblem at the bottom of the page. Using the DISCERN rubric for validity, average scores for the websites ranged from 1.7 to 3.7 (out of 5). No website scored a 5 out of 5 using this rubric, primarily due to lack of detail about the exact sources of their information.
Results of Readability
No source evaluated was categorized as having “Easy” (>80) or “Fairly Easy” (70-79) readability according to the Flesch Reading Ease Score. Gastro. org, WebMD, the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), and Womenshealth.gov scored highest with >60 on the Flesch Reading Ease Score (“Standard”). All other sites were “Fairly difficult” (50-59), “Difficult” (30-49), or “Very confusing” (<29). Table 3 also shows the Flesch-Kincaid Grade Level for each website, which is the average educational grade level required to comprehend the information contained on the webpage.
Results of Content
Only 2 sources scored an average of >70 out of 93 for the content score (Wikipedia, Medicinenet). 6 sources scored between 60-69 (NIDDK, Emedicinehealth, Patient.info, MayoClinic, Medicalnewstoday, and GI.org). A comparison of Flesch Reading Ease Score and Content Score for the websites is shown in Figure 3, organized into groups based on the established Flesch Reading Ease Score categories. The Pearson’s correlation coefficient (r) between Flesch Reading Ease Score and Content Score was -0.5 (95% CI: -0.80 to 0.06), suggesting a moderate negative association between the two variables. Given the sample size of 14 websites, this result was not statistically significant at the p < 0.05 level (p = 0.0691). In general, the higher the Content Score of the resource, the more difficult it was to read (lower Flesch Reading Ease Score).
The results of this study reveal that almost 50% of patients frequently use the internet as their main educational resource for obtaining medical information, consistent with results from prior studies.20 The results support the published data that women rely on the internet as a resource for medical information more often than men.4-7 Further, trends from this study show that older generations (those aged >70) rely less on the internet as a resource for medical information than other age groups. This older age group reported that they use their health care provider most often when seeking medical information. Our study suggests that the internet is a more valuable tool for younger generations who grew up with access to the internet, highlighting the need to provide the highest quality online resources for this growing subset of patients. High quality medical information requires accuracy and depth of content, and the information should be understandable to the patient. When evaluating individual websites from an online search on IBS, the validity, readability, and relevance of content on the website could be measured. Most online IBS resources scored similarly on measurements of validity using the DISCERN scale. Half of the websites were certified by HON. Of those websites not certified by HON, some may in fact meet HON criteria due to their association with major medical associations (e.g., the National Institute of Diabetes and Digestive and Kidney Diseases, the American Gastroenterological Association, and the American Society of Colon and Rectal Surgeons). As shown in Table 3, the highest scoring resources for readability include Gastro. org, WebMD, NIDDK, and Womenshealth. gov. The highest scoring resources for relevance of content include Wikipedia, Medicinenet, NIDDK, and Emedicinehealth. Figure 3 also shows a moderate negative association between Flesch Reading Ease Score and Content Score, indicating that, in general, as websites add more pertinent content about IBS to their education materials, the readability for the average patient declines. Though the correlation was not statistically significant at the level of p < 0.05 because of the sample size of 14 websites, investigating this relationship further with additional websites about IBS may be an avenue for additional research. Creators of online patient education materials should be cognizant of this balance and design websites that are easily readable by the public but still contain the most important content.
This evaluation of popular websites reveals that most online resources providing information about IBS have poor readability, particularly Wikipedia. Given that only 4 out of 14 websites are written at the grade level recommended by the American Medical Association for patient education resources (6th-8th grade),21 there is likely poor patient comprehension of most online resources about IBS. For example, despite having the highest Content Score, Wikipedia would not be recommended as an educational source for patients due to the very advanced grade level (“16th grade”, i.e. a graduate-level degree) required to comprehend the information. Most websites did not indicate if content was directed towards health care professionals or patients. Recognizing that patients frequently use the internet for medical questions unanswered in a clinic visit, health care providers can take a proactive role in directing patients to online resources that are valid, readable, and relevant. Figure 3 serves as a tool that providers can use when managing patients with IBS, by referring patients to higher quality and appropriate online patient education material. Ideally, this would be a resource at the very topright of Figure 3 (high readability, high content). In practice, designing such a website would be difficult. However, because each patient has a different level of education and reliance on the internet for seeking medical information, providers can use Figure 3 to tailor their recommendations for online resources about IBS based on the patient’s unique needs and background. For example, a highly educated patient may appreciate a site that is slightly less readable with more detailed content. Further research is needed to determine if tailoring internet resources for patients has a beneficial impact on patients’ understanding of their disease. This study is the only thorough evaluation of online information about IBS, an exceedingly common diagnosis in the United States. The study confirms what other studies have discovered about trends in internet usage for medical information. There are limitations to the study, including the Hawthorne effect: most patients completed the brief survey in the presence of a researcher, which may have influenced the way they responded to the questions about internet usage. Additionally, the DISCERN score and Content Scores were determined by three of the study’s researchers. However, these tools were designed to be objective to limit bias. The data presented indicate that most current online information about IBS is of variable quality. Primary care physicians managing IBS patients should use these results to take an active role in directing patients to credible online resources that have key disease information written in an easily readable format. Doing so may be able to increase patient understanding of disease, improve the doctor-patient relationship and, ultimately, improve health outcomes for patients.
The research team would like to acknowledge Vernon M. Chinchilli and Sandeep Pradhan for their contribution to the statistical analysis of the data. The project described was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1 TR002014 and Grant UL1 TR00045. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
1. Canavan C, West J, Card T. The epidemiology of irritable bowel syndrome. Clin Epidemiol. 2014;6:71.
2. Van den Houte K, Carbone F, Pannemans J, et al. Prevalence and impact of self-reported irritable bowel symptoms in the general population. United European Gastroenterol J. 2019;7.2:307-15.
3. Stevenson FA, Kerr C, Murray E, Nazareth I. Information from the Internet and the doctor-patient relationship: the patient perspective–a qualitative study. BMC Fam Pract. 2007;8.1:47.
4. Basch CH, MacLean SA, Romero RA, Ethan D. Health Information Seeking Behavior Among College Students. J Community Health. 2018;43.6:1094-9.
5. Bidmon S, Terlutter R. Gender differences in searching for health information on the internet and the virtual patient-physician relationship in Germany: exploratory results on how men and women differ and why. J Med Internet Res. 2015;17.6:e156.
6. Fox S. The social life of health information 2011. Washington, DC: Pew Internet & American Life Project; 2011.
7. Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform. 2006;75.1:8-28.
8. Lovell RM, Ford AC. Effect of gender on prevalence of irritable bowel syndrome in the community: systematic review and meta-analysis. Am J Gastroenterol. 2012;107.7:991.
9. van der Marel S, Duijvestein M, Hardwick J, et al. Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis. 2009;15.12:18916.
10. Zhang D, Schumacher C, Harris MB, Bono CM. The quality and readability of information available on the internet regarding lumbar fusion. Global Spine J. 2016;29.02:133-e8.
11. Saeed F, Anderson I. Evaluating the quality and readability of Internet information on meningiomas. World Neurosurg. 2017;97:312-6.
12. Lovett J, Gordon C, Patton S, Chen CX. Online Information on Dysmenorrhea: An Evaluation of Readability, Credibility, Quality, and Usability. J Clin Nurs. 2019.
13. Paul A. Harris, Robert Taylor, Robert Thielke, Jonathon Payne, Nathaniel Gonzalez, Jose G. Conde, “Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support,” J Biomed Inform. 2009 Apr;42(2):377-81.
14. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53.2:105-11.
15. Charnock D, Shepperd S. Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res. 2004;19.4:440-46.
16. Health on the Net Foundation. [http://www.hon.ch]. Last accessed July 11, 2019.
17. Flesch R. How to write plain English: A book for lawyers and consumers. New York, NY: Harper & Row; 1979.
18. Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Institute for Simulation and Training 1975:56. [https://stars.library.ucf.edu/istlibrary/56]
19. Drossman DA, Hasler WL. Rome IV—functional GI disorders: disorders of gut-brain interaction. Gastroenterology. 2016;150.6:1257-61.
20. Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friedmann PD, Moulton AW. Patients’ use of the Internet for medical information. J Gen Intern Med. 2002;17.3:180-5. 21. Weiss B. Health literacy. American Medical Association (2003).