News
Could AI be your new doctor? Researchers say: “probably not”
Due to the ever-increasing use of large language models like ChatGPT for health consultation, the need for digital literacy is at an all-time high. However, we still have little understanding of the true accuracy of the responses generated by these models. Further, even if the information given is accurate, the ability for individuals to understand the responses is critical, particularly when they pertain to health and nutrition.
A study by Khademi et al. aimed to explore the scientific accuracy and readability of ChatGPT responses to nutritional queries related to inflammatory bowel disease (IBD). Researchers gathered twenty frequently asked questions from Google and posed them in independent sessions to ChatGPT version 3.5. The questions varied in complexity, ranging from “Can I eat rice with Crohn’s?”, to “What nutritional deficiencies are common in inflammatory bowel disease?”. The responses to these questions were then assessed by nutritionists and dietitians specialising in IBD.
The researchers found that overall, responses to these queries were rated 4.2/5 for scientific accuracy, and 4.3/5 for comprehensibility by the the IBD specialists. In terms of readability, responses required an average of about 13 years of education to be understood, though this was highly dependent on the specific question being asked.
Though the responses may have been difficult to read, these findings suggest that for general nutritional queries, ChatGPT may be a source of relatively accurate and comprehensible information for patients with IBD. However, accuracy and readability decreased for more complex queries, so further research is needed to validate these findings and explore more specialised nutrition topics.
To see all the questions that were asked and for a breakdown of the accuracy, comprehensibility, and readability of each query, see the full article here: https://doi.org/10.1038/s41598-026-66091-2