Digesting Digital Health: A Study of Appropriateness and Readability of ChatGPT-Generated Gastroenterological Information
INTRODUCTION:
The advent of artificial intelligence–powered large language models capable of generating interactive responses to intricate queries marks a groundbreaking development in how patients access medical information. Our aim was to evaluate the appropriateness and readability of gastroenterological information generated by Chat Generative Pretrained Transformer (ChatGPT).
METHODS:
We analyzed responses generated by ChatGPT to 16 dialog-based queries assessing symptoms and treatments for gastrointestinal conditions and 13 definition-based queries on prevalent topics in gastroenterology. Three board-certified gastroenterologists evaluated output appropriateness with a 5-point Likert-scale proxy measurement of currency, relevance, accuracy, comprehensiveness, clarity, and urgency/next steps. Outputs with a score of 4 or 5 in all 6 categories were designated as “appropriate.” Output readability was assessed with Flesch Reading Ease score, Flesch-Kinkaid Reading Level, and Simple Measure of Gobbledygook scores.
RESULTS:
ChatGPT responses to 44% of the 16 dialog-based and 69% of the 13 definition-based questions were deemed appropriate, and the proportion of appropriate responses within the 2 groups of questions was not significantly different (
DISCUSSION:
ChatGPT can produce generally fitting responses to gastroenterological medical queries, but responses were constrained in appropriateness and readability, which limits the current utility of this large language model. Substantial development is essential before these models can be unequivocally endorsed as reliable sources of medical information.
Топ-30
Журналы
|
1
|
|
|
Bioengineering
1 публикация, 20%
|
|
|
Journal of Medical Internet Research
1 публикация, 20%
|
|
|
American Journal of Obstetrics and Gynecology
1 публикация, 20%
|
|
|
Frontiers in Public Health
1 публикация, 20%
|
|
|
1
|
Издатели
|
1
|
|
|
MDPI
1 публикация, 20%
|
|
|
JMIR Publications
1 публикация, 20%
|
|
|
Elsevier
1 публикация, 20%
|
|
|
Frontiers Media S.A.
1 публикация, 20%
|
|
|
Cold Spring Harbor Laboratory
1 публикация, 20%
|
|
|
1
|
- Мы не учитываем публикации, у которых нет DOI.
- Статистика публикаций обновляется еженедельно.