Using a customized GPT to provide guideline-based recommendations for management of pancreatic cystic lesions

Marchegiani G., Pollini T., Burelli A., Han Y., Jung H., Kwon W., Rocha Castellanos D.M., Crippa S., Belfiori G., Arcidiacono P.G., Capurso G., Apadula L., Zaccari P., Lariño Noia J., Gorris M., et. al.

Background and Aims Currently, most patients with branch duct intraductal papillary mucinous neoplasms (BD-IPMN) are offered indefinite surveillance, resulting in health care costs with questionable benefits regarding cancer prevention. This study sought to identify patients in whom the risk of cancer is equivalent to an age-matched population, thereby justifying discontinuation of surveillance. Methods International multicenter study involving presumed BD-IPMN without worrisome features (WFs) or high-risk stigmata (HRS) at diagnosis who underwent surveillance. Clusters of individuals at risk for cancer development were defined according to cyst size and stability for at least 5 years, and age-matched controls were used for comparison using standardized incidence ratios (SIRs) for pancreatic cancer. Results Of 3844 patients with presumed BD-IPMN, 775 (20.2%) developed WFs and 68 (1.8%) HRS after a median surveillance of 53 (interquartile range 53) months. Some 164 patients (4.3%) underwent surgery. Of the overall cohort, 1617 patients (42%) remained stable without developing WFs or HRS for at least 5 years. In patients 75 years or older, the SIR was 1.12 (95% CI, 0.23–3.39), and in patients 65 years or older with stable lesions smaller than 15 mm in diameter after 5 years, the SIR was 0.95 (95% CI, 0.11–3.42). The all-cause mortality for patients who did not develop WFs or HRS for at least 5 years was 4.9% (n = 79), and the disease-specific mortality was 0.3% (n = 5). Conclusions The risk of developing pancreatic malignancy in presumed BD-IPMN without WFs or HRS after 5 years of surveillance is comparable to that of the general population depending on cyst size and patient age. Surveillance discontinuation could be justified after 5 years of stability in patients older than 75 years with cysts

Gorelik Y., Ghersin I., Maza I., Klein A.

Background and Aims ChatGPT, an advanced language model, is increasingly utilized in diverse fields, including medicine. This study explores using ChatGPT to optimize post-colonoscopy management by providing guideline-based recommendations, addressing low adherence rates and timing issues. Methods In this proof-of-concept study twenty clinical scenarios were prepared as structured reports and free text notes, and ChatGPT's responses were evaluated by two senior gastroenterologists. Adherence to guidelines and accuracy were assessed, and inter-rater agreement was calculated using Fleiss' kappa coefficient. Results ChatGPT exhibited 90% adherence to guidelines and 85% accuracy, with a very good inter-rater agreement (Fleiss' kappa coefficient of 0.84, p

Dave T., Athaluri S.A., Singh S.

This paper presents an analysis of the advantages, limitations, ethical considerations, future prospects, and practical applications of ChatGPT and artificial intelligence (AI) in the healthcare and medical domains. ChatGPT is an advanced language model that uses deep learning techniques to produce human-like responses to natural language inputs. It is part of the family of generative pre-training transformer (GPT) models developed by OpenAI and is currently one of the largest publicly available language models. ChatGPT is capable of capturing the nuances and intricacies of human language, allowing it to generate appropriate and contextually relevant responses across a broad spectrum of prompts. The potential applications of ChatGPT in the medical field range from identifying potential research topics to assisting professionals in clinical and laboratory diagnosis. Additionally, it can be used to help medical students, doctors, nurses, and all members of the healthcare fraternity to know about updates and new developments in their respective fields. The development of virtual assistants to aid patients in managing their health is another important application of ChatGPT in medicine. Despite its potential applications, the use of ChatGPT and other AI tools in medical writing also poses ethical and legal concerns. These include possible infringement of copyright laws, medico-legal complications, and the need for transparency in AI-generated content. In conclusion, ChatGPT has several potential applications in the medical and healthcare fields. However, these applications come with several limitations and ethical considerations which are presented in detail along with future prospects in medicine and healthcare.

Ray P.P.

In recent years, artificial intelligence (AI) and machine learning have been transforming the landscape of scientific research. Out of which, the chatbot technology has experienced tremendous advancements in recent years, especially with ChatGPT emerging as a notable AI language model. This comprehensive review delves into the background, applications, key challenges, and future directions of ChatGPT. We begin by exploring its origins, development, and underlying technology, before examining its wide-ranging applications across industries such as customer service, healthcare, and education. We also highlight the critical challenges that ChatGPT faces, including ethical concerns, data biases, and safety issues, while discussing potential mitigation strategies. Finally, we envision the future of ChatGPT by exploring areas of further research and development, focusing on its integration with other technologies, improved human-AI interaction, and addressing the digital divide. This review offers valuable insights for researchers, developers, and stakeholders interested in the ever-evolving landscape of AI-driven conversational agents. This study explores the various ways ChatGPT has been revolutionizing scientific research, spanning from data processing and hypothesis generation to collaboration and public outreach. Furthermore, the paper examines the potential challenges and ethical concerns surrounding the use of ChatGPT in research, while highlighting the importance of striking a balance between AI-assisted innovation and human expertise. The paper presents several ethical issues in existing computing domain and how ChatGPT can invoke challenges to such notion. This work also includes some biases and limitations of ChatGPT. It is worth to note that despite of several controversies and ethical concerns, ChatGPT has attracted remarkable attentions from academia, research, and industries in a very short span of time.

Schweber A.B., Agarunov E., Brooks C., Hur C., Gonda T.A.

Using large-sample, real-world administrative claims data, we evaluated the prevalence of putatively asymptomatic pancreatic cysts, the historical growth in their incident diagnosis, and their risk of malignant progression.Data were sourced from IBM MarketScan administrative claims databases of more than 200 million patients. Period prevalence was assessed using 700,000 individuals without conditions that predispose to pancreatic cyst. The standardized cumulative incidence was compared with the cross-sectional abdominal imaging rate from 2010-2017. The risk of progression to pancreatic cancer for 14,279 newly diagnosed patients with a cyst was estimated using Kaplan-Meier analysis.Standardized prevalence increased exponentially with age and was 1.84% (95% confidence interval, 1.80%-1.87%) for patients older than 45. Standardized incidence nearly doubled from 2010-2017 (6.3 to 11.4 per 10,000), whereas the imaging rate changed from only 8.0% to 9.4%. The cumulative risk of pancreatic cancer at 7 years was 3.0% (95% confidence interval, 2.4%-3.5%), increasing linearly (R2 = 0.991) with an annual progression risk of 0.47%.Using large-sample data, we show a significant burden of asymptomatic pancreatic cysts, with an annual risk of progression to cancer of 0.47% for 7 years. Rapid growth in cyst diagnosis over the last decade far outpaced increases in the imaging rate.

Okasha H.H., Awad A., El-meligui A., Ezzat R., Aboubakr A., AbouElenin S., El-Husseiny R., Alzamzamy A.

Cystic pancreatic lesions involve a wide variety of pathological entities that include neoplastic and non-neoplastic lesions. The proper diagnosis, differentiation, and staging of these cystic lesions are considered a crucial issue in planning further management. There are great challenges for their diagnostic models. In our time, new emerging methods for this diagnosis have been discovered. Endoscopic ultrasonography-guided fine-needle aspiration cytology with chemical and molecular analysis of cyst fluid and EUS-guided fine needle-based confocal laser endomicroscopy, through the needle microforceps biopsy, and single-operator cholangioscopy/pancreatoscopy are promising methods that have been used in the diagnosis of cystic pancreatic lesions. Hereby we discuss the diagnosis of cystic pancreatic lesions and the benefits of various diagnostic models.

van Huijgevoort N.C., del Chiaro M., Wolfgang C.L., van Hooft J.E., Besselink M.G.

Pancreatic cystic neoplasms (PCN) are a heterogeneous group of pancreatic cysts that include intraductal papillary mucinous neoplasms, mucinous cystic neoplasms, serous cystic neoplasms and other rare cystic lesions, all with different biological behaviours and variable risk of progression to malignancy. As more pancreatic cysts are incidentally discovered on routine cross-sectional imaging, optimal surveillance for patients with PCN is becoming an increasingly common clinical problem, highlighting the need to balance cancer prevention with the risk of (surgical) overtreatment. This Review summarizes the latest developments in the diagnosis and management of PCN, including the quality of available evidence. Also discussed are the most important differences between the PCN guidelines from the American Gastroenterological Association, the International Association of Pancreatology and the European Study Group on Cystic Tumours of the Pancreas, including diagnostic and follow-up strategies and indications for surgery. Finally, new developments in the management of patients with PCN are addressed. Pancreatic cystic neoplasms are a heterogeneous group of pancreatic cysts with different biological behaviours and variable risk of progression to malignancy. This Review summarizes the latest developments in the diagnosis and management of pancreatic cysts, including the quality of available evidence.

Schenck R.J., Miller F.H., Keswani R.N.

We aimed to determine incidental pancreatic cyst ("cyst") surveillance patterns, predictors of receiving surveillance, and guideline adherence.We performed a retrospective cohort study of all patients receiving longitudinal care at a single tertiary care center with a newly diagnosed incidental pancreatic cyst over a 2-year period (2010-2011). All follow-up care was abstracted over a 5-year period.Of 3241 eligible imaging studies reviewed, 100 patients with newly diagnosed incidental cysts eligible for surveillance were identified. A majority (53%) received no follow-up. We identified 4 predictors of cyst surveillance: radiology report conclusion mentioning the cyst (odds ratio [OR], 14.9; 95% confidence interval [CI], 1.9-119) and recommending follow-up (OR, 5.5; 95% CI, 2.1-13.9), pancreas main duct dilation (OR, 10.7; 95% CI, 1.3-89), and absence of multiple cysts (OR, 2.5; 95% CI, 1.1-10.0). Of the 47 patients who received surveillance, 66% met minimum surveillance imaging intervals of at least one guideline. Conversely, 21% of patients met the criteria for overutilization in at least one guideline.Although guidelines recommend that surgically fit patients with incidental cysts undergo surveillance, most patients receive no follow-up. When follow-up occurs, surveillance patterns vary widely and infrequently conform to guidelines. Interventions to reduce care variation require study.

Evidence-based guidelines on the management of pancreatic cystic neoplasms (PCN) are lacking. This guideline is a joint initiative of the European Study Group on Cystic Tumours of the Pancreas, United European Gastroenterology, European Pancreatic Club, European-African Hepato-Pancreato-Biliary Association, European Digestive Surgery, and the European Society of Gastrointestinal Endoscopy. It replaces the 2013 European consensus statement guidelines on PCN. European and non-European experts performed systematic reviews and used GRADE methodology to answer relevant clinical questions on nine topics (biomarkers, radiology, endoscopy, intraductal papillary mucinous neoplasm (IPMN), mucinous cystic neoplasm (MCN), serous cystic neoplasm, rare cysts, (neo)adjuvant treatment, and pathology). Recommendations include conservative management, relative and absolute indications for surgery. A conservative approach is recommended for asymptomatic MCN and IPMN measuring <40 mm without an enhancing nodule. Relative indications for surgery in IPMN include a main pancreatic duct (MPD) diameter between 5 and 9.9 mm or a cyst diameter ≥40 mm. Absolute indications for surgery in IPMN, due to the high-risk of malignant transformation, include jaundice, an enhancing mural nodule >5 mm, and MPD diameter >10 mm. Lifelong follow-up of IPMN is recommended in patients who are fit for surgery. The European evidence-based guidelines on PCN aim to improve the diagnosis and management of PCN.

Elta G.H., Enestvedt B.K., Sauer B.G., Lennon A.M.

Pancreatic cysts are very common with the majority incidentally identified. There are several types of pancreatic cysts; some types can contain cancer or have malignant potential, whereas others are benign. However, even the types of cysts with malignant potential rarely progress to cancer. At the present time, the only viable treatment for pancreatic cysts is surgical excision, which is associated with a high morbidity and occasional mortality. The small risk of malignant transformation, the high risks of surgical treatment, and the lack of high-quality prospective studies have led to contradictory recommendations for their immediate management and for their surveillance. This guideline will provide a practical approach to pancreatic cyst management and recommendations for cyst surveillance for the general gastroenterologist.

Tanaka M., Fernández-del Castillo C., Kamisawa T., Jang J.Y., Levy P., Ohtsuka T., Salvia R., Shimizu Y., Tada M., Wolfgang C.L.

The management of intraductal papillary mucinous neoplasm (IPMN) continues to evolve. In particular, the indications for resection of branch duct IPMN have changed from early resection to more deliberate observation as proposed by the international consensus guidelines of 2006 and 2012. Another guideline proposed by the American Gastroenterological Association in 2015 restricted indications for surgery more stringently and recommended physicians to stop surveillance if no significant change had occurred in a pancreatic cyst after five years of surveillance, or if a patient underwent resection and a non-malignant IPMN was found. Whether or not it is safe to do so, as well as the method and interval of surveillance, has generated substantial debate. Based on a consensus symposium held during the meeting of the International Association of Pancreatology in Sendai, Japan, in 2016, the working group has revised the guidelines regarding prediction of invasive carcinoma and high-grade dysplasia, surveillance, and postoperative follow-up of IPMN. As the working group did not recognize the need for major revisions of the guidelines, we made only minor revisions and added most recent articles where appropriate. The present guidelines include updated information and recommendations based on our current understanding, and highlight issues that remain controversial or where further research is required.

Vege S.S., Ziring B., Jain R., Moayyedi P., Adams M.A., Dorn S.D., Dudley-Brown S.L., Flamm S.L., Gellad Z.F., Gruss C.B., Kosinski L.R., Lim J.K., Romero Y., Rubenstein J.H., Smalley W.E., et. al.

This article has an accompanying continuing medical education activity on page e12. Learning Objective: At the conclusion of this exercise, the learner will understand the approach to counseling patients regarding the optimal method and frequency of radiologic imaging, indications for invasive tests like endoscopic ultrasonography (EUS) and surgery, select patients for follow-up after surgery, decide the duration of such follow-up, and decide when to stop surveillance for those with and without surgery.

Sigot M., Tassoti S.

Cao H., Zhao Q., Wang H., Wang R.

Saraiva M.M., Ribeiro T., Agudo B., Afonso J., Mendes F., Martins M., Cardoso P., Mota J., Almeida M.J., Costa A., Gonzalez Haba Ruiz M., Widmer J., Moura E., Javed A., Manzione T., et. al.

Background: Several artificial intelligence systems based on large language models (LLMs) have been commercially developed, with recent interest in integrating them for clinical questions. Recent versions now include image analysis capacity, but their performance in gastroenterology remains untested. This study assesses ChatGPT-4’s performance in interpreting gastroenterology images. Methods: A total of 740 images from five procedures—capsule endoscopy (CE), device-assisted enteroscopy (DAE), endoscopic ultrasound (EUS), digital single-operator cholangioscopy (DSOC), and high-resolution anoscopy (HRA)—were included and analyzed by ChatGPT-4 using a predefined prompt for each. ChatGPT-4 predictions were compared to gold standard diagnoses. Statistical analyses included accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC). Results: For CE, ChatGPT-4 demonstrated accuracies ranging from 50.0% to 90.0%, with AUCs of 0.50–0.90. For DAE, the model demonstrated an accuracy of 67.0% (AUC 0.670). For EUS, the system showed AUCs of 0.488 and 0.550 for the differentiation between pancreatic cystic and solid lesions, respectively. The LLM differentiated benign from malignant biliary strictures with an AUC of 0.550. For HRA, ChatGPT-4 showed an overall accuracy between 47.5% and 67.5%. Conclusions: ChatGPT-4 demonstrated suboptimal diagnostic accuracies for image interpretation across several gastroenterology techniques, highlighting the need for continuous improvement before clinical adoption.

Gong E.J., Bang C.S., Lee J.J., Park J., Kim E., Kim S., Kimm M., Choi S.

Background: The large language model (LLM) has the potential to be applied to clinical practice. However, there has been scarce study on this in the field of gastroenterology. Aim: This study explores the potential clinical utility of two LLMs in the field of gastroenterology: a customized GPT model and a conventional GPT-4o, an advanced LLM capable of retrieval-augmented generation (RAG). Method: We established a customized GPT with the BM25 algorithm using Open AI’s GPT-4o model, which allows it to produce responses in the context of specific documents including textbooks of internal medicine (in English) and gastroenterology (in Korean). Also, we prepared a conventional ChatGPT 4o (accessed on 16 October 2024) access. The benchmark (written in Korean) consisted of 15 clinical questions developed by four clinical experts, representing typical questions for medical students. The two LLMs, a gastroenterology fellow, and an expert gastroenterologist were tested to assess their performance. Results: While the customized LLM correctly answered 8 out of 15 questions, the fellow answered 10 correctly. When the standardized Korean medical terms were replaced with English terminology, the LLM’s performance improved, answering two additional knowledge-based questions correctly, matching the fellow’s score. However, judgment-based questions remained a challenge for the model. Even with the implementation of ‘Chain of Thought’ prompt engineering, the customized GPT did not achieve improved reasoning. Conventional GPT-4o achieved the highest score among the AI models (14/15). Although both models performed slightly below the expert gastroenterologist’s level (15/15), they show promising potential for clinical applications (scores comparable with or higher than that of the gastroenterology fellow). Conclusions: LLMs could be utilized to assist with specialized tasks such as patient counseling. However, RAG capabilities by enabling real-time retrieval of external data not included in the training dataset, appear essential for managing complex, specialized content, and clinician oversight will remain crucial to ensure safe and effective use in clinical practice.

Gong E.J., Bang C.S., Lee J.J., Park J., Kim E., Kim S., Kimm M., Choi S.

Background As health care continues to evolve with technological advancements, the integration of artificial intelligence into clinical practices has shown promising potential to enhance patient care and operational efficiency. Among the forefront of these innovations are large language models (LLMs), a subset of artificial intelligence designed to understand, generate, and interact with human language at an unprecedented scale. Objective This systematic review describes the role of LLMs in improving diagnostic accuracy, automating documentation, and advancing specialist education and patient engagement within the field of gastroenterology and gastrointestinal endoscopy. Methods Core databases including MEDLINE through PubMed, Embase, and Cochrane Central registry were searched using keywords related to LLMs (from inception to April 2024). Studies were included if they satisfied the following criteria: (1) any type of studies that investigated the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) studies published in English, and (3) studies in full-text format. The exclusion criteria were as follows: (1) studies that did not report the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) case reports and review papers, (3) ineligible research objects (eg, animals or basic research), and (4) insufficient data regarding the potential role of LLMs. Risk of Bias in Non-Randomized Studies—of Interventions was used to evaluate the quality of the identified studies. Results Overall, 21 studies on the potential role of LLMs in gastrointestinal disorders were included in the systematic review, and narrative synthesis was done because of heterogeneity in the specified aims and methodology in each included study. The overall risk of bias was low in 5 studies and moderate in 16 studies. The ability of LLMs to spread general medical information, offer advice for consultations, generate procedure reports automatically, or draw conclusions about the presumptive diagnosis of complex medical illnesses was demonstrated by the systematic review. Despite promising benefits, such as increased efficiency and improved patient outcomes, challenges related to data privacy, accuracy, and interdisciplinary collaboration remain. Conclusions We highlight the importance of navigating these challenges to fully leverage LLMs in transforming gastrointestinal endoscopy practices. Trial Registration PROSPERO 581772; https://www.crd.york.ac.uk/prospero/

Gorelik Y.

Maida M., Celsa C., Lau L.H., Ligresti D., Baraldo S., Ramai D., Di Maria G., Cannemi M., Facciorusso A., Cammà C.

Large language models (LLMs) are transforming the medical landscape by enhancing access to information, diagnostics, treatment customization, and medical education, especially in areas like Gastroenterology. LLMs utilize extensive medical data to improve decision-making, leading to better patient outcomes and personalized medicine. These models are instrumental in interpreting medical literature and synthesizing patient data, facilitating real-time knowledge for physicians and supporting educational pursuits in medicine. Despite their potential, the complete integration of LLMs in real-life remains ongoing, particularly requiring further study and regulation. This review highlights the existing evidence supporting LLMs’ use in Gastroenterology, addressing both their potential and limitations. Recent studies demonstrate LLMs’ ability to answer questions from physicians and patients accurately. Specific applications in this field, such as colonoscopy, screening for colorectal cancer, and hepatobiliary and inflammatory bowel diseases, underscore LLMs’ promise in improving the communication and understanding of complex medical scenarios. Moreover, the review discusses LLMs’ efficacy in clinical contexts, providing guideline-based recommendations and supporting decision-making processes. Despite these advancements, challenges such as data completeness, reference suitability, variability in response accuracy, dependency on input phrasing, and a lack of patient-generated questions underscore limitations in reproducibility and generalizability. The effective integration of LLMs into medical practice demands refinement tailored to specific medical contexts and guidelines. Overall, while LLMs hold significant potential in transforming medical practice, ongoing development and contextual training are essential to fully realize their benefits.

Daungsupawong H., Wiwanitkit V.

Gorelik Y.