Developing an LLM-based Chatbot using Retrieval Augmented Generation for Families Affected by Complex Lymphatic Anomalies

Poster Number: P38
Presentation Time: 05:00 PM - 06:30 PM

Abstract Keywords: Large Language Models (LLMs), Natural Language Processing, Delivering Health Information and Knowledge to the Public, Pediatrics
Primary Track: Applications
Programmatic Theme: Clinical Informatics

We developed a GPT-based chatbot using retrieval-augmented generation (RAG) for complex lymphatic anomalies (CLAs) families. We evaluated four GPT-based models: ChatGPT, GPT (No RAG), GPT (RAG with 1 document), and GPT (RAG with 10 documents) using manual scoring on accuracy and comprehensiveness by a clinical expert. The preliminary results suggest that the CLA chatbot can generate more accurate and understandable answers when incorporating an LLM with a rare disease knowledge base.

Speaker(s):
Min Zhao
Washington University in St. Louis, School of Medicine

Author(s):
Ethan Hillis; Inez Oh, PhD - Institute for Informatics at Washington University in St. Louis, School of Medicine; Aditi Gupta - Washington University in St. Louis; Albert Lai, PhD, FACMI, FAMIA - Washington University; Bryan A. Sisk, MD, MSCI - Washington University in St Louis School of Medicine;