Don’t be Afraid of the API: A Hands-on Workshop using Large Language Model APIs to Extract Data from Clinical Notes

Presentation Type: Workshop
Not recorded for AMIA Now

Presentation Time: 10:30 AM - 12:30 PM

Abstract Keywords: Analytical Artificial Intelligence: ML, Digital Pathology, Imaging AI, Predictive Analytics, Governance, Health Data Science, Generative AI in Clinical Workflow: Ambient Listening, Chart Summarization, Automated Response with LLM
Primary Track: Big Data for Health


Clinical notes, patient messages, pathology reports, and radiology reports contain critical information in an unstructured format for patient care, quality improvement, and research. Manual chart review is commonly used to extract information from these sources, but this is time-consuming, error-prone, and can be impacted by bias. Large language models (LLMs) now make it possible to automate much of this work. Through easy-to-use Application Programming Interfaces (APIs), any clinician or informaticist now can interact with LLMs to extract key insights from clinical text without needing natural language processing (NLP) expertise. These API-based tools are widely available and can be used securely within healthcare settings to process data efficiently and at scale.

This workshop will provide a pragmatic hands-on introduction to using LLM APIs to extract clinical data from unstructured clinical text. It is designed for learners with all levels of technical skills. Instructors will introduce the fundamentals of working with LLMs and APIs, real-world use cases, basic coding, and key concepts of prompt engineering. Participants will gain hands-on experience creating and running workflows that transform unstructured notes into structured, actionable data using an annotated codebook that will walk them step-by-step through the following guided exercises: (1) load a dataset (2) pass a prompt into an LLM API (3) pass data into the LLM API (4) generate an output CSV file. The capstone of the session is a collaborative team challenge, where attendees will design and evaluate LLM queries to extract structured data from clinical text and compare outputs against gold standards.

Speaker(s):
Catherine Blebea, MD
University of California San Francisco

Sara Faghihi Kashani, MD, MPH
UCSF

Sristi Sharma, M.D., M.P.H.
UCSF

Parnaz Daneshpajouhnejad, MD
Stanford Medical Center

Author(s):
Catherine Blebea, MD - University of California San Francisco; Sara Faghihi Kashani, MD, MPH - UCSF; Sristi Sharma, M.D., M.P.H. - UCSF; Parnaz Daneshpajouhnejad, MD - Stanford Medical Center; Madhumita Sushil, PhD - UCSF; Anoop Muniyappa, MD, MS - UCSF; Julia Adler-Milstein, PhD, FACMI - UCSF School of Medicine; Raman Khanna, MD, MAS - University of California, San Francisco;

---