Graph-Augmented Transformer for Clinical Notes (GAT-CN): A Graph Neural Network Approach for Symptom Detection in Clinical Notes After Chemotherapy Initiation

Presentation Time: 02:00 PM - 02:12 PM

Abstract Keywords: Machine Learning, Deep Learning, Large Language Models (LLMs), Bioinformatics, Information Extraction, Fairness and elimination of bias
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics

This study introduces GAT-CN, a novel NLP approach that integrates transformer models with graph neural networks to identify post-chemotherapy symptoms associated with acute care utilization from clinical notes. This graph structure enables the model to learn patient node embeddings that encapsulate both the content of individual notes and the broader symptom context. GAT-CN outperforms state-of-the-art models, demonstrating its potential to augment existing transformer models and contributing to improved care for oncology patients.

Speaker:
Behzad Naderalvojoud, PhD
Stanford University

Authors:
Behzad Naderalvojoud, PhD - Stanford University; Elia Saquand, MS - ETH Zurich; Maximilian Schuessler, MD, MSc - Stanford University; Malvika Pillai, PhD - Stanford University & VA Palo Alto; Brian Travis Rice, MD - Stanford University School of Medicine; Arman Koul, MS - Stanford University School of Medicine; DOUGLAS W BLAYNEY, MD; Tina Hernandez-Boussard, PhD - Stanford University;

---

Beyond Traditional Expert Knowledge Based Keyword Lists: Enhancing Unprofessional Language Use Detection in Clinical Notes of Patients with Substance Use Disorders with LLM-Driven Rules

Presentation Time: 02:12 PM - 02:24 PM

Abstract Keywords: Nursing Informatics, Large Language Models (LLMs), Natural Language Processing
Working Group: Nursing Informatics Working Group
Primary Track: Applications
Programmatic Theme: Clinical Informatics

This study explores a hybrid approach combining LLM-driven vocabulary expansion with human validation to improve the detection of unprofessional language in clinical documentation for patients with substance use disorders. Among 102,228 discharge notes from MIMIC-IV, 61.6% contained unprofessional language. Integrating LLM-generated vocabulary improved model performance by 2.5% (F-score 0.91), compared to the rule-based approach alone, highlighting the value of a human-in-the-loop approach in refining LLM-generated terms for clinical use.

Speaker:
Jiyoun Song, PhD
University of Pennsylvania School of Nursing

Authors:
Sue Hyon Kim, MS, RN - University of Pennsylvania School of Nursing; Yoonjae Lee, DNP, APRN - University of Pennsylvania School of Nursing; Mollie Hobensack, PhD, RN - Icahn School of Medicine at Mount Sinai; Aviv Landau, PhD - University of Pennsylvania;

---

Detecting Emergencies in Patient Portal Messages Using Large Language Models and Knowledge Graph-based Retrieval-Augmented Generation

Presentation Time: 02:24 PM - 02:36 PM

Abstract Keywords: Large Language Models (LLMs), Artificial Intelligence, Natural Language Processing
Primary Track: Applications

This study developed and evaluated an approach using large language models (LLMs) integrated with a knowledge graph to triage emergency patient messages at Vanderbilt University Medical Center. Among four models, the RAG from Knowledge Graph with global search achieved the highest accuracy (0.99), sensitivity (0.98), and specificity (0.99) compared to other methods. Our findings suggest that integrating external knowledge improves emergency message triage. Future research should focus on expanding the knowledge graph and deploying the system in real clinical environments to evaluate its impact on patient outcomes.

Speaker:
Siru Liu, PhD
Vanderbilt University Medical Center

Authors:
Aileen Wright, MD - Vanderbilt; Allison McCoy, PhD, ACHIP, FACMI, FAMIA - Vanderbilt University Medical Center; Sean Huang, MD - Vanderbilt University; Bryan Steitz, PhD - Vanderbilt University Medical Center; Adam Wright, PhD - Vanderbilt University Medical Center;

---

Hybrid Retrieval and Reasoning to Predict Procedure Codes from Key Terms Using Knowledge Graphs and Large Language Models: A Formative Study

Presentation Time: 02:36 PM - 02:48 PM

Abstract Keywords: Knowledge Representation and Information Modeling, Large Language Models (LLMs), Controlled Terminologies, Ontologies, and Vocabularies, Information Retrieval, Information Extraction
Primary Track: Applications
Programmatic Theme: Clinical Informatics

We used four terms to predict procedure codes and explored the application of hybrid retrieval and reasoning techniques involving ChatGPT with reasoning, synonym-based and vector-based knowledge graph transversal. The semantics quality in a knowledge graph, the entry points in knowledge graph for the key terms, and the multi-hop transversals are all pivotal to the quality of retrieval from knowledge graph for large language models.

Speaker:
Zheng Milgrom, M.D., M.P.H.
Semedy Inc.

Author:
Roberto Rocha, MD, PhD, FACMI - Semedy Inc.;

---

Probabilistic Medical Predictions of Large Language Models

Presentation Time: 02:48 PM - 03:00 PM

Abstract Keywords: Large Language Models (LLMs), Natural Language Processing, Artificial Intelligence, Machine Learning, Clinical Decision Support, Clinical Guidelines
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics

Generating credible prediction probability is crucial in clinical practice and medical research when applying artificial intelligence (AI) to healthcare1. Large Language Models (LLMs) have demonstrated remarkable general-purpose capabilities and the ability to perform few-shot or zero-shot learning, enabling accurate predictions with little or no annotated data. However, probabilistic predictions of LLMs in healthcare have rarely been reported and evaluated. In this study, we extensively examined the reliability of the probabilistic output of LLMs by comparing the explicit probability and the implicit probability, where the explicit probability is defined as the probability that is obtained directly through text generation by LLMs, and the implicit probability is defined as the probability that corresponds to the prediction token. 6 open-sourced LLMs and 5 medical datasets were used. The evaluation metrics included the Area Under the Receiver Operating Characteristic (AUROC) and the Area Under the Precision Recall Curve (AUPRC). The contributions include a large-scale evaluation of the probabilistic medical predictions of LLMs and a framework for evaluating the reliability of probabilistic outputs of LLMs in medical predictions.

Speaker:
Jie Yang, PhD, FAMIA
Brigham and Women's Hospital/Harvard Medical School

Authors:
BOWEN GU, MS - Brigham and Women's Hospital; Rishi Desai - Brigham and Women's Hospital; Joshua Kueiyu Joshua, MD, MPH, SCD - Brigham and Women's Hospital; Jie Yang, PhD, FAMIA - Brigham and Women's Hospital/Harvard Medical School;

---

Machine Learning–Based Prediction of Antimicrobial Susceptibility: A Step Towards Precision Antimicrobial Stewardship

Presentation Time: 03:00 PM - 03:12 PM

Abstract Keywords: Machine Learning, Clinical Decision Support, Infectious Diseases and Epidemiology, Bioinformatics, Artificial Intelligence
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics

Antimicrobial resistance (AMR) represents an urgent global health crisis exacerbated by the frequent empirical use of broad-spectrum antibiotics. AMR is exacerbated by inherent delays in obtaining culture results and antimicrobial susceptibility data after sample collection. In this study, we developed and validated machine learning models using routinely collected electronic health records (EHR) data from inpatient and outpatient encounters to predict antibiotic resistance at the time of blood, urine or respiratory bacterial culture collection. The models demonstrated robust predictive accuracy, particularly in inpatient settings where clinical data was more consistently available. Notably, the model independently identified patterns that predict resistance, similar to how a clinician would attempt to predict resistance using prior culture and susceptibility data combined with their clinical training and knowledge of microbiological resistance patterns. Integrating these predictive tools into clinical workflows could significantly enhance empirical antibiotic selection, reduce unnecessary broad-spectrum antibiotic use, and meaningfully advance antimicrobial stewardship efforts.

Speaker:
Fatemeh Amrollahi, PhD
Stanford University

Authors:
Fatemeh Amrollahi, PhD - Stanford University; Fateme Nateghi Haredasht, PhD - Stanford University; Nicholas Marshall, MD - Stanford; Arin Vansomphone, BSC - Stanford; Manoj Maddali, MD - Stanford; Stephen Ma, MD, PhD - Stanford University School of Medicine; Amy Chang, MD, PharmD - Stanford University; Stanley Deresinsk, MD - Stanford; Mary Goldstein, MD, MS in HSR - Stanford University; Sanjat Kanjilal, MD MPH; Richard Medford, MD - ECU Health; Lauren Cooper, MS - NA; Steven Asch, MD - Stanford; Niaz Banaei, MD - Stanford; Jonathan Chen, MD, PhD - Stanford University Hospital;

---