Improving Emergency Department Visit Risk Prediction: Exploring the Operational Utility of Applied Patient Portal Messages

Presentation Time: 11:00 AM - 11:15 AM

Abstract Keywords: Large Language Models (LLMs), Natural Language Processing, Clinical Decision Support, Patient / Person Generated Health Data (Patient Reported Outcomes), Mobile Health, Population Health
Primary Track: Applications
Programmatic Theme: Clinical Informatics

Patient portal messages represent a unique source of clinical data due to how they represent the voice of the patient, provide a glimpse into care delivery between episodic synchronous appointments, and capture variations in patient behavior and health literacy. There is little understanding of how to best apply modern natural language processing (NLP) approaches, such as large, pre-trained language models (LLMs), to patient messages. In this study, we aim to explore different approaches in incorporating patient messages into an existing ED visit risk prediction model currently deployed at Stanford Health Care. With the addition of patient message frequencies to the baseline we were able to achieve an improved AUC of .83 and a significant jump in the F1 score. In future work, we aim to build upon these findings and further test combination models to incorporate features around patient message content, in addition to message frequencies.

Speaker(s):
Hanna Kiani, MHA
Stanford Health Care

Author(s):
Sohaib Hassan, BA Genetics - Stanford University; Sohaib Hassan, PhD Biomedical Data Science (in-training) - Stanford Department of Biomedical Data Science;