- Home
- 2025 Annual Symposium Gallery
- Explainable Suicide Phenotyping from Initial Psychiatric Evaluation Notes Using Reasoning Large Language Models
Custom CSS
double-click to edit, do not edit in source
11/17/2025 |
3:30 PM – 4:45 PM |
Room 8
S50: Unmasking the Shadows: Informatics Innovations in Suicide & Violence Prevention
Presentation Type: Oral Presentations
Identification of temporal condition patterns associated with suicide from claims data using sequence pattern mining
Presentation Time: 03:30 PM - 03:42 PM
Abstract Keywords: Population Health, Data Mining, Real-World Evidence Generation, Public Health
Primary Track: Applications
Programmatic Theme: Public Health Informatics
Understanding how suicide risk unfolds over time remains a significant challenge in suicide research. We applied sequence pattern mining and stratified analysis to Maryland claims data to identify temporal clinical sequences preceding suicide death. High-risk patterns varied by demographic subgroup and included psychiatric conditions, chronic pain, and neoplasm-related conditions. We developed a sequence catalog with network-based visualization to support interpretation and inform targeted screening and prevention strategies.
Speaker:
Anas Belouali, MEng, MS
Johns Hopkins
Authors:
Christopher Kitchen, MS - Johns Hopkins University; Emily Haroz, PhD - Johns Hopkins Center for Indigenous Health; Harold Lehmann, MD, PhD - Johns Hopkins University; Paul Nestadt, MD - Department of Psychiatry, Johns Hopkins School of Medicine, Baltimore, MD, USA.; Holly C Wilcox, PhD - Department of Mental Health, Johns Hopkins School of Public Health, Baltimore, MD, USA; Hadi Kharrazi, MD, PhD, FAMIA, FACMI - Johns Hopkins University;
Presentation Time: 03:30 PM - 03:42 PM
Abstract Keywords: Population Health, Data Mining, Real-World Evidence Generation, Public Health
Primary Track: Applications
Programmatic Theme: Public Health Informatics
Understanding how suicide risk unfolds over time remains a significant challenge in suicide research. We applied sequence pattern mining and stratified analysis to Maryland claims data to identify temporal clinical sequences preceding suicide death. High-risk patterns varied by demographic subgroup and included psychiatric conditions, chronic pain, and neoplasm-related conditions. We developed a sequence catalog with network-based visualization to support interpretation and inform targeted screening and prevention strategies.
Speaker:
Anas Belouali, MEng, MS
Johns Hopkins
Authors:
Christopher Kitchen, MS - Johns Hopkins University; Emily Haroz, PhD - Johns Hopkins Center for Indigenous Health; Harold Lehmann, MD, PhD - Johns Hopkins University; Paul Nestadt, MD - Department of Psychiatry, Johns Hopkins School of Medicine, Baltimore, MD, USA.; Holly C Wilcox, PhD - Department of Mental Health, Johns Hopkins School of Public Health, Baltimore, MD, USA; Hadi Kharrazi, MD, PhD, FAMIA, FACMI - Johns Hopkins University;
Anas
Belouali,
MEng, MS - Johns Hopkins
Every Kid Counts: Closing gaps in data about youth suicidality with non-traditional sources
Presentation Time: 03:42 PM - 03:54 PM
Abstract Keywords: Informatics Implementation, Data Mining, Pediatrics, Public Health
Primary Track: Foundations
Programmatic Theme: Clinical Research Informatics
Youth suicide is a growing public health challenge. We explore non-traditional, first responder electronic health records (EHRs) to identify suicide-related incidents that might not be captured in hospital EHRs. We find suicide-related incidents are not always coded, are often embedded in notes, and first responder EHRs capture more information than hospital EHRs alone. Spatial analysis suggests higher frequency in historically under-represented areas. Our findings support research using integrated EHRs for public health research and reporting.
Speaker:
Natalie Kane, PhD
Authors:
Mark Hoffman, PhD - Children's Mercy Kansas City; Natalie Kane, PhD;
Presentation Time: 03:42 PM - 03:54 PM
Abstract Keywords: Informatics Implementation, Data Mining, Pediatrics, Public Health
Primary Track: Foundations
Programmatic Theme: Clinical Research Informatics
Youth suicide is a growing public health challenge. We explore non-traditional, first responder electronic health records (EHRs) to identify suicide-related incidents that might not be captured in hospital EHRs. We find suicide-related incidents are not always coded, are often embedded in notes, and first responder EHRs capture more information than hospital EHRs alone. Spatial analysis suggests higher frequency in historically under-represented areas. Our findings support research using integrated EHRs for public health research and reporting.
Speaker:
Natalie Kane, PhD
Authors:
Mark Hoffman, PhD - Children's Mercy Kansas City; Natalie Kane, PhD;
Natalie
Kane,
PhD -
Explainable Suicide Phenotyping from Initial Psychiatric Evaluation Notes Using Reasoning Large Language Models
Presentation Time: 03:54 PM - 04:06 PM
Abstract Keywords: Artificial Intelligence, Large Language Models (LLMs), Natural Language Processing
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics
Clinical phenotyping is the process of extracting patient's observable symptoms and traits to better understand their disease condition. Suicide phenotyping focuses more on behavioral and cognitive characteristics, such as suicide ideation, attempt, and self-injury, to identify suicide risks and improve interventions. In this study, we leveraged the latest reasoning models, namely 4o, o1, and o3-mini, to perform note-level multi-label classification and reasoning generation tasks using previously annotated psychiatric evaluation notes from a safety-net psychiatric inpatient hospital in Harris County, Texas. Compared with the previously finetuned GPT-3.5 model, the out-of-box reasoning models prompted with in-context learning achieved comparable and better performance, with the highest accuracy of 0.94 and F1 of 0.90. We implemented novel clinical justification generation from these models on the traditional classification tasks. This finding marked a promising direction for performing clinical phenotyping that is interpretable and actionable using smaller, efficient reasoning models.
Speaker:
Zehan (Leo) Li, PhD
The Univeristy of Texas Health Science Center at Houston (UTHealth) School of Biomedical Informatics
Authors:
Wanjing Wang, M.S - UTHealth; Lokesh Shahani, M.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Rodrigo M Vieira, M.D, Ph.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Salih Selek, M.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Jair Soares, M.D, Ph.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Hongfang Liu, PhD - University of Texas Health Science Center at Houston; Ming Huang, PhD - UTHealth Houston;
Presentation Time: 03:54 PM - 04:06 PM
Abstract Keywords: Artificial Intelligence, Large Language Models (LLMs), Natural Language Processing
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics
Clinical phenotyping is the process of extracting patient's observable symptoms and traits to better understand their disease condition. Suicide phenotyping focuses more on behavioral and cognitive characteristics, such as suicide ideation, attempt, and self-injury, to identify suicide risks and improve interventions. In this study, we leveraged the latest reasoning models, namely 4o, o1, and o3-mini, to perform note-level multi-label classification and reasoning generation tasks using previously annotated psychiatric evaluation notes from a safety-net psychiatric inpatient hospital in Harris County, Texas. Compared with the previously finetuned GPT-3.5 model, the out-of-box reasoning models prompted with in-context learning achieved comparable and better performance, with the highest accuracy of 0.94 and F1 of 0.90. We implemented novel clinical justification generation from these models on the traditional classification tasks. This finding marked a promising direction for performing clinical phenotyping that is interpretable and actionable using smaller, efficient reasoning models.
Speaker:
Zehan (Leo) Li, PhD
The Univeristy of Texas Health Science Center at Houston (UTHealth) School of Biomedical Informatics
Authors:
Wanjing Wang, M.S - UTHealth; Lokesh Shahani, M.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Rodrigo M Vieira, M.D, Ph.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Salih Selek, M.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Jair Soares, M.D, Ph.D - Faillace Department of Psychiatry & Behavioral Sciences, UTHealth; Hongfang Liu, PhD - University of Texas Health Science Center at Houston; Ming Huang, PhD - UTHealth Houston;
Zehan (Leo)
Li,
PhD - The Univeristy of Texas Health Science Center at Houston (UTHealth) School of Biomedical Informatics
Detection of Youth Suicide Interventions in Clinical Record Text using an Open-Source Language Model
Presentation Time: 04:06 PM - 04:18 PM
Abstract Keywords: Patient Safety, Information Extraction, Large Language Models (LLMs), Healthcare Quality
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics
This study presents an automated approach to detect youth suicide prevention interventions documented in clinical notes. Expert review classified four interventions across 1,794 notes from 200 emergency visits for suicidality among children aged 6—17. The open-source language model Llama3.3-70B generated Likert scores (-3 to +3) for intervention presence. Scores approximated human classification: Lethal Means Restriction (AUROC 0.972, 95%CI:0.961–0.982), Hotline (AUROC 0.980, 95%CI:0.966–0.990), Outpatient Referral (AUROC 0.935, 95%CI:0.922–0.947), Safety Planning (AUROC 0.954, 95%CI:0.930–0.975). Application to 6,687 notes from 723 encounters revealed increased odds of omitted interventions among youth with prior visits (OR 0.454-0.625), missing screening questions (OR 0.169-0.653), and ideation (vs. acts or attempts) (OR 0.296, 95%CI 0.133-0.657). Findings demonstrate systematic clinical text analysis with an open-source language model can expose new targets to inform decision support and strengthen evidence-based care for youth suicide.
Speaker:
Juliet Edgcomb, MD PhD
David Geffen School of Medicine - UCLA
Authors:
Juliet Edgcomb, MD PhD - David Geffen School of Medicine - UCLA; Juliet Edgcomb, MD PhD - UCLA; Alexandra Klomhaus, PhD - UCLA; Joshua Lee, BS - UCLA; Chrislie Ponce, BA BS - UCLA; Elyse Tascione, MA - UCLA; Angshuman Saha, PhD - UCLA;
Presentation Time: 04:06 PM - 04:18 PM
Abstract Keywords: Patient Safety, Information Extraction, Large Language Models (LLMs), Healthcare Quality
Primary Track: Applications
Programmatic Theme: Clinical Research Informatics
This study presents an automated approach to detect youth suicide prevention interventions documented in clinical notes. Expert review classified four interventions across 1,794 notes from 200 emergency visits for suicidality among children aged 6—17. The open-source language model Llama3.3-70B generated Likert scores (-3 to +3) for intervention presence. Scores approximated human classification: Lethal Means Restriction (AUROC 0.972, 95%CI:0.961–0.982), Hotline (AUROC 0.980, 95%CI:0.966–0.990), Outpatient Referral (AUROC 0.935, 95%CI:0.922–0.947), Safety Planning (AUROC 0.954, 95%CI:0.930–0.975). Application to 6,687 notes from 723 encounters revealed increased odds of omitted interventions among youth with prior visits (OR 0.454-0.625), missing screening questions (OR 0.169-0.653), and ideation (vs. acts or attempts) (OR 0.296, 95%CI 0.133-0.657). Findings demonstrate systematic clinical text analysis with an open-source language model can expose new targets to inform decision support and strengthen evidence-based care for youth suicide.
Speaker:
Juliet Edgcomb, MD PhD
David Geffen School of Medicine - UCLA
Authors:
Juliet Edgcomb, MD PhD - David Geffen School of Medicine - UCLA; Juliet Edgcomb, MD PhD - UCLA; Alexandra Klomhaus, PhD - UCLA; Joshua Lee, BS - UCLA; Chrislie Ponce, BA BS - UCLA; Elyse Tascione, MA - UCLA; Angshuman Saha, PhD - UCLA;
Juliet
Edgcomb,
MD PhD - David Geffen School of Medicine - UCLA
Prospective Validation of a Suicide Event Risk Model in Transgender Patients
Presentation Time: 04:18 PM - 04:30 PM
Abstract Keywords: Evaluation, Diversity, Equity, Inclusion, and Accessibility, Fairness and elimination of bias
Primary Track: Applications
Programmatic Theme: Clinical Informatics
Suicide risk prediction models offer a promising avenue for early intervention, but their effectiveness in
underrepresented populations remain uncertain. This study evaluated VSAIL’s, a real-world, externally validated,
and deployed suicide risk prediction model, performance in predicting suicide risk among transgender individuals.
Transgender individuals were identified from electronic health record data and transgender status was verified via
manual chart review. Results indicated modest discriminative ability (AUROC=0.777, AUPRC=0.115), however, a
high rate of false negatives (77%), and significant miscalibration (Brier=0.023, Spiegelhalter’s z-statistic p<0.001)
reduced clinical utility. Findings underscore the importance of targeted subgroup validation and highlight limitations
of general population-trained models in accurately identifying suicide risk among transgender patients. They also
suggest the need for ongoing algorithm monitoring and subgroup-aware modeling strategies to improve predictive
equity in marginalized populations.
Speaker:
Alex Becker, MS
Department of Biomedical Informatics, Vanderbilt University
Authors:
Alex Becker, MS - Department of Biomedical Informatics, Vanderbilt University; Colin Walsh, MD MA - Department of Biomedical Informatics, Vanderbilt University;
Presentation Time: 04:18 PM - 04:30 PM
Abstract Keywords: Evaluation, Diversity, Equity, Inclusion, and Accessibility, Fairness and elimination of bias
Primary Track: Applications
Programmatic Theme: Clinical Informatics
Suicide risk prediction models offer a promising avenue for early intervention, but their effectiveness in
underrepresented populations remain uncertain. This study evaluated VSAIL’s, a real-world, externally validated,
and deployed suicide risk prediction model, performance in predicting suicide risk among transgender individuals.
Transgender individuals were identified from electronic health record data and transgender status was verified via
manual chart review. Results indicated modest discriminative ability (AUROC=0.777, AUPRC=0.115), however, a
high rate of false negatives (77%), and significant miscalibration (Brier=0.023, Spiegelhalter’s z-statistic p<0.001)
reduced clinical utility. Findings underscore the importance of targeted subgroup validation and highlight limitations
of general population-trained models in accurately identifying suicide risk among transgender patients. They also
suggest the need for ongoing algorithm monitoring and subgroup-aware modeling strategies to improve predictive
equity in marginalized populations.
Speaker:
Alex Becker, MS
Department of Biomedical Informatics, Vanderbilt University
Authors:
Alex Becker, MS - Department of Biomedical Informatics, Vanderbilt University; Colin Walsh, MD MA - Department of Biomedical Informatics, Vanderbilt University;
Alex
Becker,
MS - Department of Biomedical Informatics, Vanderbilt University
Intimate Partner Homicide Among Women of Childbearing Age: Identifying Multilevel Risk Factors with Machine Learning
Presentation Time: 04:30 PM - 04:42 PM
Abstract Keywords: Health Equity and Social Determinants of Health (SDoH), Machine Learning, Population Health, Geospatial (GIS) Data/Analysis
Primary Track: Applications
Programmatic Theme: Public Health Informatics
Intimate partner homicide (IPH) remains a major yet understudied cause of maternal mortality among U.S. women of childbearing age (WCBA). We leveraged the National Violent Death Reporting System (NVDRS) and county-level Maternal Vulnerability Index (MVI) data from 2018–2022 to train three machine learning models—logistic regression, random forest, and XGBoost—to classify whether homicides were IPH-related. Among 11,973 homicides involving WCBA, 36% were IPH. XGBoost achieved the best performance (F1-score=0.83, sensitivity = 0.89, AUPRC = 0.86), prompting further examination of key predictors via model explainability. Results indicated that acute interpersonal conflicts (e.g., arguments, jealousy), prior IPV victimization, and structural vulnerabilities (e.g., reproductive healthcare access, physical environment) were influential predictors of IPH. By illustrating the interplay of individual, interpersonal, and broader community-level risk factors, our study shows how machine learning can inform multilevel strategies to prevent IPH and improve maternal health.
Speaker:
Snigdha Peddireddy, MPH
Emory University
Authors:
Shifan Yan, BS - Emory University; Sangmi Kim, PhD, MPH, RN - Nell Hodgson Woodruff School of Nursing, Emory University; Ran Xiao, PhD - Emory University;
Presentation Time: 04:30 PM - 04:42 PM
Abstract Keywords: Health Equity and Social Determinants of Health (SDoH), Machine Learning, Population Health, Geospatial (GIS) Data/Analysis
Primary Track: Applications
Programmatic Theme: Public Health Informatics
Intimate partner homicide (IPH) remains a major yet understudied cause of maternal mortality among U.S. women of childbearing age (WCBA). We leveraged the National Violent Death Reporting System (NVDRS) and county-level Maternal Vulnerability Index (MVI) data from 2018–2022 to train three machine learning models—logistic regression, random forest, and XGBoost—to classify whether homicides were IPH-related. Among 11,973 homicides involving WCBA, 36% were IPH. XGBoost achieved the best performance (F1-score=0.83, sensitivity = 0.89, AUPRC = 0.86), prompting further examination of key predictors via model explainability. Results indicated that acute interpersonal conflicts (e.g., arguments, jealousy), prior IPV victimization, and structural vulnerabilities (e.g., reproductive healthcare access, physical environment) were influential predictors of IPH. By illustrating the interplay of individual, interpersonal, and broader community-level risk factors, our study shows how machine learning can inform multilevel strategies to prevent IPH and improve maternal health.
Speaker:
Snigdha Peddireddy, MPH
Emory University
Authors:
Shifan Yan, BS - Emory University; Sangmi Kim, PhD, MPH, RN - Nell Hodgson Woodruff School of Nursing, Emory University; Ran Xiao, PhD - Emory University;
Snigdha
Peddireddy,
MPH - Emory University
Explainable Suicide Phenotyping from Initial Psychiatric Evaluation Notes Using Reasoning Large Language Models
Category
Paper - Student
Description
Custom CSS
double-click to edit, do not edit in source
11/17/2025 04:45 PM (Eastern Time (US & Canada))