Alignment of Machine Learning Risk Predictions with Clinical Reasoning: A Qualitative Study in Neurocritical Care

Presentation Type: Podium Abstract

Click to View Presentation

Presentation Time: 02:00 PM - 02:12 PM

Abstract Keywords: Clinical Decision Support, Artificial Intelligence, Human-computer Interaction, Critical Care, User-centered Design Methods, Machine Learning, Qualitative Methods, Usability
Programmatic Theme: Clinical Informatics

AI-based clinical decision support is often misaligned with clinician reasoning, limiting adoption. Through semi-structured interviews and case-based reasoning tasks with neurocritical care clinicians using a delayed cerebral ischemia ML risk score, we identified three barriers to cognitive alignment: clinicians’ need for contextualized and temporal anchoring, reliance on pattern recognition built from accumulated clinical exposure, and an iterative diagnostic therapeutic reasoning framework. We propose corresponding design directions for AI-CDS that support clinical reasoning in critical care.

Speaker(s):
Kayla Schiffer-Kane, MA
Columbia University
Author(s):
Tammam Alalqum, BA - Program for Hospital and Intensive Care Informatics, Department of Neurology, Columbia University Irving Medical Center; Soojin Park, MD - Columbia University Medical Center; Lena Mamykina, PhD - Columbia University;

---

Agentic AI enhances physician trust in clinical decision making

Presentation Type: Paper - Student

Click to View Presentation

Presentation Time: 02:12 PM - 02:24 PM

Abstract Keywords: Artificial Intelligence, Clinical Decision Support, Deep Learning, Evaluation, Large Language Models (LLMs)
Programmatic Theme: Clinical Informatics

Medical AI has shifted from reasoning to agentic AI, a new paradigm that autonomously invokes external tools during reasoning, rendering intermediate reasoning steps and tool outputs transparent to users. Although proven to outperform previous models, physician trust in agentic AI remains largely unexplored. To address this, three physicians evaluated 315 multimodal clinical cases quantifying both process-oriented cognitive trust and outcome-oriented behavioral reliance. Comparing agentic AI against non-agentic baselines, physicians exhibited significantly higher cognitive and behavioral trust for the agentic model (P<0.001). Specifically, on treatment planning tasks, physicians trusted the agentic reasoning most, preferring it in 89.57% of cases. Furthermore, process-oriented cognitive trust is significantly associated with outcome-oriented behavioral reliance (P < 0.001). However, measurable over-reliance on incorrect agentic outputs still exists, highlighting the inherent limitations of decision-logic transparency alone and underscoring the continuous need for rigorous clinician oversight.

Speaker(s):
Zhiling Yan, MS
Lehigh University
Author(s):
Zhiling Yan, MS - Lehigh University; Zhe Fang, MD, PhD - Harvard T.H. Chan School of Public Health; David J King, MD - University of Florida; Ann Pongsakul, MD - University of Texas Health Science Center at Houston; Eashan Adhikarla, MS - Lehigh University; Hui Ren, MD PhD MPH - MGH; Sunyang Fu, PhD, MHI - UTHealth Houston; Quanzheng Li, PhD - Massachusetts General Hospital and Harvard Medical School; Lifang He, PhD - Lehigh University; Xiang Li, PhD - Massachusetts General Hospital and Harvard Medical School; Hongfang Liu, PhD - University of Texas Health Science Center at Houston; Yonghui Wu, PhD - University of Florida; Lichao Sun, PhD - Lehigh University;

---

Can Large Language Models Support Multi-Objective Decision Making?

Presentation Type: Podium Abstract

Click to View Presentation

Presentation Time: 02:24 PM - 02:36 PM

Abstract Keywords: Large Language Models (LLMs), Policy, Public Health, Evaluation, Human-computer Interaction, Infectious Diseases & Epidemiology, Artificial Intelligence, Fairness and elimination of bias
Programmatic Theme: Public Health Informatics

Though AI has been entering government workflows, its ability to assist in complex decision-making remains untested. We evaluated whether DeepSeek, Gemini, Grok, and ChatGPT could guide pandemic policies balancing infection risks, hospital capacity, and economic impact. The models largely ignored these contextual factors when making recommendations, yet frequently cited them in their reasoning. Some models also quietly mirrored user preferences without disclosing this influence, creating the illusion of balanced reasoning.

Speaker(s):
Ritviksiddha Penchala, B.S.
University of California, San Francisco
Author(s):
Ritviksiddha Penchala, B.S. - University of California, San Francisco; Reza Yaesoubi, PhD - UCSF;

---

Clinician Perspectives of Automated Communication Feedback Tool for Assessing Social Signals in Telehealth Visits

Presentation Type: Paper - Student

Click to View Presentation

Presentation Time: 02:36 PM - 02:48 PM

Abstract Keywords: Usability, User-centered Design Methods, Artificial Intelligence, Information Visualization, Health Equity, Diversity, Equity, Inclusion, and Accessibility, Human-computer Interaction
Programmatic Theme: Clinical Informatics

Effective patient-provider communication can facilitate trust, patient-centered care, and improved care quality and outcomes. We developed a web-based tool called ConverSense that provides an automated assessment of patient-provider communication and provides clinicians with feedback on those conversations in the form of social signals (i.e., assertiveness, warmth, interactivity, and engagement). We conducted surveys and interviews with 24 clinicians after they interacted with standardized patients in simulated telehealth visits that were assessed by ConverSense. We also collected participant System Usability Scale, implementation outcomes for feasibility, acceptability, appropriateness, and bias awareness data to further evaluate perceptions of the utility and impact of the automated feedback. Based on interviews, we identified four qualitative themes that describe how participants view the automated feedback from ConverSense, the implementation of it into their practice, the ability for it to prompt participants to reflect on their patient conversations, and its potential for raising bias awareness.

Speaker(s):
Claire Lai, B.S.
Univeristy of Pennsylvania
Author(s):
Claire Lai, B.S. - Univeristy of Pennsylvania; Raina Langevin, PhD - University of Washington; Reggie Casanova-Perez, Biomedical and Health Informatics - University of Washington; Anuujin Tsedenbal, BS - University of California San Diego; Simran Saxena, MS - Microsoft; Manas Satish Bedmutha, BS; Serena Jinchen Xie, Masters - Biomedical Informatics and Medical Education, University of Washington; Wanda Pratt, PhD, FACMI - University of Washington; Janice Sabin - University of Washington; Brian Wood, MD - University of Washington; Nadir Weibel, PhD - UC San Diego; Andrea Hartzler, PhD FACMI - University of Washington;

---

Understanding Clinician Edits to Ambient AI Draft Notes: A Feasibility Analysis Using Large Language Models

Presentation Type: Paper - Student

Click to View Presentation

2026 Symposium 25x5 Presentation

Presentation Time: 02:48 PM - 03:00 PM

Abstract Keywords: Documentation Burden, Artificial Intelligence, Large Language Models (LLMs)
Programmatic Theme: Clinical Informatics

Ambient AI documentation tools generate draft notes that clinicians can review and edit before signing off in electronic health records. Scalable computational approaches to characterize how clinicians modify drafts remain limited, yet are essential for evaluating and improving AI effectiveness. We examined the feasibility of a few-shot prompted large language model (LLM) for categorizing sentence-level edits between AI drafts and final documentation. We developed five label-specific binary models targeting medication, symptom, diagnosis, orders/tests/procedures, and social history edits, and refined prompts using adversarial negatives and verification gates. Evaluation was performed against a human-annotated corpus. Medication and symptom models achieved promising performance (F1=0.787 and 0.780), whereas remaining models were precision-limited. Errors clustered in long, complex edits and category-boundary ambiguity. Therefore, prompt engineering is reliable for categorizing edits with explicit clues, while for complex context-dependent categories they are better suited for triage by labeling edits for human review.

Speaker(s):
Yiliang Zhou, PhD
University of California, Irvine
Author(s):
Yawen Guo, MISM - University of California - Irvine; Yiliang Zhou, MS - University of California, Irvine; Di Hu, Master of Science in Information Systems - University of California - Irvine; Sairam Sutari, MSHI - University of California, Irvine; Emilie Chow, MD - University of California, Irvine; Steven Tam, MD - UC Irvine; Danielle Perret, MD - The University of California Irvine; Deepti Pandita, MD, FACP, FAMIA - University of California Irvine; Kai Zheng, PhD - University of California, Irvine;

---

Error Provenance in AI Clinical Ambient Scribes: Separating ASR

Presentation Type: Podium Abstract

Click to View Presentation

2026 Symposium 25x5 Presentation

Presentation Time: 03:00 PM - 03:12 PM

Abstract Keywords: Artificial Intelligence, Documentation Burden, Evaluation
Programmatic Theme: Clinical Informatics

LLM-powered ambient scribes are being adopted at scale, yet errors cannot be attributed to
specific pipeline stages. We applied nine audio degradation conditions to a single 18-minute
pseudo-patient encounter and generated notes with four LLM tiers. Signal degradation and
connectivity dropout are qualitatively distinct failure modes: acoustic noise raises ASR word
error rate while connectivity loss removes entire content segments the LLM cannot reconstruct.
Model capability drives a larger omission gap than audio quality.

Speaker(s):
Brian Chapman, PhD
UT Southwestern Medical Center
Author(s):
Robin Higashi, PhD - UT Southwestern Medical Center; Justin Rousseau, MD, MMSc - University of Texas Southwestern Medical Center; Daniel Capurro, MD, PhD, FACMI, FAMIA - The University of Melbourne; Brian Chapman, PhD - UT Southwestern Medical Center;

---