Times are displayed in (UTC-07:00) Pacific Time (US & Canada) Change
5/21/2025 |
8:00 AM – 9:00 AM |
Avila A
S05: Transforming Clinical Documentation (ADD)
Presentation Type: Oral Presentations
Building Trust in AI Scribes: Consensus-Driven Metrics for Monitoring Their Impact in Academic Health Systems
2025 Clinical Informatics Conference 25x5 Presentation
Presentation Time: 08:00 AM - 08:15 AM
Abstract Keywords: Artificial Intelligence/Machine Learning, Ambient documentation, Usability and Measuring User Experience, Disruptive and Innovative Technologies
Primary Track: AI and Care Outcomes
Programmatic Theme: Clinical Decision Support and Analytics
Health systems urgently need processes to develop robust AI monitoring metrics that ensure safety, equity, and value for evolving AI deployments. We developed a streamlined, consensus-based methodology to address this need, piloting it for our AI scribe implementation. Measure concepts spanning fairness, robustness, workflow impact, value, and provider experience were identified via literature review and consensus, mapped to the UCSF IMPACC (Impact Monitoring Platform for AI in Clinical Care) AI Monitoring Metrics Framework, and refined through a modified Delphi process, resulting in 14 measure concepts.
Speaker(s):
Sara Murray, MD, MAS
UCSF
Author(s):
Jinoos Yazdany, MD MPH - UCSF; Hossein Soleimani, PhD - UCSF Health; Orianna DeMasi, PhD - UCSF; Rhiannon Croci, RN, BSN - UCSF; Robert Thombley, BS - UCSF; Aris Oates, MD - UCSF Health; Maria Byron, MD - University of California, San Francisco; cynthia fenton, MD - UCSF; Sarah Beck, MDiv - UCSF; Julia Adler-Milstein, PhD - UCSF School of Medicine; Sara Murray, MD, MAS - UCSF;
2025 Clinical Informatics Conference 25x5 Presentation
Presentation Time: 08:00 AM - 08:15 AM
Abstract Keywords: Artificial Intelligence/Machine Learning, Ambient documentation, Usability and Measuring User Experience, Disruptive and Innovative Technologies
Primary Track: AI and Care Outcomes
Programmatic Theme: Clinical Decision Support and Analytics
Health systems urgently need processes to develop robust AI monitoring metrics that ensure safety, equity, and value for evolving AI deployments. We developed a streamlined, consensus-based methodology to address this need, piloting it for our AI scribe implementation. Measure concepts spanning fairness, robustness, workflow impact, value, and provider experience were identified via literature review and consensus, mapped to the UCSF IMPACC (Impact Monitoring Platform for AI in Clinical Care) AI Monitoring Metrics Framework, and refined through a modified Delphi process, resulting in 14 measure concepts.
Speaker(s):
Sara Murray, MD, MAS
UCSF
Author(s):
Jinoos Yazdany, MD MPH - UCSF; Hossein Soleimani, PhD - UCSF Health; Orianna DeMasi, PhD - UCSF; Rhiannon Croci, RN, BSN - UCSF; Robert Thombley, BS - UCSF; Aris Oates, MD - UCSF Health; Maria Byron, MD - University of California, San Francisco; cynthia fenton, MD - UCSF; Sarah Beck, MDiv - UCSF; Julia Adler-Milstein, PhD - UCSF School of Medicine; Sara Murray, MD, MAS - UCSF;
Changes in Ambulatory Physician Burnout and EHR Time with Use of Artificial Intelligence Scribe Technology
2025 Clinical Informatics Conference 25x5 Presentation
Presentation Time: 08:15 AM - 08:30 AM
Abstract Keywords: Ambient documentation, Clinician Burnout, Data Science
Primary Track: Documentation Burden, Clinician Well-Being and Care Outcomes
Programmatic Theme: Usability, Efficiency, and User Experience
Time on the EHR, particularly after-hours, has been associated with burnout. While past efforts at enhancing documentation support through in-person and virtual human scribes have had beneficial effects on physician’s time expenditure and perceptions of documentation burden, these solutions were difficult to scale. Ambient documentation technology, also referred to as artificial intelligence (AI) scribe technology, drafts clinical notes from patient visits for physician review and editing. The University of California at San Francisco (UCSF) delivers care in more than 2 million visits annually to individuals in San Francisco and Northern California. UCSF made available AI scribe technology to an initial set of ambulatory physicians. Physicians were asked to complete a survey prior to initiating use of the AI scribe technology and post-adoption of the technology, including questions on perceptions of burnout. We also extracted data to measure additional outcomes on time spent in the EHR. We summarized demographics of the respondents and the distribution of visits for which respondents used AI scribes. Based on the latter, we identified three categories of AI utilization (low, medium, and high). After summarizing burnout scores pre and post AI scribe use, we assess changes in burnout scores after AI-scribe adoption for the overall cohort of physician respondents, by vendor, and by level of AI scribe usage. We additionally summarized the three EHR use measures pre and post AI-scribe use for the overall cohort of physician respondents and by vendor. Burnout scores significantly decreased after scribe usage in addition to some EHR time measures.
Speaker(s):
Lisa Rotenstein, MD, MBA, MSc
UCSF
Author(s):
Orianna DeMasi, Phd - UCSF; Hossein Soleimani, PhD - UCSF Health; Sarah Beck, MDiv - UCSF; Bei Cao, MS-HAIL - UCSF; Asia Stephens, BS - UCSF; Robert Thombley, BA - UCSF; Sara Murray, MD, MAS - UCSF; Maria Byron, MD - University of California, San Francisco; Julia Adler-Milstein, PhD - UCSF School of Medicine; Inga Lennes, MD MBA MPH - UCSF; Lisa Rotenstein, MD, MBA, MSc - UCSF;
2025 Clinical Informatics Conference 25x5 Presentation
Presentation Time: 08:15 AM - 08:30 AM
Abstract Keywords: Ambient documentation, Clinician Burnout, Data Science
Primary Track: Documentation Burden, Clinician Well-Being and Care Outcomes
Programmatic Theme: Usability, Efficiency, and User Experience
Time on the EHR, particularly after-hours, has been associated with burnout. While past efforts at enhancing documentation support through in-person and virtual human scribes have had beneficial effects on physician’s time expenditure and perceptions of documentation burden, these solutions were difficult to scale. Ambient documentation technology, also referred to as artificial intelligence (AI) scribe technology, drafts clinical notes from patient visits for physician review and editing. The University of California at San Francisco (UCSF) delivers care in more than 2 million visits annually to individuals in San Francisco and Northern California. UCSF made available AI scribe technology to an initial set of ambulatory physicians. Physicians were asked to complete a survey prior to initiating use of the AI scribe technology and post-adoption of the technology, including questions on perceptions of burnout. We also extracted data to measure additional outcomes on time spent in the EHR. We summarized demographics of the respondents and the distribution of visits for which respondents used AI scribes. Based on the latter, we identified three categories of AI utilization (low, medium, and high). After summarizing burnout scores pre and post AI scribe use, we assess changes in burnout scores after AI-scribe adoption for the overall cohort of physician respondents, by vendor, and by level of AI scribe usage. We additionally summarized the three EHR use measures pre and post AI-scribe use for the overall cohort of physician respondents and by vendor. Burnout scores significantly decreased after scribe usage in addition to some EHR time measures.
Speaker(s):
Lisa Rotenstein, MD, MBA, MSc
UCSF
Author(s):
Orianna DeMasi, Phd - UCSF; Hossein Soleimani, PhD - UCSF Health; Sarah Beck, MDiv - UCSF; Bei Cao, MS-HAIL - UCSF; Asia Stephens, BS - UCSF; Robert Thombley, BA - UCSF; Sara Murray, MD, MAS - UCSF; Maria Byron, MD - University of California, San Francisco; Julia Adler-Milstein, PhD - UCSF School of Medicine; Inga Lennes, MD MBA MPH - UCSF; Lisa Rotenstein, MD, MBA, MSc - UCSF;
Integrating Social and Environmental Determinants of Health for Substance Use Disorder in Veterans: A Comprehensive Repository and Analytical Framework
2025 Clinical Informatics Conference On Demand
2025 Clinical Informatics Conference DEI/Health Equity Presentation
Presentation Time: 08:30 AM - 08:45 AM
Abstract Keywords: Social Determinants of Health, Environmental Health and the Exposome, Big Data
Working Group: Clinical Decision Support Working Group
Primary Track: Innovation in Driving Digital Equity and Social Care
Programmatic Theme: Informatics-Driven Value-Based Healthcare
The Department of Veterans Affairs (VA), in collaboration with Oak Ridge National Laboratory (ORNL), is developing a repository of Social and Environmental Determinants of Health (SEDH) to address health disparities and substance use disorder (SUD) among veterans. Aligned with the National Drug Control Strategy, this initiative includes datasets and indices like the Social Capital and Environmental Health Indices, aimed at analyzing SEDH impacts on health. Despite advancements, standardized, veteran-specific SEDH data gaps remain. This repository integrates geographic, socioeconomic, and longitudinal data tailored for veterans, supporting SUD trend analysis, social isolation studies, and regional climate impacts on health. The repository, serving as a foundational resource for policy and intervention, currently holds 56 social and 3 environmental datasets, emphasizing neighborhood and healthcare determinants. These datasets have been utilized to explore different aspects of veteran health, and future work will expand data coverage on food security and education, enhancing policy-driven, proactive veteran care and closing critical gaps in veteran health research.
Speaker(s):
Hilda Klasky, FAMIA
Oak Ridge National Laboratory (ORNL) - UT Battelle
Author(s):
Hilda Klasky, FAMIA - Oak Ridge National Laboratory (ORNL) - UT Battelle; Kevin Sparks, MS - Oak Ridge National Laboratory; Alina Peluso, PhD - Oak Ridge National Laboratory; Josh Grant, MS - Oak Ridge National Laboratory; Adam Spannaus, PhD - Oak Ridge National Laboratory; Joseph Tuccillo, MS - Oak Ridge National Laboratory; Heidi Hanson, PhD - Oak Ridge National Laboratory; Susana Martins, MD - Veterans Affairs, Palo Alto Health Care System; Jodie Trafton, PhD - Veterans Affairs, Palo Alto Health Care System; Anuj Kapadia, PhD - Oak Ridge National Laboratory;
2025 Clinical Informatics Conference On Demand
2025 Clinical Informatics Conference DEI/Health Equity Presentation
Presentation Time: 08:30 AM - 08:45 AM
Abstract Keywords: Social Determinants of Health, Environmental Health and the Exposome, Big Data
Working Group: Clinical Decision Support Working Group
Primary Track: Innovation in Driving Digital Equity and Social Care
Programmatic Theme: Informatics-Driven Value-Based Healthcare
The Department of Veterans Affairs (VA), in collaboration with Oak Ridge National Laboratory (ORNL), is developing a repository of Social and Environmental Determinants of Health (SEDH) to address health disparities and substance use disorder (SUD) among veterans. Aligned with the National Drug Control Strategy, this initiative includes datasets and indices like the Social Capital and Environmental Health Indices, aimed at analyzing SEDH impacts on health. Despite advancements, standardized, veteran-specific SEDH data gaps remain. This repository integrates geographic, socioeconomic, and longitudinal data tailored for veterans, supporting SUD trend analysis, social isolation studies, and regional climate impacts on health. The repository, serving as a foundational resource for policy and intervention, currently holds 56 social and 3 environmental datasets, emphasizing neighborhood and healthcare determinants. These datasets have been utilized to explore different aspects of veteran health, and future work will expand data coverage on food security and education, enhancing policy-driven, proactive veteran care and closing critical gaps in veteran health research.
Speaker(s):
Hilda Klasky, FAMIA
Oak Ridge National Laboratory (ORNL) - UT Battelle
Author(s):
Hilda Klasky, FAMIA - Oak Ridge National Laboratory (ORNL) - UT Battelle; Kevin Sparks, MS - Oak Ridge National Laboratory; Alina Peluso, PhD - Oak Ridge National Laboratory; Josh Grant, MS - Oak Ridge National Laboratory; Adam Spannaus, PhD - Oak Ridge National Laboratory; Joseph Tuccillo, MS - Oak Ridge National Laboratory; Heidi Hanson, PhD - Oak Ridge National Laboratory; Susana Martins, MD - Veterans Affairs, Palo Alto Health Care System; Jodie Trafton, PhD - Veterans Affairs, Palo Alto Health Care System; Anuj Kapadia, PhD - Oak Ridge National Laboratory;
Generating SOAP Clinical Notes using Structured Clinical Decision Support and Large Language Models
Presentation Time: 08:45 AM - 09:00 AM
Abstract Keywords: Adaptive Clinical Decision Support, Artificial Intelligence/Machine Learning, Health IT Standards (USCDI, FHIR®, SMART, etc.), Documentation Burden, Disruptive and Innovative Technologies, Interoperability, Clinical Content and IT Project Governance, Clinical informatics organizational models
Primary Track: Industry and Commercial Partnership
Programmatic Theme: Clinical Decision Support and Analytics
Expanding and evolving clinical knowledge, patients with complex chronic conditions, ineffective health IT technology, and increasing administrative burden is impacting the delivery of evidence-based and personalized patient care. We combine structured FHIR resources, clinical decision support logic, and reasoning with a large language model (LLM) to generate a grounded clinical SOAP (subjective, objective, assessment, and plan) note. We evaluate the approach over varied clinical and patient scenarios through multiple LLM agents using a standardized instrument.
Speaker(s):
Maulik Kamdar, PhD
Optum
Author(s):
Maulik Kamdar, PhD - Optum; Aldo Córdova-Palomera, PhD - Optum AI; Vijay Nori, PhD - Optum AI; Joao Nogueira, PhD - Optum AI; Jeffrey Danford, MS - Optum; Steve Ross, MD - Optum Insight; Dominic King, MD PhD - Optum Insight; Eran Halperin, PhD - Optum AI; Kevin Larsen, MD - Optum;
Presentation Time: 08:45 AM - 09:00 AM
Abstract Keywords: Adaptive Clinical Decision Support, Artificial Intelligence/Machine Learning, Health IT Standards (USCDI, FHIR®, SMART, etc.), Documentation Burden, Disruptive and Innovative Technologies, Interoperability, Clinical Content and IT Project Governance, Clinical informatics organizational models
Primary Track: Industry and Commercial Partnership
Programmatic Theme: Clinical Decision Support and Analytics
Expanding and evolving clinical knowledge, patients with complex chronic conditions, ineffective health IT technology, and increasing administrative burden is impacting the delivery of evidence-based and personalized patient care. We combine structured FHIR resources, clinical decision support logic, and reasoning with a large language model (LLM) to generate a grounded clinical SOAP (subjective, objective, assessment, and plan) note. We evaluate the approach over varied clinical and patient scenarios through multiple LLM agents using a standardized instrument.
Speaker(s):
Maulik Kamdar, PhD
Optum
Author(s):
Maulik Kamdar, PhD - Optum; Aldo Córdova-Palomera, PhD - Optum AI; Vijay Nori, PhD - Optum AI; Joao Nogueira, PhD - Optum AI; Jeffrey Danford, MS - Optum; Steve Ross, MD - Optum Insight; Dominic King, MD PhD - Optum Insight; Eran Halperin, PhD - Optum AI; Kevin Larsen, MD - Optum;