Beyond the Hype: Building a Risk-Grounded Framework for Safe Generative AI - A Practical Workshop (Workshop)

Presentation Type: Workshop
Not recorded for AMIA Now

Presentation Time: 08:00 AM - 10:00 AM

Abstract Keywords: Generative AI in Clinical Workflow: Ambient Listening, Chart Summarization, Automated Response with LLM, Workforce Automation, Communication, and Workflow Efficiency, Clinician Well-Being, Analytical Artificial Intelligence: ML, Digital Pathology, Imaging AI, Predictive Analytics, Governance
Primary Track: Driving Change at Scale through Effective Leadership and Governance


A surge of generative artificial intelligence (AI) tools is being evaluated for use in medicine, yet the burdens of rigorous evaluations often fall on health systems. Thorough and pragmatic local evaluation remains a major challenge. Unlike predictive models, generative AI tools often lack objective gold standards for comparison, and tool performance varies based on local data and workflows. Thus, health systems must frequently evaluate new AI tools using resource-intensive approaches, without a universally agreed upon evaluation framework to guide this work.

Developing a pragmatic initial evaluation framework grounded in model risk and local context can guide the selection of appropriate evaluative approaches to support feasible, safe, effective, and equitable implementation. Such a framework could help institutions determine when to perform model validation and/or in-workflow assessments, as well as how to combine human review, automated metrics, and user feedback to balance rigor and feasibility.

This workshop will draw on the diverse expertise of AMIA professionals in research, operations, and industry to expand our draft framework. We will:
1. Review evaluation strategies from the literature.
2. Share a conceptual map for understanding evaluation strategies and a draft framework grounded in real-world experiences at two institutions to guide pragmatic selection of appropriate evaluation methods.
3. Collaboratively refine the draft framework with attendee use cases and experiences

This session will be especially relevant for healthcare IT professionals evaluating AI solutions, industry partners building safe and equitable products, policymakers developing evaluation requirements, and researchers creating the next generation of AI tools.

Speaker(s):
Rohith Palli, MD, PhD
University of Washington

Emily Larimer, MD
University of Washington

Jason Lau, MD, MS
University of Washington

Yu-Hsiang Lin, MD
Seattle Children's Hospital

Andrew White, MD
University of Washington

Author(s):
Rohith Palli, MD, PhD - University of Washington; Emily Larimer, MD - University of Washington; Jason Lau, MD, MS - University of Washington; Yu-Hsiang Lin, MD - Seattle Children's Hospital; Andrew White, MD - University of Washington;

---