Times are displayed in (UTC-04:00) Eastern Time (US & Canada) Change
3/10/2025 |
8:30 AM – 12:00 PM |
Urban
W01: Collaborative Workshop: A Datathon for Exploring Potential Biases in Medical Artificial Intelligence
Presentation Type: Workshop
Session Credits: 3
Collaborative Workshop: A Datathon for Exploring Potential Biases in Medical Artificial Intelligence
Presentation Time: 08:30 AM - 12:00 PM
Abstract Keywords: Data Quality, Machine Learning, Generative AI, and Predictive Modeling, Ethical, Legal, and Social Issues
Primary Track: Data Science/Artificial Intelligence
Programmatic Theme: Health Data Science and Artificial Intelligence Innovation: From Single-Center to Multi-Site
The potential adverse impacts of biases in clinical machine learning/artificial intelligence systems are increasingly well understood, but strategies for minimizing risks may not be obvious to developers and data scientists. Analyses of the impacts of classification algorithms across racial groups has led to discussion and changes in clinical recommendations. However, varying definitions of fairness, associated metrics, techniques for ameliorating bias, and strategies for reporting model details demonstrate the complexities involved in minimizing risk. Datathons aimed at bringing diverse teams together to explore challenging problems with medical data have been shown to be effective tools for educating both clinicians and practitioners about the limits of data and computing approaches. This full-day datathon will introduce participants to approaches for thinking about potential biases and challenge them to explore biases in a published open-source clinical prediction model. The workshop will begin with an overview of the difficulties associated with bias in machine learning models, an introduction of some possible metrics used to assess biases, and an introduction to frameworks for examining potential biases, including the (PROBAST) checklist. Participants will be introduced to open-source critical care risk models including eGOSSIS, a novel severity score designed to be generalizable across healthcare contexts. Participants will be divided into teams and challenged to explore and identify possible biases in a subset of the data used to train these models, including the eICU-CRD subset of the eGOSSIS dataset. The workshop will conclude with a presentation of results from the participating teams.
Speaker(s):
Leo Celi, MD
MIT/Harvard Medical School
Harry Hochheiser, PhD
University of Pittsburgh Department of Biomedical Informatics
Shyam Visweswaran, MD PhD
University of Pittsburgh
Christopher Horvat, MD, MHA
UPMC Children's Hospital of Pittsburgh
Presentation Time: 08:30 AM - 12:00 PM
Abstract Keywords: Data Quality, Machine Learning, Generative AI, and Predictive Modeling, Ethical, Legal, and Social Issues
Primary Track: Data Science/Artificial Intelligence
Programmatic Theme: Health Data Science and Artificial Intelligence Innovation: From Single-Center to Multi-Site
The potential adverse impacts of biases in clinical machine learning/artificial intelligence systems are increasingly well understood, but strategies for minimizing risks may not be obvious to developers and data scientists. Analyses of the impacts of classification algorithms across racial groups has led to discussion and changes in clinical recommendations. However, varying definitions of fairness, associated metrics, techniques for ameliorating bias, and strategies for reporting model details demonstrate the complexities involved in minimizing risk. Datathons aimed at bringing diverse teams together to explore challenging problems with medical data have been shown to be effective tools for educating both clinicians and practitioners about the limits of data and computing approaches. This full-day datathon will introduce participants to approaches for thinking about potential biases and challenge them to explore biases in a published open-source clinical prediction model. The workshop will begin with an overview of the difficulties associated with bias in machine learning models, an introduction of some possible metrics used to assess biases, and an introduction to frameworks for examining potential biases, including the (PROBAST) checklist. Participants will be introduced to open-source critical care risk models including eGOSSIS, a novel severity score designed to be generalizable across healthcare contexts. Participants will be divided into teams and challenged to explore and identify possible biases in a subset of the data used to train these models, including the eICU-CRD subset of the eGOSSIS dataset. The workshop will conclude with a presentation of results from the participating teams.
Speaker(s):
Leo Celi, MD
MIT/Harvard Medical School
Harry Hochheiser, PhD
University of Pittsburgh Department of Biomedical Informatics
Shyam Visweswaran, MD PhD
University of Pittsburgh
Christopher Horvat, MD, MHA
UPMC Children's Hospital of Pittsburgh