Mixed Methods Assessment of the Influence of Demographics on Medical Advice of ChatGPT

Poster Number: P192
Presentation Time: 05:00 PM - 06:30 PM

Abstract Keywords: Diagnostic Systems, Diversity, Equity, Inclusion, Accessibility, and Health Equity, Machine Learning, Fairness and Elimination of Bias, Delivering Health Information and Knowledge to the Public
Primary Track: Applications
Programmatic Theme: Consumer Health Informatics

This study investigates the impact of demographic data on diagnostic consistency between ChatGPT and WebMD symptom checker. Analyzing 540 prompts with varied demographics, ChatGPT demonstrated 91% diagnostic match with WebMD. ChatGPT's urgent care recommendations and demographic tailoring were presented significantly more to 75-year-olds versus 25-year-olds (p<0.01), but were not statistically different among race/ethnicity and sex groups. Readability of ChatGPT's responses exceeded recommended levels, signaling potential health literacy implications.

Speaker(s):
Katerina Andreadis, MS
NYU Grossman School of Medicine

Author(s):
Katerina Andreadis, MS - NYU Grossman School of Medicine; Devon Newman, - - Brown University; Chelsea Twan, MS - NYU Grossman School of Medicine; Amelia Shunk, MMCi - NYU Grossman School of Medicine; Devin Mann, MD - NYU Grossman School of Medicine; Elizabeth Stevens, PhD, MPH - NYU Grossman School of Medicine;