Evaluation Of Diagnosis And Triage Performance Of Ada, Webmd Symptom Checkers, And ChatGPT On Patients Presenting For Urgent Primary Care

Poster Number: P127
Presentation Time: 05:00 PM - 06:30 PM

Abstract Keywords: Evaluation, Diagnostic Systems, Large Language Models (LLMs), Mobile Health, Personal Health Informatics, Governance of Artificial Intelligence
Primary Track: Applications
Programmatic Theme: Consumer Health Informatics

Diagnostic and triage output from symptom checkers (SCs), as well as Large Language Models (LLMs) like ChatGPT, may influence patient care decisions. Little research exists on evaluation for these system using real patient cases. In this study, we examine WebMD, Ada Health, and ChatGPT versions 3.5 and 4.0 diagnostic and triage accuracy on patients visiting a primary care clinic.

Speaker(s):
Daven Crossland, B.A.
Brown University

Author(s):
Hamish Fraser, MBChB, MRCP, MSc - Brown University; Chloe Kim, B.A. - Brown University; Caroline Bailey, BS - Brown University; Kenon Graham, BA - Brown University; Ross Hilliard, MD, FACP - MaineHealth; Ian Bacher - Brown University; Drew Nagle, MD - Brown Medicine; Megan Ranney, MD, MPH - Dept. of Emergency Medicine, Alpert Medical School, Brown Univ.; Tracy Madsen, M.D., Ph.D - Brown University; Daven Crossland, B.A. - Brown University;