Automating Procedure Billing Using A ChatGPT Decision Tree Query Architecture

Poster Number: P39
Presentation Time: 05:00 PM - 06:30 PM

Abstract Keywords: Documentation Burden, Large Language Models (LLMs), Surgery, Workflow, Standards
Primary Track: Applications
Programmatic Theme: Clinical Informatics

This study assesses ChatGPT's ability to autonomously assign CPT codes to hand surgery operative notes, using a direct comparison between a single-query and a refined two-step query model. Analyzing 150 cases, the two-step model showed improved accuracy (74.3%) over the single-query model (52.0%). The two-step model demonstrates the potential for decision tree architectures for expanding and enhancing model performance. Further refinement is needed for precise code differentiation, particularly in complex cases like distal radius fractures.

Speaker(s):
Patrick Donohue, BS
Washington University in Saint Louis

Author(s):
Patrick Donohue, BS - Washington University in Saint Louis; Emilie Zoldos, MD - Washington University in Saint Louis; Christopher Dy, MD, MPH - Washington University in Saint Louis; Adam Wilcox, PhD - Washington University in St. Louis;