Derm2Vec: Image-Text Alignment with Improved Dermatology Diagnosis Classification Robustness and Explainability

Poster Number: P58
Presentation Time: 05:00 PM - 06:30 PM

Abstract Keywords: Natural Language Processing, Deep Learning, Diagnostic Systems
Working Group: Natural Language Processing Working Group
Primary Track: Applications
Programmatic Theme: Clinical Informatics

Skin diseases are among the most common reasons for medical consultations. Accurate automatic skin disease classification can help reduce workload on healthcare professionals, and improve patient outcomes. This study aims to develop a scalable method to address the heterogeneity, and improve the interpretability of skin disease classification. Our contributions include (1) a dermatology-centric, static, medical concept embedding (Derm2Vec), which is capable of clustering related dermatological concepts. (2)demonstrating a new method for dermatology image classification that utilizes vector similarities between image and static diagnosis embeddings, with improved +1.3% accuracy through pretraining on a different dataset. (3) showing the potential for better diagnosis interpretability through zero-shot symptom prediction.

Speaker(s):
Yujuan Fu, BSE
University of Washington

Author(s):
Yujuan Fu, BSE - University of Washington; Zhaoyi Sun, Master of Science - University of Washington; Wen-Wai Yim - Augmedix; Meliha Yetisgen, PhD - University of Washington;