Using think-aloud protocol to identify cognitive events to generate data-driven scientific hypotheses by inexperienced clinical researchers

Presentation Time: 04:30 PM - 04:45 PM

Abstract Keywords: Human-computer Interaction, Information Visualization, Controlled Terminologies, Ontologies, and Vocabularies
Primary Track: Foundations
Programmatic Theme: Clinical Research Informatics

We conducted a data-driven hypothesis generation study with clinical researchers using VIADS (a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies) or other analytical tools (as control, e.g., SPSS, SAS, R). The participants analyzed the same datasets and developed hypotheses using a think-aloud verbal protocol. Their screen activities and audio were recorded, transcribed, and coded for cognitive events. We analyzed the recordings to identify the cognitive events (e.g., “Analyze data,” “Seek connection”) during hypothesis generation. The VIADS group exhibited the lowest mean number of cognitive events per hypothesis with the smallest standard deviation. The highest percentages of cognitive events in hypothesis generation were “Using analysis results” (30%) and “Seeking connections” (23%). The results suggest that VIADS may guide participants better than the control group. The results provide evidence to explain the shorter average time needed by the VIADS group to generate each hypothesis.

Speaker(s):
Xia Jing, MD, PhD
Clemson University

Author(s):
Xia Jing, MD, PhD - Clemson University; Brooke Draghi, BS - Clemson University; Mytchell Ernst, BS - Clemson University; Vimla Patel, PhD - New York Academy of Medicine; James Cimino, MD, FACMI, FACP, FAMIA - Department of Biomedical Informatics and Data Science, Heersink School of Medicine, University of Alabama at Birmingham; Jay Shubrook, DO - Touro University; Yuchun Zhou, PhD - Ohio University; Chang Liu, PhD - Ohio University; Sonsoles De Lacalle, MD, PhD - California State University Channel Islands;