From Retrieving Information to Reasoning with AI: Exploring Different Interaction Modalities to Support Human-AI Coordination in Clinical Decision-Making

AI in healthcare

Published: arXiv: 2601.22338v1

Authors

Behnam Rahdari Sameer Shaikh Jonathan H Chen Tobias Gerstenberg Shriti Raj

Abstract

LLMs are popular among clinicians for decision-support because of simple text-based interaction. However, their impact on clinicians' performance is ambiguous. Not knowing how clinicians use this new technology and how they compare it to traditional clinical decision-support systems (CDSS) restricts designing novel mechanisms that overcome existing tool limitations and enhance performance and experience. This qualitative study examines how clinicians (n=12) perceive different interaction modalities (text-based conversation with LLMs, interactive and static UI, and voice) for decision-support. In open-ended use of LLM-based tools, our participants took a tool-centric approach using them for information retrieval and confirmation with simple prompts instead of use as active deliberation partners that can handle complex questions. Critical engagement emerged with changes to the interaction setup. Engagement also differed with individual cognitive styles. Lastly, benefits and drawbacks of interaction with text, voice and traditional UIs for clinical decision-support show the lack of a one-size-fits-all interaction modality.

Paper Summary

Problem

Clinical decision-making is a complex task that involves diagnosing and treating patients. While large language models (LLMs) have shown promise in improving clinician performance, their impact on clinical decision-making is still unclear. Clinicians are not using these models as intended, and it's unclear how they compare to traditional clinical decision-support systems (CDSS). This lack of understanding restricts the design of new mechanisms that can overcome existing tool limitations and enhance performance and experience.

Key Innovation

This study explores how clinicians interact with LLMs in different ways, including text-based conversation, interactive user interfaces, and voice-based systems. The researchers used think-aloud case walkthroughs, interviews, and UI design probes to understand how clinicians interpret, verify, and incorporate AI output under realistic constraints. The study found that clinicians tend to use LLMs as a tool for targeted retrieval and confirmation, but that deeper engagement occurs when the interaction setup positions the model in a familiar consult role and when reasoning is externalized into stable visual artifacts.

Practical Impact

The findings of this study have practical implications for the design of clinical decision-support systems. By understanding how clinicians interact with LLMs, developers can design systems that support clinician-AI coordination and enhance performance and experience. The study suggests that different interaction modalities (text, visual, voice) are better suited for different tasks, and that a one-size-fits-all approach may not be effective. This knowledge can inform the development of more effective and user-friendly clinical decision-support systems.

Analogy / Intuitive Explanation

Imagine you're working with a colleague who has expertise in a particular area. You ask them a question, and they respond with a brief answer. However, if you ask them to explain their thought process and reasoning behind their answer, you may get a more detailed and insightful response. This is similar to how clinicians interact with LLMs in this study. They tend to use the models as a tool for targeted retrieval and confirmation, but when the interaction setup allows for deeper engagement and the model is positioned as a specialist, clinicians are more likely to engage with the model and benefit from its expertise.

Paper Information

Categories:

cs.HC cs.AI

Published Date:

arXiv ID:

2601.22338v1

Quick Actions

Back to Home