Solving the explainable AI conundrum by bridging clinicians’ needs and developers’ goals
OPEN ACCESS
Loading...
Author / Producer
Date
2023-05-22
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Explainable artificial intelligence (XAI) has emerged as a promising solution for addressing the implementation challenges of AI/ML in healthcare. However, little is known about how developers and clinicians interpret XAI and what conflicting goals and requirements they may have. This paper presents the findings of a longitudinal multi-method study involving 112 developers and clinicians co-designing an XAI solution for a clinical decision support system. Our study identifies three key differences between developer and clinician mental models of XAI, including opposing goals (model interpretability vs. clinical plausibility), different sources of truth (data vs. patient), and the role of exploring new vs. exploiting old knowledge. Based on our findings, we propose design solutions that can help address the XAI conundrum in healthcare, including the use of causal inference models, personalized explanations, and ambidexterity between exploration and exploitation mindsets. Our study highlights the importance of considering the perspectives of both developers and clinicians in the design of XAI systems and provides practical recommendations for improving the effectiveness and usability of XAI in healthcare.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
6 (1)
Pages / Article No.
94
Publisher
Macmillan Publishers
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03356 - Grote, Gudela / Grote, Gudela
Notes
Funding
187331 - From Tools to Teammates: Human-AI Teaming Success Factors in High-risk Industries (SNF)
Related publications and datasets
Is new version of: https://doi.org/10.3929/ethz-b-000587164