Do It For Me vs. Do It With Me: Investigating User Perceptions of Different Paradigms of Automation in Copilots for Feature-Rich Software
METADATA ONLY
Loading...
Author / Producer
Date
2025
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Large Language Model (LLM)-based in-application assistants, or copilots, can automate software tasks, but users often prefer learning by doing, raising questions about the optimal level of automation for an effective user experience. We investigated two automation paradigms by designing and implementing a fully automated copilot (AutoCopilot) and a semi-automated copilot (GuidedCopilot) that automates trivial steps while offering step-by-step visual guidance. In a user study (N=20) across data analysis and visual design tasks, GuidedCopilot outperformed AutoCopilot in user control, software utility, and learnability, especially for exploratory and creative tasks, while AutoCopilot saved time for simpler visual tasks. A follow-up design exploration (N=10) enhanced GuidedCopilot with task-and state-aware features, including in-context preview clips and adaptive instructions. Our findings highlight the critical role of user control and tailored guidance in designing the next generation of copilots that enhance productivity, support diverse skill levels, and foster deeper software engagement.
Permanent link
Publication status
published
External links
Book title
CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
Journal / series
Volume
Pages / Article No.
880
Publisher
Association for Computing Machinery
Event
ACM Conference on Human Factors in Computing Systems (CHI 2025)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Feature-rich software; Large language models; Software copilots; User control; Semi-automation; Human-AI collaboration
Organisational unit
09820 - Wang, April Yi / Wang, April Yi