Learning Attention Propagation for Compositional Zero-Shot Learning
OPEN ACCESS
Loading...
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Compositional zero-shot learning aims to recognize unseen compositions of seen visual primitives of object classes and their states. While all primitives (states and objects) are observable during training in some combination, their complex interaction makes this task especially hard. For example, wet changes the visual appearance of a dog very differently from a bicycle. Furthermore, we argue that relationships between compositions go beyond shared states or objects. A cluttered office can contain a busy table; even though these compositions don’t share a state or object, the presence of a busy table can guide the presence of a cluttered office. We propose a novel method called Compositional Attention Propagated Embedding (CAPE) as a solution. The key intuition to our method is that a rich dependency structure exists between compositions arising from complex interactions of primitives in addition to other dependencies between compositions. CAPE learns to identify this structure and propagates knowledge between them to learn class embedding for all seen and unseen compositions. In the challenging generalized compositional zeroshot setting, we show that our method outperforms previous baselines to set a new state-of-the-art on three publicly available benchmarks.
Permanent link
Publication status
published
External links
Editor
Book title
2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Journal / series
Volume
Pages / Article No.
3817 - 3826
Publisher
IEEE
Event
23rd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03514 - Van Gool, Luc (emeritus) / Van Gool, Luc (emeritus)
Notes
Conferene lecture held on January 6, 2023.