Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
METADATA ONLY
Loading...
Author / Producer
Date
2024-03
Publication Type
Journal Article
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. A problem with standard CZSL is the assumption of knowing which unseen compositions will be available at test time. In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions. To address this problem, we propose a new approach, Compositional Cosine Graph Embedding (Co-CGE), based on two principles. First, Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network. The graph propagates information from seen to unseen concepts, improving their representations. Second, since not all unseen compositions are equally feasible, and less feasible ones may damage the learned representations, Co-CGE estimates a feasibility score for each unseen composition, using the scores as margins in a cosine similarity-based loss and as weights in the adjacency matrix of the graphs. Experiments show that our approach achieves state-of-the-art performances in standard CZSL while outperforming previous methods in the open world scenario.
Permanent link
Publication status
published
External links
Editor
Book title
Volume
46 (3)
Pages / Article No.
1545 - 1560
Publisher
IEEE
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Compositional zero-shot learning; Graph neural networks; Open-World recognition; Scene understanding
Organisational unit
03514 - Van Gool, Luc (emeritus) / Van Gool, Luc (emeritus)
09664 - Schölkopf, Bernhard / Schölkopf, Bernhard