Abstract Visual Reasoning Enabled by Language
METADATA ONLY
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
While artificial intelligence (AI) models have achieved human or even superhuman performance in many well-defined applications, they still struggle to show signs of broad and flexible intelligence. The Abstraction and Reasoning Corpus (ARC), a visual intelligence benchmark introduced by François Chollet, aims to assess how close AI systems are to human-like cognitive abilities. Most current approaches rely on carefully handcrafted domain-specific program searches to brute-force solutions for the tasks present in ARC. In this work, we propose a general learning-based framework for solving ARC. It is centered on transforming tasks from the vision to the language domain. This composition of language and vision allows for pre-trained models to be leveraged at each stage, enabling a shift from handcrafted priors towards the learned priors of the models. While not yet beating state-of-the-art models on ARC, we demonstrate the potential of our approach, for instance, by solving some ARC tasks that have not been solved previously.
Permanent link
Publication status
published
External links
Editor
Book title
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Journal / series
Volume
Pages / Article No.
2643 - 2647
Publisher
IEEE
Event
Workshop and Challenges for New Frontiers in Visual Language Reasoning: Compositionality, Prompts and Causality (NFVLR 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); FOS: Computer and information sciences
Organisational unit
03604 - Wattenhofer, Roger / Wattenhofer, Roger