The Role of Representation Learning in Advancing Deep Learning: Efficiency, Scalability, and Reasoning
OPEN ACCESS
Loading...
Author / Producer
Date
2025
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
This thesis explores the multifaceted role of representation learning in improving the performance of deep neural networks in various applications. We delve into the paradigm of active learning, investigating its application in iterative combinatorial auctions and the integration of diversity and uncertainty-based sampling strategies in the context of self-supervised, pre-trained models. Our exploration extends to disentangled representation learning, where we introduce a novel training procedure for variational autoencoders that overcomes the challenge of hyperparameter selection and enables consistent learning of disentangled representations. We also leverage implicit neural representations for domain-agnostic super-resolution, demonstrating the ability to upscale any arbitrary data type. In addition, we address the challenge of scaling transformers to large inputs by proposing a hierarchical tree-based architecture. Finally, we investigate the reasoning capabilities of large language models. We demonstrate the feasibility of using language for reasoning in abstract visual tasks.
We then introduce a benchmark for evaluating algorithmic reasoning and analyze the scaling behavior of reasoning language models on complex logic puzzles. Through these diverse investigations, this thesis contributes to a deeper understanding of representation learning and its potential to advance the development of more robust, efficient, and intelligent AI systems.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Wattenhofer, Roger
Examiner : Martius, Georg
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Representation learning; Reasoning; Active learning
Organisational unit
03604 - Wattenhofer, Roger / Wattenhofer, Roger