Probing the Robustness of Independent Mechanism Analysis for Representation Learning
Metadata only
Date
2022-08Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
One aim of representation learning is to recover the original latent code that generated the data, a task which requires additional information or inductive biases. A recently proposed approach termed Independent Mechanism Analysis (IMA) postulates that each latent source should influence the observed mixtures independently, complementing standard nonlinear independent component analysis, and taking inspiration from the principle of independent causal mechanisms. While it was shown in theory and experiments that IMA helps recovering the true latents, the method's performance was so far only characterized when the modeling assumptions are exactly satisfied. Here, we test the method's robustness to violations of the underlying assumptions. We find that the benefits of IMA-based regularization for recovering the true sources extend to mixing functions with various degrees of violation of the IMA principle, while standard regularizers do not provide the same merits. Moreover, we show that unregularized maximum likelihood recovers mixing functions which systematically deviate from the IMA principle, and provide an argument elucidating the benefits of IMA-based regularization. Show more
Publication status
publishedExternal links
Book title
First Workshop on Causal Representation Learning, UAI 2022. Accepted PapersPublisher
OpenReviewEvent
Subject
Independent Mechanism Analysis; Causal Representation Learning; Blind Source Separation; Independent Causal MechanismsOrganisational unit
09664 - Schölkopf, Bernhard / Schölkopf, Bernhard
More
Show all metadata
ETH Bibliography
yes
Altmetrics