Morpho-Mnist: Quantitative assessment and diagnostics for representation learning
- Journal Article
Rights / licenseCreative Commons Attribution 4.0 International
Revealing latent structure in data is an active field of research, having introduced excitingtechnologies such as variational autoencoders and adversarial networks, and is essentialto push machine learning towards unsupervised knowledge discovery. However, a majorchallenge is the lack of suitable benchmarks for an objective and quantitative evaluation oflearned representations. To address this issue we introduce Morpho-MNIST, a frameworkthat aims to answer: “to what extent has my model learned to represent specific factors ofvariation in the data?” We extend the popular MNIST dataset by adding a morphometricanalysis enabling quantitative comparison of trained models, identification of the rolesof latent variables, and characterisation of sample diversity. We further propose a setof quantifiable perturbations to assess the performance of unsupervised and supervisedmethods on challenging tasks such as outlier detection and domain adaptation. Data andcode are available athttps://github.com/dccastro/Morpho-MNIST. Show more
Journal / seriesJournal of Machine Learning Research
Pages / Article No.
Subjectrepresentation learning; generative models; empirical evaluation; disentanglement; morphometrics
MoreShow all metadata