Navigating the Latent Spaces of Deep Neural Networks using Adversarial Techniques

Abstract

Deep learning has become a pervasive tool in the field of machine learning, delivering unprecedented advances in fields such as computer vision, natural language processing, and speech recognition. Deep neural networks, which comprise the class of models behind deep learning, transform their input signal into the desired output by means of sequential, layer-wise transformation of that data into successive latent representations. The careful study of these latent representations can open up novel ways to build better models. This work investigates the connection of latent spaces and adversarial techniques in deep learning. Adversarial techniques are a set of methods that deliberately craft input data perturbations to achieve the desired goal. The two most notable examples are adversarial examples, which are imperceptible data perturbations that force neural classifiers to output high-confidence errors, and generative adversarial networks (GANs), which comprise state-of-the-art generative models for images by arranging two neural networks in a minimax game. Specifically, our contributions are (1) a statistical detector for adversarial examples in neural classifiers, (2) a formal connection between adversarial training and spectral norm regularization, (3) an improved method for semantic interpolation in implicit models, and (4) a quantifiable metric for measuring the suitability of prior distributions in GANs. Our work is extendable in many ways and allows for the development of higher-performing, more robust, better understandable, and more easily trainable deep learning systems.