MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer

Open access
Date
2018Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
We introduce MIDI-VAE, a neural network model basedon Variational Autoencoders that is capable of handlingpolyphonic music with multiple instrument tracks, as wellas modeling the dynamics of music by incorporating notedurations and velocities. We show that MIDI-VAE can per-form style transfer on symbolic music by automaticallychanging pitches, dynamics and instruments of a musicpiece from, e.g., a Classical to a Jazz style. We evaluate the efficacy of the style transfer by training separatestyle validation classifiers. Our model can also interpolatebetween short pieces of music, produce medleys and cre-ate mixtures of entire songs. The interpolations smoothlychange pitches, dynamics and instrumentation to create aharmonic bridge between two music pieces. To the best ofour knowledge, this work represents the first successful at-tempt at applying neural style transfer to complete musicalcompositions. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000292318Publication status
publishedBook title
Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR 2018)Pages / Article No.
Publisher
dblpEvent
Organisational unit
03604 - Wattenhofer, Roger / Wattenhofer, Roger
More
Show all metadata
ETH Bibliography
yes
Altmetrics