Advancing Predictions for Video Streaming with Transformers

Abstract

The Transformer is a sequence-to-sequence (seq2seq) neural network architecture that has proven itself useful for a wide variety of applications. We compare the Transformers performance to several baseline models on a video streaming task to predict transmission times. At the moment the models used for this task are not optimized for seq2seq predictions. To address this we use the Transformer which is tailored to efficiently find long-term dependencies in the data. We find that the Transformer does not significantly outperform the other models. We suspect a lack of long-term dependencies in our dataset or the lack of essential features to find those dependencies. Nevertheless the Transformer shows better performance than the other models for the tail loss. A transformer variant using probabilistic regression is able to marginally outperform the other models in the mean loss. Additionally we describe the adaptations we made to the Transformer to make it compatible with multi horizon timeseries prediction tasks.