This record has been edited as far as possible, missing data will be added when the version of record is issued.
Into the Wild: Real-World Testing for ML-Based ABR
METADATA ONLY
Loading...
Author / Producer
Date
2025
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Machine learning (ML)-based Adaptive Bitrate (ABR) algorithms often struggle to bridge the gap between simulation and reality. Their strong performance in synthetic environments frequently fails to generalize to real-world conditions. Researchers have therefore begun testing these algorithms over the Internet to incorporate real-world feedback into their design. In this paper, we show that since network conditions vary significantly across the globe, testing in individual real-world environments can suffer from the same generalization issues as lab-based testing. Existing testing platforms face (and might even be oblivious to) this limitation because they cover a small geographical region and rely on a narrow set of users affected by survivorship bias. As a result, their insights on an algorithm's performance generalize poorly to other deployments across the Internet, hindering the widespread adoption of ML-based ABR methods in practice.
To address this gap, we present ABR-Arena, a global testing platform that enables researchers to evaluate the performance of ABR algorithms across a diverse set of regions around the globe. As a result of its worldwide coverage, ABR-Arena can reveal the performance shortcomings of several state-of-the-art ML-based approaches. It is extensible and easy to deploy in additional locations. We will make ABR-Arena available to the community to support the development of new ML-based approaches and to facilitate meaningful improvements to existing algorithms.
Permanent link
Publication status
published
External links
Editor
Book title
PACMI '25: Proceedings of the 4th Workshop on Practical Adoption Challenges of ML for Systems
Journal / series
Volume
Pages / Article No.
105 - 109
Publisher
Association for Computing Machinery
Event
4th Workshop on Practical Adoption Challenges of ML for Systems (PACMI 2025)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Adaptive Bitrate Streaming; Machine Learning; Learning-based Control; Computer networks
Organisational unit
09477 - Vanbever, Laurent / Vanbever, Laurent