Mobile Systems and Analytics
Paper: Large Scale “Speedtest” Experimentation in Mobile Broadband Networks
Abstract: Characterizing and evaluating the performance of Mobile Broadband (MBB) networks is a vital need for today’s societies.
Testbed-based measurements are of great significance in this context, since they allow for controlled and longitudinal experimentation. In this work, we focus on “speed” as an important Quality of Service (QoS) indicator for MBB networks, and work with MONROE-Nettest, an open source speedtest tool running as an Experiment as a Service (EaaS) on the Measuring Mobile Broadband Networks in Europe (MONROE) testbed. We conduct an extensive longitudinal measurement campaign spanning 2 countries over 2 years, and provide our experiment results together with rich metadata as an open dataset.
We characterize this open dataset in detail, as well as derive insights from it regarding the impact of network context, spatio-temporal effects, roaming, and mobility on network performance. We describe our experiences about conducting speedtest measurements in MBB, and discuss the challenges associated with large scale testbed experimentation in operational MBB networks. Tackling one of the said challenges further, we introduce the notion of adaptive speedtest duration, and leverage a Machine Learning (ML) based algorithm to provide a proof-of-concept implementation called “Speedtest++”. Finally, we describe the lessons we have learned, as well as provide an overall discussion of how open datasets can support MBB research, and comment on open challenges, in the hope that these can serve as discussion points for future work.
Open Dataset: MONROE-Nettest Base Experiments
This dataset includes a part of the raw data from MONROE-Nettest base experiments between 01.01.2018 and 31.12.2019, along with experiment metadata from the MONROE platform. There are 2 types of files in the dataset:
Overall, the dataset contains over 200K MONROE-Nettest measurements from 6 SIM and network operators, with an average of 7218 measurements per month.