Large Scale Speedtest Experimentation in Mobile Broadband Networks

Paper: Large Scale “Speedtest” Experimentation in Mobile Broadband Networks

Abstract: Characterizing and evaluating the performance of Mobile Broadband (MBB) networks is a vital need for today’s societies.
Testbed-based measurements are of great significance in this context, since they allow for controlled and longitudinal experimentation. In this work, we focus on “speed” as an important Quality of Service (QoS) indicator for MBB networks, and work with MONROE-Nettest, an open source speedtest tool running as an Experiment as a Service (EaaS) on the Measuring Mobile Broadband Networks in Europe (MONROE) testbed. We conduct an extensive longitudinal measurement campaign spanning 2 countries over 2 years, and provide our experiment results together with rich metadata as an open dataset.
We characterize this open dataset in detail, as well as derive insights from it regarding the impact of network context, spatio-temporal effects, roaming, and mobility on network performance. We describe our experiences about conducting speedtest measurements in MBB, and discuss the challenges associated with large scale testbed experimentation in operational MBB networks. Tackling one of the said challenges further, we introduce the notion of adaptive speedtest duration, and leverage a Machine Learning (ML) based algorithm to provide a proof-of-concept implementation called “Speedtest++”. Finally, we describe the lessons we have learned, as well as provide an overall discussion of how open datasets can support MBB research, and comment on open challenges, in the hope that these can serve as discussion points for future work.

Open Dataset: MONROE-Nettest Base Experiments

This dataset includes a part of the raw data from MONROE-Nettest base experiments between 01.01.2018 and 31.12.2019, along with experiment metadata from the MONROE platform. There are 2 types of files in the dataset:

• Summary results per day: monroe-nettest-<yyyy-mm-dd>.CSV files combine the configuration parameters, reported performance metrics (DLdata rate, UL data rate, RTT), and corresponding metadata (Received Signal Strength Indicator (RSSI), Reference SignalReceived Power (RSRP), Reference Signal Received Qual-ity (RSRQ), mobile technology, and mobility indicator) for each MONROE-Nettest measurement within a given day day, and include one line per measurement.
• Detailed results per measurement: <measurement-id>.JSON files per each MONROE-Nettest measurement provide all the information pertaining to themeasurement in the corresponding daily aggregate file speci-fied above, as well as detailed time series for all phases (e.g., timestamped records of downloaded bytes per chunk in the downlink test phase). One JSON file is generated for each measurement, and files are sorted into daily folders.

Overall, the dataset contains over 200K MONROE-Nettest measurements from 6 SIM and network operators, with an average of 7218 measurements per month. 

Download sample (anonymised CSV)

Source Code: MONROE-Nettest

Source Code: Speedtest++