On the quest for new KPIs in mobile networks: The impact of throughput fluctuations on QoE

Downlink throughput is the most widely used and accepted Quality of Service (QoS) related feature within the networking community, specially in the operational field. Current quality monitoring and reporting systems as well as quality benchmarking campaigns use the Average Downlink Throughput (ADT) as metric to assess the performance of the network. For example, flow-based monitoring systems normally report the per flow ADT as a throughput-based Key Performance Indicator (KPI), which is then aggregated at different temporal (e.g., 15 minutes), geographical (e.g., per radio cell) and/or logical (e.g., per service) scales to reflect the health of the network. A similar direction is currently followed within the Quality of Experience (QoE) research domain, where ADT is translated into a measure of user satisfaction for bandwidth-sensitive services (e.g., video streaming, file sharing, etc.). We claim that the ADT is not always an accurate KPI in terms of QoE for bandwidth-sensitive services, and present results showing that the variation of downlink throughput can actually have a major impact on the perceived quality of the end user. In this paper we present a complete study of the QoE undergone by 52 mobile users in controlled subjective lab tests, using different mobile applications such as YouTube, Facebook and Gmaps. By shaping the traffic of the users through multiple bandwidth fluctuation patterns, we conclude that novel downlink-throughput related KPIs must be defined for QoE-based traffic analysis in mobile networks. Based on this observation, we propose some very first simple throughput fluctuation models to define such KPIs.