Gini in A Bottle: A Case Study of Pareto’s Principle in the Wild

This paper presents a case study of socio-economic disparities – human factors – having tremendous impact on the performance and behavior of a cloud-based software system. Failures to take such factors into account lead to serious design, implementation, and operational problems. A detailed empirical analysis of a commercial mobile network address book web application, serving over 8.3 million subscribers, was conducted for the joint purposes of building realistic subscriber behavior and data models and to explain certain performance characteristics and expectations. Extensive analysis of anonymized production data revealed that many aspects of usersʹ data and activity exhibited strongly-heavy-tailed characteristics, particularly characteristics affecting database performance and interactive request latencies, which could be ameliorated by traditional techniques such as caching or multi-threading. Several performance-critical aspects of usersʹ data were found to be well-described by the Log-Normal probability distribution, were heavily-skewed to the right, and exhibited Gini coefficients consistent with income inequalities in the Western world. The analytical model was translated into enhanced simulation and performance tooling, enabling more realistic performance and capacity testing of the product. Our deeper understanding also lead to changes in monitoring and system performance evaluation and quality-of-service parameters, statements of performance and capacity ratings, and architecture that would not otherwise have been justifiable.