Flying with instruments: characterizing the NCAR MSS-III workload

The NCAR Mass Storage System, MSS-III, generates 10 megabytes a day of transaction log, containing information about its workload. Traditional metrics, such as the average amount of data stored and retrieved per hour, are useful but omit information regarding temporality, locality, and burstiness. This information is critical to characterizing and understanding the MSS workload. NCAR has begun to use metrics usually applied to virtual memories, hardware caches, and network traffic to analyze the MSS-III transaction logs. Current MSS-III workload characterization falls into three broad categories: parametric statistics (for example, mean and variance for various file and data metrics), trace-driven analysis (for example, working set size), and trace-driven simulation (for example, compulsory and capacity cache miss ratios). Results from all of these methods are presented. Graphs of MSS-III transactions across a range of time scales show a self-similarity or “fractal burstiness", typical of network traffic. This suggests that measurements of self-similarity (for example, the Hurst parameter) may be useful. Also, the lack of normal distribution suggests that application of nonparametric statistics might be fruitful