Evaluating stream buffers as a secondary cache replacement

Today´s commodity microprocessors require a low latency memory system to achieve high sustained performance. The conventional high-performance memory system provides fast data access via a large secondary cache. But large secondary caches can be expensive, particularly in large-scale parallel systems with many processors (and thus many caches). The authors evaluate a memory system design that can be both cost-effective as well as provide better performance, particularly for scientific workloads: a single level of (on-chip) cache backed up only by Jouppi´s stream buffers and a main memory. This memory system requires very little hardware compared to a large secondary cache and doesn´t require modifications to commodity processors. The authors use trace-driven simulation of fifteen scientific applications from the NAS and PERFECT suites in their evaluation. They present two techniques to enhance the effectiveness of Jouppi´s original stream buffers: filtering schemes to reduce their memory bandwidth requirement and a scheme that enables stream buffers to prefetch data being accessed in large strides. The results show that, for the majority of the benchmarks, stream buffers can attain hit rates that are comparable to typical hit rates of secondary caches. Also, the authors find that as the data-set size of the scientific workload increases the performance of streams typically improves relative to secondary cache performance, showing that streams are more scalable to large data-set sizes