Performance tradeoffs in cache design

A series of simulations that explore the interactions between various organizational decisions and program execution time are presented. The tradeoffs between cache size and CPU/cache cycle-time, set associativity and cycle time, and block size and main-memory speed, are investigated. The results indicate that neither cycle time nor cache size dominates the other across the entire design space. For common implementation technologies, performance is maximized when the size is increased to the size is increased to the 32-kB to 128-kB range with modest penalties to the cycle time. If set associativity impacts the cycle time by more than a few nanoseconds, it increases overall execution time. Since the block size and memory-transfer rate combine to affect the cache miss penalty, the optimum block size is substantially smaller than that which minimizes the miss rate. The interdependence between optimal cache configuration and the main memory speed necessitates multilevel cache hierarchies for high-performance uniprocessors