Latency lags bandwidth

Summary form only given. As I review performance trends, I am struck by a consistent theme across many technologies over many years: bandwidth improves much more quickly than latency for four different technologies: disks, networks, memories and processors. A rule of thumb to quantify the imbalance is: bandwidth improves by more than the square of the improvement in latency. This paper lists a half-dozen performance milestones to document this observation, many reasons why it happens, a few ways to cope with it, and two small examples of how you might design systems differently if you kept this simple rule of thumb in mind.