Web-conscious storage management for Web proxies

Many proxy servers are limited by their file I/O needs. Even when a proxy is configured with sufficient I/O hardware, the file system software often fails to provide the available bandwidth to the proxy processes. Although specialized file systems may offer a significant improvement and overcome these limitations, we believe that user-level disk management on top of industry-standard file systems can offer similar performance advantages. We study the overheads associated with file I/O in Web proxies, we investigate their underlying causes, and we propose Web-conscious storage management, a set of techniques that exploit the unique reference characteristics of Web-page accesses in order to allow Web proxies to overcome file I/O limitations. Using realistic trace-driven simulations, we show that these techniques can improve the proxy´s secondary storage I/O throughput by a factor of 15 over traditional open-source proxies, enabling a single disk to serve over 400 (URL-get) operations per second. We implement Foxy, a Web proxy which incorporates our techniques. Experimental evaluation suggests that Foxy outperforms traditional proxies, such as SQUID, by more than a factor of four in throughput, without sacrificing response latency.