Title :
Improving HDFS performance using local caching system
Author :
Sang-Deok Yoon ; In-Yang Jung ; Ki-Hyun Kim ; Chang-Sung Jeong
Author_Institution :
Dept. of Electr. Eng., Korea Univ., Seoul, South Korea
Abstract :
I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.
Keywords :
cache storage; distributed databases; file organisation; network operating systems; public domain software; HDFS performance; Hadoop distributed file system performance; I/O performance; cache algorithm; content aware application performance; distributed parallel clusters; local caching system; local file system; Computer architecture; Educational institutions; File systems; Google; Pipelines; Servers; Writing; HDFS; cache; distributed file system; hadoop; locality;
Conference_Titel :
Future Generation Communication Technology (FGCT), 2013 Second International Conference on
Conference_Location :
London
Print_ISBN :
978-1-4799-2974-0
DOI :
10.1109/FGCT.2013.6767200
