Exploiting data compression in collective I/O techniques

Author

Filgueira, Rosa ; Singh, David E. ; Pichel, Juan C. ; Carretero, Jesus

Author_Institution

Dept. of Comput. Sci., Univ. Carlos III of Madrid, Madrid

fYear

2008

fDate

Sept. 29 2008-Oct. 1 2008

Firstpage

479

Lastpage

485

Abstract

This paper presents Two-Phase Compressed I/O (TPC I/O,) an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to reduce network traffic, TPC I/O employs LZO algorithm to compress and decompress exchanged data in the inter-node communication operations. The compression algorithm has been fully implemented in the MPI collective technique, allowing to dynamically use (or not) compression. Compared with Two-Phase I/O, Two-Phase Compressed I/O obtains important improvements in the overall execution time for many of the considered scenarios.

Keywords

data compression; message passing; LZO algorithm; MPI collective technique; ROMIO; TPC I/O; collective I/O techniques; data compression; data exchange; inter-node communication operations; network traffic reduction; optimization; two-phase compressed I/O; Checkpointing; Compression algorithms; Computational modeling; Computer science; Concurrent computing; Cooperative caching; Data compression; File systems; Merging; Telecommunication traffic;

fLanguage

English

Publisher

ieee

Conference_Titel

Cluster Computing, 2008 IEEE International Conference on

Conference_Location

Tsukuba

ISSN

1552-5244

Print_ISBN

978-1-4244-2639-3

Electronic_ISBN

1552-5244

Type

conf

DOI

10.1109/CLUSTR.2008.4663811

Filename

4663811