Implementation Tradeoffs of the Array Files Library for Out-of-Core Computations

The array files (AF) API facilitates the manipulation of a large number of records in a message-passing cluster application. Three AF implementations are studied. AFv1 and AFv2 assume the existence of an underlying parallel file system and can be used by PVM or MPI message passing programs. Testing over NSF, PVFS, and Lustre were performed. AFv1 uses separate files on the parallel file system to store different records. AFv2 uses a large file on the parallel file system to store all the records. File locking is used to enforce POSIX file access semantics, and AFv1 and AFv2 provide an application-level locking service for parallel file systems that do not support a native lock subsystem. AFv3 is implemented using MPI-2 and can be used by MPI-2 applications. AFv3 implements parallel I/O with an object-based striping strategy, has no central server, uses a hash function to locate an I/O server, and provides POSIX file access semantics to the user application. AFv2 over PVFS shows somewhat better performance for some workloads with very large records. AFv3 is shown to have better performance than both AFv1 and AFv2 for the broadest range of workloads