Custom virtual memory policies for an image reconstruction application

Some scientific applications have very large memory requirements, and are required to move data between primary and secondary storage during execution. The I/O between processor and disk can be done using standard file interfaces or virtual memory. The use of virtual memory, though simple and straight-forward, has been criticized because of poor performance. However some modern operating systems provide techniques to “customize” virtual memory to obtain better performance. The authors present results from an image reconstruction application where 3D structures of viruses are reconstructed from 2D electron microscope images. The platform is the NEC Cenju-3 parallel computer running the Cenju-3/DE, a Mach-based operating system. The virtual memory interface allows the user to specify custom page replacement and prefetching policies. For the largest data set, using custom virtual memory reduced the the data access time by a factor of 25 compared to unformatted Fortran I/O