loading...
RFS: efficient and flexible remote file access for MPI-IO
San Diego, CA, USA September 20-September 23
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/CLUSTR.2004.1392604Sixth IEEE International Conference o ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Jonghyun Lee, Math. & Comput. Sci. Div., Argonne Nat. Lab., IL, USA
R. Ross, Math. & Comput. Sci. Div., Argonne Nat. Lab., IL, USA
R. Thakur, Math. & Comput. Sci. Div., Argonne Nat. Lab., IL, USA
Scientific applications often need to access remote file systems. Because of slow networks and large data size, however, remote I/O can become an even more serious performance bottleneck than local I/O performance. In this work, we present RFS, a high-performance remote I/O facility for ROMIO, which is a well-known MPI-IO implementation. Our simple, portable, and flexible design eliminates the shortcomings of previous remote I/O efforts. In particular, RFS improves the remote I/O performance by adopting active buffering with threads (ABT), which hides I/O cost by aggressively buffering the output data using available memory and performing background I/O using threads while computation is taking place. Our experimental results show that RFS with ABT can significantly reduce the remote I/O visible cost, achieving up to 92% of the theoretical peak throughput. The computation slowdown caused by concurrent I/O activities was 0.2-6.2%, which is dwarfed by the overall performance improvement in application turnaround time.
Citation:
Jonghyun Lee, R. Ross, R. Thakur, Xiaosong Ma, M. Winslett, "RFS: efficient and flexible remote file access for MPI-IO," cluster, pp.71-81, Sixth IEEE International Conference on Cluster Computing (CLUSTER'04), 2004
Usage of this product signifies your acceptance of the Terms of Use.