loading...
Data Management on Grid Filesystem for Data-Intensive Computing
Hiroshima, Japan January 15-January 19
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/SAINT-W.2007.382007 International Symposium on Appli ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Hitoshi Sato, Tokyo Institute of Technology, Japan
Satoshi Matsuoka, Tokyo Institute of Technology, Japan
In parallel computing environments such as HPC clusters and the Grid, data-intensive applications involve large overhead costs due to a concentration of access to the files on common nodes. To avoid this problem in traditional distributed filesystems, users have to distribute the file access manually. However, such solution has some difficulties for users in the Grid environment. We propose a data management mechanism for data-intensive computing on Grid filesystem. Our technique improves the file access performance by automatically scheduling the file access and the data management on the filesystem. The filesystem is based on dynamically configured node groups corresponding to the network topology. Utilizing the configuration, it monitors file access to detect concentrated situations, creates the file replica, and schedules its placement and access. We applied the proposal technique to the Gfarm, a filesystem that scales to the Grid. We emulate real application workloads using a job scheduler and confirmed a speedup of factor 3.7 compared with a filesystem without automatic file access distribution techniques.
Citation:
Hitoshi Sato, Satoshi Matsuoka, "Data Management on Grid Filesystem for Data-Intensive Computing," saint-w, pp.36, 2007 International Symposium on Applications and the Internet Workshops (SAINTW'07), 2007
Usage of this product signifies your acceptance of the Terms of Use.