loading...
Dynamic Load Sharing With Unknown Memory Demands in Clusters
Mesa, AZ April 16-April 19
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/ICDSC.2001.91893921st IEEE International Conference on ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Songqing Chen, College of William and Mary
Li Xiao, College of William and Mary
Xiaodong Zhang, College of William and Mary
Abstract: A compute farm is a pool of clustered workstations to provide high performance computing services for CPU-intensive, memory-intensive, and I/O active jobs in a batch mode. Existing load sharing schemes with memory considerations assume jobs' memory demand sizes are known in advance or predictable based on users' hints. This assumption can greatly simplify the designs and implementations of load sharing schemes, but is not desirable in practice. In order to address this concern, we present three new results and contributions in this study. (1) Conducting Linux kernel instrumentation, we have collected different types of workload execution traces to quantitatively characterize job interactions, and modeled page fault behavior as a function of the overloaded memory sizes and the amount of jobs' I/O activities. (2) Based on experimental results and collected dynamic system information, we have built a simulation model which accurately emulates the memory system operations and job migrations with virtual memory considerations. (3) We have proposed a memory-centric load sharing scheme and its variations to effectively process dynamic memory allocation demands, aiming at minimizing execution time of each individual job by dynamically migrating and remotely submitting jobs to eliminate or reduce page faults and to reduce the queuing time for CPU services. Conducting trace-driven simulations, we have examined these load sharing policies to show their effectiveness.
Citation:
Songqing Chen, Li Xiao, Xiaodong Zhang, "Dynamic Load Sharing With Unknown Memory Demands in Clusters," icdcs, pp.0109, 21st IEEE International Conference on Distributed Computing Systems (ICDCS'01), 2001
Usage of this product signifies your acceptance of the Terms of Use.