loading...
A Tool for Prioritizing DAGMan Jobs and Its Evaluation
Paris June 19-June 23
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/HPDC.2006.16521462006 15th IEEE International Conferen ...
 This Article 
 
PDF
HTML
 
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
It is often difficult to perform efficiently a collection of jobs with complex job dependencies due to temporal unpredictability of the grid. One way to mitigate the unpredictability is to schedule job execution in a manner that constantly maximizes the number of jobs that can be sent to workers. A recently developed scheduling theory provides a basis to meet that optimization goal. Intuitively, when the number of such jobs is always large, high parallelism can be maintained, even if the number of workers changes over time in an unpredictable manner. In this paper we present the design, implementation, and evaluation of a practical scheduling tool inspired by the theory. Given a DAGMan input file with interdependent jobs, the tool prioritizes the jobs. The resulting schedule significantly outperforms currently used schedules under a wide range of system parameters, as shown by simulation studies. For example, a scientific data analysis application, AIRSN, was executed at least 13% faster with 95% confidence. An implementation of the tool was integrated with the Condor high-throughput computing system
Index Terms:
Internet-based computation, DAGMan job scheduling, job dependency, scheduling tool, scientific data analysis application, Condor high-throughput computing system
Citation:
G. Malewicz, I. Foster, A.L. Rosenberg, M. Wilde, "A Tool for Prioritizing DAGMan Jobs and Its Evaluation," hpdc, pp.156-168, 2006 15th IEEE International Conference on High Performance Distributed Computing, 2006
Usage of this product signifies your acceptance of the Terms of Use.


Suggestions