Hierarchical neural model for workflow scheduling in Utility Management Systems

Abstract

The emerging computational grid infrastructure consists of heterogeneous resources in widely distributed autonomous domains, which makes job scheduling very challenging. Although there is much work on static scheduling approaches for workflow applications in parallel environments, little work has been done on a real-world Grid environment for industrial systems. Utility Management Systems (UMS) are executing very large numbers of workflows with very high resource requirements. Unlike the grid approach for standard scientific workflows, UMS workflows have different set of computation requirements and thereby optimization of resource usage has to be made in a different way. This paper proposes architecture for a new scheduling mechanism that dynamically executes a scheduling algorithm using near real-time feedback about current status Grid nodes. Two Artificial Neural Networks (ANN) were created in order to solve scheduling problem. First ANN predicts future state of Grid based on current state and types of workflows that are currently executing. Second ANN output is optimal workflow type that should be executed. Inputs for second ANN are current state of the Grid and predicted future state (output of first ANN). Performance tests show that significant improvement of overall execution time can be achieved by this Hierarchical Artificial Neural Networks. © 2010 IEEE.