An Automatic and Scalable Testing Tool for Workflow Systems

Nowadays workflow systems are widely deployed around the world, especially within large international corporations. Thus the performance evaluation of these workflow systems becomes an important topic, especially in grid environment. We have developed an automatic and scalable testing tool to evaluate workflow systems. This tool is named Scalable Workflow systEm Evaluation Tool (SWEET). SWEET can automatically test against workflow systems according to a highly customizable process template specified by users. It can also provide multi-layer performance metrics related to response time, throughput and fairness. Further, it can adoptively determine the maximum concurrency of a workflow system under a desired average processing time, using control theory (PI Controller). In this paper, we will present the design and architecture of SWEET. We will also discuss the evaluation metrics and our implementation of such a framework. Finally, we will give detailed experimental results and analysis.