C-SWF: A Lightweight Scientific Workflow System for Astronomical Data Processing

Due to the urgent requirements of mass data processing in the field of astronomy, with the advent of network grid computing techniques, a scientific workflow technique was presented and quickly adopted for distributed astronomical data processing. However, current existing scientific workflow systems are too complex and enormous in system deployment and system manipulation. Few people can master it in a short period. In this paper, we presented a lightweight scientific workflow system, C-SWF, which is specially designed for astronomy. All required fundamental functions, such as task customization, data movement, provenance and task re-run mechanisms, are fully implemented. Comparing with the existing scientific workflow system, C-SWF provides many useful features such as simple, high performance and is easy to deploy in order to meet the requirements of astronomers.