Industrial Analytics Pipelines

Decreasing cost and increasing capabilities of instrumentation, networks and data repositories have pervaded the industrial automation and power markets and opened the door for large scale collection and analysis of data. There are a variety of technology stacks that can be applied to these types of activities. However, no single infrastructure or architecture fits all the scenarios. With limited data science training and experience, it is difficult and time consuming for highly specialized domain experts to choose the optimal approach. In this paper, we introduce an architectural pattern for the design of a flexible core analytics platform which is extensible using different pipelines. The pipeline pattern provides an accelerated start to implementing industrial analytics applications. The platform enables domain experts to compose pipelines in series and in parallel at scale with the right quality attribute trade-offs to deliver significant business value. Our use of the proposed platform is illustrated with real-world industrial applications, which necessitate various data handling and processing capabilities. These examples show the importance of the platform to non-data experts: reducing the learning curve for applying data science, providing a systematic rating process for choosing the pipeline types, and lowering the barriers for industrial businesses to leverage analytics.