MB++: An Integrated Architecture for Pervasive Computing and High-Performance Computing

MB++ is a system that caters to the dynamic needs of applications in a distributed, pervasive computing environment that has a wide variety of devices that act as producers and consumers of stream data. The architecture encompasses several elements: The type server allows clients to dynamically inject transformation code that operates on data streams. The transformation engine executes dataflow graphs of transformations on high-performance computing resources. The stream server manages all data streams in the system and dispatches new dataflow graphs to the transformation engine. We have implemented the architecture and show performance results that demonstrate that our implementation scales well with increasing workload, commensurate with the available HPC resources. Further, we show that our implementation can exploit opportunities for parallelism in dataflow graphs, as well as efficiently sharing common subgraphs between dataflow graphs.