DocumentCode :
1854453
Title :
P2G: A Framework for Distributed Real-Time Processing of Multimedia Data
Author :
Espeland, Håvard ; Beskow, Paul B. ; Stensland, Håkon K. ; Olsen, Preben N. ; Kristoffersen, Ståle ; Griwodz, Carsten ; Halvorsen, Pål
Author_Institution :
Dept. of Inf., Univ. of Oslo, Oslo, Norway
fYear :
2011
fDate :
13-16 Sept. 2011
Firstpage :
416
Lastpage :
426
Abstract :
The computational demands of multimedia data processing are steadily increasing as consumers call for progressively more complex and intelligent multimedia services. New multi-core hardware architectures provide the required resources, but writing parallel, distributed applications remains a labor-intensive task compared to their sequential counter-part. For this reason, Google and Microsoft implemented their respective processing frameworks MapReduce and Dryad, as they allow the developer to think sequentially, yet benefit from parallel and distributed execution. An inherent limitation in the design of these batch processing frameworks is their inability to express arbitrarily complex workloads. The dependency graphs of the frameworks are often limited to directed acyclic graphs, or even pre-determined stages. This is particularly problematic for video encoding and other algorithms that depend on iterative execution. With the Nornir runtime system for parallel programs, which is a Kahn Process Network implementation, we addressed and solved several of these limitations. However, it is more difficult to use than other frameworks due to its complex programming model. In this paper, we build on the knowledge gained from Nornir and present a new framework, called P2G, designed specifically for developing and processing distributed real-time multimedia data. P2G supports arbitrarily complex dependency graphs with cycles, branches and deadlines, and provides both data- and task-parallelism. The framework is implemented to scale transparently with available (heterogeneous) resources, a concept familiar from the cloud computing paradigm. We have implemented an (interchangeable) P2G kernel language to ease development. In this paper, we present a proof of concept implementation of a P2G execution node and some experimental examples using complex workloads like Motion JPEG and K-means clustering. The results show that the P2G system is a feasible approach to multimedia proce- - ssing.
Keywords :
multimedia computing; parallel programming; Kahn process network implementation; Nornir runtime system; P2G execution node; P2G kernel language; cloud computing; complex dependency graphs; data parallelism; distributed real-time multimedia data processing; intelligent multimedia services; k-means clustering; motion JPEG; parallel programs; proof of concept implementation; task parallelism; video encoding; Batch production systems; Kernel; Multimedia communication; Parallel processing; Pipelines; Runtime; Streaming media; Distributed Execution System; Heterogeneous Processing; Multimedia Processing; Parallel Processing;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Parallel Processing Workshops (ICPPW), 2011 40th International Conference on
Conference_Location :
Taipei City
ISSN :
1530-2016
Print_ISBN :
978-1-4577-1337-8
Electronic_ISBN :
1530-2016
Type :
conf
DOI :
10.1109/ICPPW.2011.22
Filename :
6047051
Link To Document :
بازگشت