A Two-Layer Graphical Model for Combined Video Shot and Scene Boundary Detection

In this work we present a novel two-layer hybrid graphical model for combined shot and scene boundary detection in videos. In the first layer of the model, low-level features are used to detect shot boundaries. The shot layer is connected to a higher layer that detects scene or chapter boundaries from semantic features. With this structure, the model optimises the alignment for both layers at the same time and the detection results are interconnected. Experimental results on real video data show, that both layers highly benefit from this sharing of information. Compared to a baseline threshold method with the same features, the F-measure result for the shot detection has been improved by 12.6% absolute. For the scene boundary detection, the result has been improved by more than 11% absolute