Separating transparent layers of repetitive dynamic behaviors

In this paper, we present an approach for separating two transparent layers of complex nonrigid scene dynamics. The dynamics in one of the layers is assumed to be repetitive, while the other can have any arbitrary dynamics. Such repetitive dynamics includes, among other, human actions in video (e.g., a walking person), or a repetitive musical tune in audio signals. We use a global to local space time alignment approach to detect and align the repetitive behavior. Once aligned, a median operator applied to space time derivatives is used to recover the intrinsic repeating behavior, and separate it from the other transparent layer. We show results on synthetic and real video sequences. In addition, we show the applicability of our approach to separating mixed audio signals (from a single source).