From Label Maps to Label Strokes: Semantic Segmentation for Street Scenes from Incomplete Training Data

This paper proposes a novel image parsing framework to solve the semantic pixel labeling problem from only label strokes. Our framework is based on a network of voters, each of which aggregates both a self voting vector and a neighborhood context. The voters are parameterized using sparse convex coding. To efficiently learn the parameters, we propose a regularized energy function that propagates label information in the training data while taking into account of context interaction and a backward composition algorithm for efficient gradient computation. Our framework is capable of handling label strokes and is scalable to a code book of millions of bases. Our experiment results show the effectiveness of our framework on both synthetic examples and real world applications.