Contextual recognition of hand-drawn diagrams with conditional random fields

Hand-drawn diagrams present a complex recognition problem. Fragments of the drawing are often individually ambiguous, and require context to be interpreted. We present a recognizer based on conditional random fields (CRFs) that jointly analyze all drawing fragments in order to incorporate contextual cues. The classification of each fragment influences the classification of its neighbors. CRFs allow flexible and correlated features, and take temporal information into account. Training is done via conditional MAP estimation that is guaranteed to reach the global optimum. During recognition we propagate information globally to find the joint MAP or maximum marginal solution for each fragment. We demonstrate the framework on a container versus connector recognition task.