Trajectory recovery and stroke reconstruction of handwritten mathematical symbols

With the increasing strength of online handwriting recognition systems, it is natural to try to exploit their power also for offline handwriting recognition problems, by reconstructing the necessary information on direction of strokes using a technique called trajectory recovery. While most work on trajectory recovery has focused on characters with single strokes, to widen the impact of this technique, the chronological order and direction of strokes for characters composed of multiple strokes needs to be resolved. In this paper, we tackle this problem for the case of recognising combinations of mathematical symbols and Latin characters from images of whiteboards. We propose two methods for trajectory recovery on multi-stroke characters: one using combinatorial reconstruction and exhaustive search and one employing information on colour and ink artefacts and heuristic search. We have evaluated our techniques in the context of character recognition from whiteboards, on a test set with nearly 1400 images obtained with differing quality cameras, from a high resolution camera to the device build into Google Glass. Overall, we achieve an accuracy of 84.54% for the recognition of symbols with our informed reconstruction method.