Graph-based segmentation of letters in handwriting words with Lucas Kanade template warping

Handwriting Character Recognition (HCR) is the task of detecting and recognizing characters or symbols in a handwritten material, captured by some physical device. This thesis develops a new tool of HCR for a particular context in which the input text in the handwritten sources is known a priori since it results from a dictation task. We also assume in this work that the beginning and the end of the words have been identified. In contrast to previous related works, the objective of this method is to determine the segmentation of the words into letters instead of aiming to decode its content. We work with handwriting data recorded by a pressure-sensitive touchscreen device. We reconstruct an image of the handwritten contents and compute its distance transform. Our major contribution is to combine template matching techniques with graph theory to identify the letters on the processed image. Specifically, we first warp reference letter templates using the Lucas-Kanade optimization algorithm. In a second step, we consider that each warped template can be translated locally, and build a graph that connects all possible displacements of consecutive letters. Costs of edges in the graph account for (i) the Normalized Cross Correlation between each translated template and the word to segment and (ii) the distance between the end of a template and the beginning of the next one; mainly given by the sequential order of the letters in the word. The segmented solution is obtained on the graph representation by means of the Dijkstra shortest-path algorithm. The shortest-path solution jointly optimizes the similarity of the handwritten letters with warped templates and optimizes the spatial coherence between the letters. This method is designed to be robust in front of a high degree of variability in the handwriting style. Doing so, we want to make it applicable to segment handwritten words from children who are still acquiring handwriting skills; especially to make it applicable to handwritten words from children with ...