Recognizing Handwritten Mathematics

Nicholas Matsakis Erik G. Miller Paul Viola

The last few years has seen the advent of a number of commercially successful products which are able to recognize a user's natural handwriting and use this ability to perform simple tasks such as scheduling appointments and writing memos. Most scientists and engineers, however, are unable to take advantage of these computers for their technical work due to the lack of effective algorithms for interpreting more complex handwritten expressions, particularly diagrams, graphs, equations, and other mathematical forms. While computers can store these forms as "digital ink," the inability to work with the expressions in a meaningful way after they have been entered has prevented these products from replacing pencil and paper.

As more powerful computers with better displays and input devices become available, demand will increase substantially for software systems which can work with the type of handwritten data that one would find in a research notebook or technical document. Mathematical expressions are a natural place to begin such research as they are critical to virtually all technical writing and there already exists a wide body of literature on recognizing handwritten letters and words, major subcomponents of these expressions. Combining mathematical expression recognition capabilities with existing algebra solving software, graphing programs, and simulation systems would be a first step towards a superior user interface for doing technical work with a computer.

More Information

Bibliography

PublicationPDF
Erik Miller, Nicholas Matsakis, and Paul Viola. Learning from One Example Through Shared Densities on Transforms. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, 2000. PDF
Nicholas Matsakis. Recognition of Handwritten Mathematical Expressions. Master's thesis, Massachusetts Institute of Technology, Cambridge, MA, May, 1999. PDF
Erik Miller and Paul Viola. Ambiguity and Constraint in Mathematical Expression Recognition. In Proceedings of the National Conference of Artificial Intelligence, 1998. PDF

Last Modified Thursday, 25-Sep-2003 13:31:28 EDT

Please send comments or questions to matsakis@mit.edu