Recognizing Handwritten Mathematics

Nicholas Matsakis Erik G. Miller Paul Viola

The last few years has seen the advent of a number of commercially successful products which are able to recognize a user's natural handwriting and use this ability to perform simple tasks such as scheduling appointments and writing memos. Most scientists and engineers, however, are unable to take advantage of these computers for their technical work due to the lack of effective algorithms for interpreting more complex handwritten expressions, particularly diagrams, graphs, equations, and other mathematical forms. While computers can store these forms as "digital ink," the inability to work with the expressions in a meaningful way after they have been entered has prevented these products from replacing pencil and paper.

As more powerful computers with better displays and input devices become available, demand will increase substantially for software systems which can work with the type of handwritten data that one would find in a research notebook or technical document. Mathematical expressions are a natural place to begin such research as they are critical to virtually all technical writing and there already exists a wide body of literature on recognizing handwritten letters and words, major subcomponents of these expressions. Combining mathematical expression recognition capabilities with existing algebra solving software, graphing programs, and simulation systems would be a first step towards a superior user interface for doing technical work with a computer.

More Information