Automatic detection and understanding the text in natural images, such as photographs of city outdoors or building indoors, is a challenging problem. There is a considerable gap between detecting and understanding text in scanned documents (which is a mature technology) and detecting and understanding text in the natural images. Detection and understanding the text in natural photographs involves localizing the text as well as removing the variation factors, such as varying text orientation, font, color and lighting. Examples of natural scene texts are shown below.

Acknowledgements

This project is supported by Microsoft Research programs in Russia.