Automatic Recognition of Pictograms with Convolutional Neural Network Approach for Literacy

Literacy is a fundamental and intrinsic right for every person; therefore, government agencies, NGOs, and private donors seek to implement sustainable and long-term plans that allow citizens to access quality, inclusive, and equitable education. But despite their efforts, illiteracy has not yet been fully eradicated, so educational programs and tools have been created to work to solve this problem. In this research, a web application is developed that takes the resources of the Google QuickDraw API to generate an interactive image interpretation tool, which aims to support the learning of reading and writing in people with illiteracy. The tool allows an illiterate person to draw pictograms that are interpreted by the software and through convolutional neural networks with deep learning, a prediction of the image is generated. The result is presented in a graphical user interface, which displays a real image corresponding to the prediction together with text and audio assistance, essential for user learning. The results show a performance of 75% accuracy in the training and 56% in the test. The performance tests yielded indicate that the tool meets the minimum software requirements for its future implementation in institutions that work in the education of people with illiteracy.