Gamification of a Visual Question Answer System

Picture books and charts are widely used to introduce children to concepts such as alphabets and numbers and to teach them to identify objects like animals and birds. However, this can get monotonous and is beset with the possibility of the child associating a specific picture with only the depicted context. Introducing multiple images with the concept would facilitate learning and gamification would provide a framework to assess how much a child is able to generalize through novel images. In this paper, we have used a deep learning technique, called visual question answering (VQA), to present a proof-of-concept, as a context-specific, game-based application targeted at children in the age group of 3 – 4 years. The application is structured into levels of varying detail to teach children about various animals and sports. Further, the application is voice enabled and may be generalized to many categories. Finally, the model is able to identify the category of the image and generate a suitable question, without any user intervention.