Interactive Image Descriptions

Mission

Enhancing the accessibility of books with images for readers with visual impairments through intelligent Deep Learning/AI tools for enhanced reading experience of people who are blind or have low vision.

The challenge

Existing image descriptions often lack context or detail and fail to meet the diverse needs of people with visual impairment. Preferences vary widely based on factors such as the type and onset of visual impairment, age, and personal interests. A one-size-fits-all approach to image descriptions is inadequate, highlighting the need for customizable, context-aware solutions. As mentioned in the article “The Accessible Books Consortium: what it means for publishers” by Catherline Jewell in WIPO magazine in 2018, globally, less than 10% of published works are available in accessible formats, leaving a significant gap inclusive for people with visual impairments. Accessibility and inclusivity has often been an ignored segment. This project investigates how artificial intelligence (AI) and interactive technologies can bridge this gap - and develops other integrate the solution that fits well with the need and real life scenario of target users.

The solution

Using intelligent Deep Learning/AI tools to describe images in books with right context of books using audio and haptic feedback. The results are personalized based on individual need/interest/preferences, and offers voice-activated interactions, and haptic feedback to provide seamless, personalized image accessibility in book reading without disrupting the reading experience.