AI-integrated Apps for Those with Visual Impairments: Camera-Based Identifiers and Readers
Author: Yerin Seung, Samantha Goldman; info@ciddl.org
Associated material
Picture a classroom where every student, including those with visual impairments, can access learning materials and environments. Using the Universal Design for Learning framework, combined with artificial intelligence (AI), teachers are able to make this dream a reality in their classrooms. AI has empowered students with visual impairments to explore their potential and tackle academic tasks.
Technologies to support those with visual impairments such as braille displays, and electronic magnifiers are not new. However, AI offers an increased potential for empowerment, self-reliance, and independence. In a recent study, AI was found to significantly improve the sight problems in the daily lives of those with visual impairments. It especially helped when they were walking on the road or reading text by translating visual information into audible cues.
Why AI for students with visual impairments?
Some of the challenges students with visual impairments experience include reading, seeing objects at a distance, and/or participating in class. AI can support reading and seeing objects by providing audible information rather than visual information. What makes AI even better is that this cutting-edge technology is accessible via apps on smartphones. Built-in AI assistants such as Siri on iPhones, Microsoft’s Cortana, Google’s Allo, and Samsung’s Bixby, are natural language processors that perceive voice, execute orders, and/or provide auditory feedback. This blog post will focus on camera-based identifiers and screen readers that were particularly designed for those who are blind or have vision impairments. Additionally, these supports have the potential to promote a more inclusive learning environment.
AI-integrated Camera-based Identifier and Reader Apps
Camera-based identifiers are apps designed to assist individuals with visual impairments by using imaging technology to recognize and translate visual information in more accessible ways. With AI integration, these supports provide more personalized experience by analyzing the environment instantly. Though each has unique features, some common features are identifying objects, texts from documents or images, barcodes, and currency. The information is interpreted into audible sound so users can understand and interact with the environment.
Seeing AI, available on iOS, also identifies people and the amount of light. It is currently testing the function to sense scenery, color, and handwriting as well. Another app, Lookout by Google, available on Android, has its strength in recognizing food labels. Envision AI, available on iOS and Android, also scans colors and faces. Additionally, it can find the specific object you designate and notify you. Supersense, available on iOS and Android, has a smart scanner mode that automatically determines whether the camera is facing text, document, currency, objects, or barcode and provides information accordingly. You can use all four apps free of charge.
How does it work in classrooms?
Researchers are reporting that students with visual impairments find AI-integrated apps very useful. Though students and teachers need explicit training on how to use the apps to support their needs as mandated in the Individuals with Disabilities Education Act (IDEA), there are advantages to using them in classrooms. One benefit of these apps is they are intuitive and respond to the environment. They automatically read texts or images picked up by the camera and use text-to-speech to translate them into voice. As a result, instructions and learning environments are more accessible to all students. Students with visual impairments can perceive, understand, explore, and engage with the classroom and their peers, with more independence through the use of AI.
Share your thoughts and experience
Do you have experience using these apps or similar ones in your classroom? How do you integrate technology to support learners with visual impairments in your pre-service preparation programs? Try these out and tell us about your experiences in the CIDDL community!