Augmenting Reality To Assist the Blind

March 11th, 2016

Aira’s platform today using browser-based Agent Dashboard

Aira’s platform today using browser-based Agent Dashboard

Augmented Reality (AR), along with its close cousin Virtual Reality (VR), are technologies that have already claimed “Next Big Thing” status in entertainment-driven markets like video games and movies. However industry experts predict that the most dramatic and lasting impact for AR & VR will instead come in the enterprise — particularly in service-focused sectors.

The appeal of AR and VR lies in their ability to immerse us in real or enhanced environments, enabling people to experience a wide range of situations, surroundings and resulting emotions like never before. AR and VR are fast expanding to key business service sectors such as healthcare, medicine, education, news media and real estate. Goldman Sachs predicts that 2016 will see AR and VR emerge as the next-generation computing platform, and by 2025 they will likely become an $80 billion market, roughly the size of the desktop PC market today.

At Aira, we are particularly interested in leveraging AR technology to serve our unique market: the blind and visually impaired. In this regard, AR will serve as a viable tool in Aira´s mission to create innovations and assistive services that bring increased mobility, independence, confidence and efficiency into the lives of the blind. AR will also further enable our trained remote Agents to assist blind users effectively.

To give more perspective on how augmented reality will assist Aira in the future, here is a brief description of how our dashboard technology platform currently works: (1) The platform, which uses connection from major network providers, is activated when our blind users´ wearable smart glass device (such as Google Glass) is paired with their smart phone; (2) Users then simply activate their smart glass to request to be connected in real time with a certified Aira Agent via the platform´s Uber-style routing system; (3) Once the request is accepted, the Agent will see what the user sees with <1 second latency while receiving real-time streaming updates from such sources as Google Maps, GPS, Street View, object/face recognition, and step/distance readings; and (4) Agents are then able to assist users in performing a myriad of daily activities — ranging from navigating city streets and catching a plane, ordering meals from a restaurant menu, exploring new neighborhoods and literally traveling the world.

Aira’s platform today using browser-based Agent Dashboard

Aira’s platform today using browser-based Agent Dashboard

Aira provides prompt and efficient service to the blind by augmenting a real-time context remotely of users and their surroundings. These data streams can be from a number of different Internet of Things (IoT) devices. The dashboard platform does the heavy lifting and manages events, processes video frames, analyzes sensor data, and over time builds a personalized experience through machine learning.

Aira envisions additional AR-inspired innovations to assist its blind users. These approaches will involve developing ways to truly immerse Aira Agents in the world of the blind, helping our Agents ¨to really see and experience¨ this world through the wearable smart glasses of the people they are assisting. AR glasses, such as Oculus Rift, worn by Agents, instead of a looking at dashboard on a monitor, would give Agents a 360-degree view of the blind person and his/her environment — thereby allowing greater assistance in such activities as crossing busy streets, navigation at night, and locating objects and faces , to be done in a safer, more comprehensive and efficient manner.

In exploring how AR application can further enhance Aira´s assistive service, I am particularly impressed with Oculus Rift, the VR head-mounted device set to be released in March 2016 as one of the first viable consumer-targeted virtual reality headsets. All this bodes well for Aira as it works to develop viable AR approaches to enhance its existing platform technology, including video streaming, and greater sensor capability that allows Agents to better detect ambient light, sounds and temperature in the blind person´s environment.

At the same time, Aira is also working on other technological fronts to further personalize its services. This includes leveraging machine learning systems such as Clarifai to benefit its dashboard platform with a highly functional artificial intelligence-driven system. In this project Aira is using Agents´ human skills to train Machine Learning (ML) algorithms how to aid the visually impaired more efficiently.

We at Aira look forward to the promise that augmented reality and other leading technologies hold, including their ability to assist us in our ultimate mission to serve as visual interpreters for the blind.