Developing Assistive Technology using Augmented Reality


When Apple introduced the iPhone X, it came along with in-build augmented reality (AR) hardware. The framework was dubbed as the ARKit, and it consisted of 3D sensors in its TrueDepth camera. Apple used the technology to enable its Face ID unlock feature and used it in a few AR software’s that could be used to place animals and weird creatures in the living room. However, eBay has now developed a software that utilizes the Apple’s TrueDepth camera and ARKit to transmit messages on the smartphone by just moving your head. The software could aid people with disabilities to use smartphones and could be an integral part of next-generation assistive technology.

The system has been named HeadGaze by eBay and works in a similar fashion as eye-tracking technology – which is generally used to scroll down pages on a smartphone. The sophisticated head-tracking system tracks the movement of a user’s head and accordingly moves the cursor on the screen. For eBay, the technology was developed keeping in mind that it would assist people in conducting online shopping – without having to use your hands to browse through multiple items.

However, for its creator Muratcan Çiçek, the technology has far-reaching applications. Çiçek, a software engineer interning at eBay, suffers from motor impairment disability. He uses assistive technology on a daily basis to assist him in day-to-day activities. Through the development of HeadGaze, he hopes to provide a software for people with little motor function skills to navigate the internet using nothing but head movements.

Assistive technology has seen unprecedented development since the advent of AR and Artificial Intelligence (AI). Gadgets are now available that implements either both or just one of these state-of-the-art technologies to provide a substitute for limbs, ears, and eyes. Now, thanks to the work done by Çiçek, even people with impaired motor skills can have a more accessible way to use mobile devices. Along with HeadGaze, Çiçek has also created a shopping app called HeadSwipe that utilizes the same technology to just conduct online shopping.

Such head-tracking systems are not new in concept, as many technology firms have developed the system. However, the simplicity by which HeadGaze works makes it accessible to a wide range of disabled people. Çiçek is currently exploring the domain of eye-tracking technology as well. In a blog post about HeadGaze, Çiçek writes, “In addition to this head gazing experience, we’re exploring an experience that tracks eye movements. The fusion of these gazing experiences opens up a broader possibility on defining various hands-free gestures, enabling much more interesting applications.”

To ensure swift development in HeadGaze, Çiçek has made the project open source – will all the code and documentation available in GitHub. Although it currently works in iPhone X, the implementation of AR technology on future smartphones will help spread the software.