Augmented Reality Contact Lens Startup Develops Apps With Early Adopters-to-Be

Last year, startup Mojo Vision unveiled an early prototype of a contact lens that contains everything it needs to augment reality-an image sensor, display, motion sensors and wireless radios all safe and comfortable to tuck into your eye.

These days, the company's engineers are charging hard on development" says Steve Sinclair, Mojo Vision's senior vice president of product and marketing. While most of Mojo Vision's hundred employees are working at home, the pandemic only minimally affected the company's ability to use its laboratories and clean room facilities as needed. The company's engineers, for instance, are helping a vendor of motion sensors thin its dies for better wearability, partnering with a battery manufacturer to build custom batteries, and refining their own designs of displays, image sensors, and power management electronics. And Mojo last year signed an agreement with Japanese contact lens manufacturer Menicon to fine-tune the materials and coatings of the lens itself.

About a dozen of the company's employees have worn early prototypes of its lenses. The next generation of prototypes, now under development, is expected to be ready for such testing later this year.

While commercial release is still a few years out, requiring FDA approval as a medical device, development of the first generation of applications is well underway. While Sinclair says the company long anticipated that the earliest adopters of its AR contacts would be the visually impaired, exactly what applications would be useful wasn't fully clear.

Ashley Tuan, Mojo Vision's vice president of medical devices, has a personal interest in the technology-her father has limited vision, due to a rare retinal degeneration disease.

With a Ph.D. in vision science, Tuan has also studied the biology. People with low vision, she says, can't see fine detail because photoreceptors have died. That is typically solved with magnification, though people generally don't like to carry a magnifier around. They also have an issue with contrast sensitivity, that is, the ability to sense light grey, for example, from a white background. This can stop them from going into unfamiliar surroundings because they don't feel safe. Most of us subconsciously use shadows to identify something coming up, something that we might trip over. In studies, even a slight reduction in contrast sensitivity stops people from going outside."

Mojo Vision is developing apps to address these issues. Enhancing contrast is easy to do with our technology," Tuan says. Because we are projecting an image onto the retina, we can easily increase the contrast of that image.

We can also do edge detection, which I see as being one step above contrast enhancement, by highlighting the edges of objects with light."

Mojo Vision has these tools implemented in prototypes. In the future, Tuan expects the technology will evolve to adjust what it is highlighting according to the context-a street sign, perhaps, or the facial expressions of someone talking to the wearer.

Magnification, with the current prototype, is triggered by the user zooming using the image sensor. In the future, the company expects to make magnification context dependent as well, with the system determining when a wearer is looking at the menu in a restaurant and therefore needs magnification, or looking for the bathroom, and instead needs the letters of the sign and the frame of the door brightened.

Mojo Vision's development team is now trying to take these basic ideas for apps and turn them into something that they hope will be immediately useful for their early adopters. To do that, they turned to a Silicon Valley resource, the Palo Alto Vista Center's Corporate Partners Program.

Says Alice Turner, Vista Center's director of community and corporate relations: We have been working with tech companies in the product design realm for about three years, about eight to ten companies so far, including Facebook, Microsoft, Samsung, and other companies that haven't announced products yet. I know partnering with us results in a better product, better suited, and the population that it is addressing will be better served."

Mojo Vision," she says, is a huge success story for our partnership program."

The tech tools that eventually come to market through this program help some of Vista Center's clients. But the program also provides more immediate benefits; the fees it charges the companies provide a steady source of revenue for the nonprofit.

When brought in on a product development project by a tech company, Turner taps into Vista Center's client database, some 3400 individuals representing a wide range of vision impairments and demographics. She selects people who are appropriate matches for the technology under development, confirms that they have some basic tech skills and are comfortable talking freely about their condition and experiences, and sets up one-on-one meetings (virtual in these pandemic times) and focus groups during which the developers can get feedback for anything from an idea to a rough demo to a working prototype.

David Hobbs, director of product management at Mojo Vision explained that, for Mojo, the process started with interviewing to find out more about problems that the technology could potentially solve. After identifying the problems, the team brought a subset of Vista's clients in for a deeper dive.

For example," he said, When we are trying to understand how to cross a crosswalk, we may talk for hours about different crosswalk situations."

Then the Mojo Vision design team builds prototype software for use with customized virtual reality headsets. The Vista clients test the prototype software and give feedback. The company will do clinical trials with lens prototypes after it gets FDA Breakthrough Device approval.

We are learning," Hobbs said, that vision is uniquely intimate. Everyone sees differently, so finding a way to provide the information that someone wants in the way they want it is really challenging. And different scenarios require different levels of detail and context."

While this development process continues, the Mojo Vision developers have already learned from the tests conducted so far.

When we looked at edge detection," Hobbs said, we were just taking an image and, wherever there was a lot of difference between two pixels, drawing a line. We created detailed models of the world in this way. Our bias was that all of this detail was valuable.

But the feedback we got was all these lines were a lot of noise," he continues. It turns out that there is a level of information that we can provide that can help people on their journey without being overwhelming."

Hobbs expects many the AR applications developed for the visually impaired to evolve into technology useful for everyone. Fulfilling the needs of the most demanding users provides a lot of capabilities for general users as well. Take the ability to see in the dark. Going into a dark stairwell and having enhanced vision flip on would be valuable to many."