Emotion-recognition software for Autistic Children

We can only imagine how tough their lives can be, but a Stanford University student Catalin Voss has gone beyond a distant empathy and has developed an emotional learning aid for children with autism based on smart glasses like Google Glass. The emotion-recognition software is designed to tell an autistic child wearing the device whether a person the child looks at is happy, sad or angry.

Autistic children find it hard to make an eye contact or recognize emotions and social cues exchanged with other people. A bunch of tech entrepreneurs hopes Google Glass could aid these children in better identifying conversational nuances in real time—and one such entrepreneur is Catalin Voss who received a vote of confidence in his work on Tuesday, taking home the $15,000 “Cure it!” 2016 Lemelson–MIT Student Prize, which rewards technology-based health care inventions.

For Voss, behavior-focused therapies for Autism in which therapists use flashcards to promote learning and language are becoming redundant. “But there are just not enough therapists to meet increasing demand, and flash cards are removed from real life,” Voss says. The Autism Glass would give kids real-time social cues in a daily life setting while providing parents and psychologists data to better understand autism.

“The promise of using Google Glass is that it’s transportable out of a therapeutic setting into an everyday social situation, whether it be school or a birthday party,” says Rebecca Landa, director of the Center for Autism and Related Disorders at the Kennedy Krieger Institute.

The Autism Glass setup consists of a Google Glass used in conjunction with a smartphone, which runs software that analyzes data from the head-mounted display and provides feedback to the user. The system also records a video for parents to review and help kids improve their learning. Voss’s goal is that after a limited learning period, kids would not need the device.

A preliminary study showed that children expressed keenness to wear the device and interact with it and an in-home trial that started in January is now giving Voss data to fix bugs and refine the software. Voss, however, is in no mood to relax and is also working on perhaps the most critical challenge: how to help kids respond to the emotions they detect and identify.

Voss wants to eventually design a system that can track multiple faces engaged in conversation. He wants to make it learn and adapt to a particular person’s facial expressions, say the child’s parent or therapist. And he wants to make it more nuanced.

With validation from in-home trials involving 100 children, Voss plans to license the technology or form a start-up himself. The software could be adapted for wearables other than Google Glass, he says, or could simply be combined with a button camera and earpiece. This versatility is important given Google’s unclear commitment to its Glass technology, which was put on the shelf in 2014 only for a new version to be seemingly resurrected in a U.S. Federal Communications Commission filing last December.