A team of doctoral candidates and a University of Houston professor are developing various augmented reality and virtual reality programs, which include programs designed to help people with Asperger’s identify emotions. The programs are being developed primarily for the Microsoft HoloLens, an augmented reality headset that places holograms and information over the physical world, said computer science professor Chang Yun. Some, however, are made in virtual reality using the HTC Vive.
Another program the team is working on aims to help people with Asperger’s or high functioning autism converse with others. It uses the Microsoft HoloLens and tells the user what emotion — happy, sad, angry, neutral or surprised — is displayed on the face whoever they are talking to.
“If we can rely on it, imagine how much accessibility we can give those patients, so they can practice at least in their homes or even with their friends,” said Mohammed Alshair, a doctoral candidate working on the project. “They can wear it or just look and know this person is getting frustrated, getting mad, getting angry, and over time, they get better.”
They are working with doctors from the University of Texas Health Science Center to test the viability of such a program. Yun has been working for several years to find ways to help those with Asperger’s or high functioning autism, previously by using Kinect-based games at the UH Clear Lake Center for Autism and Developmental Disabilities.
“The high functioning autistic patients or Asperger’s (patients) having problems recognizing social cue, like the facial expression,” Yun said. “So they fail conversing with other people, time after time after time. In the end, what happens is people, after enough defeat, they give up. Now we are providing a tool, so they can understand how other people are feeling, at least facially.”
Five years ago, a family with a son who has autism came to Yun. The son’s mother told Yun that teaching him to program would be more than enough — more so than earning a degree. “That kind of does things on you, literally,” Yun said. “Seeing a mother beg me to help… After then, just chances came by, and I met this faculty director leading UHCL’s Autism Center.”
MRI data typically come in slices with parts of the brain shown in each slice, Yun said, which really limits what can be observed. The team has an AR program that creates a 3-D hologram of the brain in the physical world, so doctors and physicians can walk around and view the hologram and any tumors or blood vessels connected to it as large as they want.
“Say a doctor needed to find a tumor in the lower portion of the brain. We can take that and just put a little color on that, it’ll pop straight out,” Holtkamp said. “Surgery is obviously something very serious at that point in time, so minimizing potential problems is a huge win.”
Yun said he is optimistic about how prevalent augmented reality will be 5/10 years from now with its ability to overlay more information on top of anything and as AR headsets become smaller and cheaper.
“This is like iPhone One. Nobody had a true concept of what smartphones would be able to do. Now everybody knows it’s an essential,” Yun said. “Ten years from now, this will be more like an essential. All of this information that you need to know and want to know, in addition. Have a imagination of what this type of device can do for you to better your living.”
The team is dealing with the initial problems and finding the solutions for AR to pave the way for those who want to build on it in the future. “If enough people are impressed, screw it. We are going to make this work,” Alshair said. “I’m one of them. Annoying, yes, I’m cursing Microsoft everyday for like why, why, why. But at the end, I’m not choosing not to, because it’s worth the trouble.”
We could not agree more!