UCLA bioengineers have developed a glove-like tool that will use a mobile app to turn American Sign Language into English speech in real time.
“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” said Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering and the principal investigator on the research. “In addition, we hope it can help more people learn sign language themselves.”
The device includes a pair of gloves with small, stretchable sensors which run each of the five fingers in duration. These sensors, made from yarns that are conducted electrically, pick up hand motions and finger placements that stand for individual letters , numbers, words and phrases.
And the tool converts the finger motions into electrical signals that are transmitted to a dollar-coin-sized circuit board worn on the handle. The board wirelessly transmits those signals to a smartphone, which translates them into spoken words at a rate of about one word per second.
The researchers also added adhesive sensors to the faces of testers to capture facial expressions that are part of American Sign Language — between their eyebrows and on one side of their mouths.
Previous wearable devices that provided American Sign Language communication were or were difficult to wear constrained by cumbersome and heavy interface designs, Chen said.
The system built by the UCLA team is constructed of lightweight polymers that are cheap yet long-lasting, stretchable. The electronic sensors are very flexible and cost-effective too.
The researchers were collaborating with four individuals who are deaf and who use American Sign Language while developing the app. Each hand movement was replicated 15 times by wearers. A design algorithm in machine-learning transformed such movements into the shapes, numbers, and words they described. The machine recognized 660 signs like through alphabet letter and numbers 0 through 9.
UCLA also secured a Software patent. Chen said a commercial model based on this technology would require additional vocabulary and even faster translation time.
