Innovation That Matters

The app uses a computers webcam to track user's hand gestures and determine if they match the correct position. | Photo source Hello Monday

Fingerspelling app offers new way to learn sign language

Education

A browser-based app teaches people to finger-sign the ASL alphabet

Spotted: Creative studio, Hello Monday, which specialises in digital products and experiences, has teamed with the American Society for Deaf Children to create an app that teaches the ASL sign language alphabet. In ASL, fingerspelling is used for proper nouns, or when a person doesn’t know the sign for a word. The app teaches and corrects hand positions in real-time to make it easier for people to communicate with the deaf. 

The browser-based “fingerspelling.xyz” app is somewhat similar to popular language-learning app Duolingo but has had to incorporate vastly different technology in order to guide users in learning the correct hand positions. As users sign the word, their webcam tracks the hand movements and gives feedback so users can simultaneously correct their hand positions. This helps users develop skills more quickly.  

The app uses machine learning and “advanced hand recognition technology” to analyse users’ hand shapes and offers suggestions for improvement, turning learning into a game. Target users include family and friends of deaf people, who want to learn quickly to communicate. According to Happy Monday founding partner Anders Jessen, the app offers a more engaging learning experience than learning fingerspelling by reading or watching videos. 

Jessen adds that, “It’s fun to think about how this highly trained machine-learning model, which researchers have spent countless hours on training, now does the opposite – it now trains us back, and makes us better at fingerspelling.” 

Technology may have its faults, but it has proven very useful in helping people to better communicate, including the deaf. Springwise has seen many innovations aimed at helping improving communication between the hearing and the hearing impaired. These include a smart glove that translates sign language into spoken word and an app that allows sign language users to communicate with Alexa

Written By: Lisa Magloff

Explore more: Computing and Tech

Email: hello@hellomonday.com

Website: hellomonday.com

Download PDF

Springwise Services:
Our expertise in spotting the latest innovations is the best resource to empower your team’s future planning.

Find out More