A Smartwatch App Can Alert People With Hearing Loss to Nearby Sounds

Dhruv Jain, who presented this new system, shares that the main motivation for the creation of this app came from his own experience and conversations that his colleagues and he had had with deaf and hard-of-hearing people over several years. He presented the phone and smartwatch application called SoundWatch, and it’s the virtual ASSETS conference on October 28th.

The smartwatch app - SoundWatchSound Awareness App

Jain, who has trouble hearing, uses SoundWatch at home to avoid sleeping through a smoke alarm. He also says that on a nature walk, it will tell him that birds are chirping and that there is a waterfall nearby. The sounds make him feel more present and connected to the world.

Such sound awareness applications for smartphones already exist. However, Jain prefers the immediacy of sound notifications on his wrist, rather than in his pocket. Surveys of deaf people or those who are hard of hearing show that he is not alone.

The SoundWatch application pairs an Android phone or smartwatch. The watch records ambient noises. It then sends the data to the smartphone for processing. When the phone detects a sound of interest, the smartwatch vibrates and a notification is displayed on its face.

More About the Smartwatch App

Jain, who is a computer scientist at the Washington University in Seattle, and colleagues designed the application to identify up to twenty noises. During the experiments, the SoundWatch application correctly identified those twenty sounds 81.2% of the time.

In the second prototype of HomeSound, the tablets sent information to a smartwatch, which is how the researchers got the idea to make the standalone app. When it’s set to listen only for door knocks, fire alarms, alarm clocks, or other urgent noises, the app shows 97.6% accuracy. Eight hard-of-hearing and deaf people who used the application around a university campus gave the app broadly favorable reviews. However, it should be noted that the app misclassified some of the noisy outdoor settings and sounds.

Jain and his colleagues are now working on a newer version of SoundWatch that users can train to recognize new sounds, such as their house alarm, using just a few recordings.