Easing the communication gap!

Integrating smart glasses and mobile platform via an application which will help Deaf or Hard of Hearing (DHH) and hearing people to communicate with ease by connecting an online interpreted to the conversation.

Problem Statement

Currently, Deaf or Hard of Hearing (DHH) people does use technologies like typing text messages, software to translate speech to text, and old fashioned method of pen and paper. The problem with all the mentioned methods is that they are time-consuming and they require a lot of effort from the end of DHH people. In an ideal world, we would want an advanced technology that is quick, easy, and hands-free.

Our Solution: 

We needed a device which is:

  • Ease of use, daily-life accessibility like cellphones

  • Feels natural and intuitive

  • Hands-free device

  • More information in small space

  • Should not look awkward when using, social-acceptance

We came up with a solution to have smart glasses as people are already used to using glasses so they are intuitive and natural, they are hand-frees allowing DHH people to do signs and they are portable which allows them to be accessed on a daily basis just like a cellphone. Smart glasses designed for general communication among DHH people and also among hearing people. We think smart glasses are effective in easing the communication gap between DHH people and hearing people. Glasses could use different technologies to fix the issues, which have been lingering for ages, by translating audio to text and sending the text format obtained to DHH people and for hearing people it could convert sign to text by means of using an online interpreter and sending the obtained text format to hearing people.

Sign-to-Text conversion: In the smart glasses, we focus on how to translate the sign language to text by using an online interpreter. The glasses have a camera in front of them at the center over the nose bridge which captures the hand movement (sign language) done by DHH people and shows these signs to the interpreter which is on the other end. The interpret can speak out what DHH is signing, this would get converted in speech-to-text and it will send the text/audio file to hearing people. 

Voice-to-Text Conversion: For DHH people, the microphone in the glasses is recording what a hearing person is saying, now we can have the interpreter sign it out for DHH. Also, we can include audio-to-text translation and show it on the screen of the glasses to help DHH people view what hearing people said. 


Low fidelity:

High fidelity:

Design Specification-documentation: