This app is built for deaf people — for situations where someone is speaking but hearing can no longer catch it. A conversation at the table, an announcement, a phone call.
What already works today
- Live captions — whatever is spoken appears on screen as text in real time
- Type→Speak — typed text is spoken out loud, for the other direction of the conversation
- Conversation transcript — a whole conversation gets written down, to read back instead of only catching the moment
Three of four planned modes are real and functional, no mockups.
What’s still open
The fourth mode, sound alert — detecting when the doorbell or a smoke alarm goes off and turning that into a visible notification — is currently still a placeholder. The audio recognition model it needs (YAMNet) isn’t built in yet.
And one topic is still missing entirely, and I’m aware of it: sign language. For many deaf people, sign language is their actual native language — not text on a screen. Captions and transcripts are a start, not a replacement.
This isn’t just about the direction „spoken language → text for the deaf person“. The reverse direction matters just as much: if someone answers in sign language and the other person doesn’t understand it, the barrier becomes just as real for the hearing person. So the phone would need to translate the other way too — sign to speech or text, live. Technically that’s the far bigger challenge, but it belongs on the list. Honestly marked as open instead of quietly skipped.
This series: Part 1 · Part 2 · Part 3 — Vision