Skip to content

ADR-048: Voice-to-text for rationale inputs via Web Speech API

Status: Accepted Date: 2026-05-09 Context: Rationale boxes are the primary signal channel for the judgment layer — they're where players explain their decisions, peer predictions, trust conditions, and mind-changes. Several playtest observations pointed at friction here: Natascha (slow reader, visual thinker) noted extra cognitive load after reading a scenario; mobile players find typing a full sentence mid-game feels costly. Voice input lowers the barrier to producing richer, longer rationale without slowing the game rhythm.

Decision

Add a microphone toggle button to every rationale input (textarea and text input), powered by the browser-native Web Speech API. The feature is silently hidden on unsupported browsers (Firefox) — no layout shift, no fallback copy.

Privacy note (corrected): Web Speech API in Chrome/Edge is not on-device — audio is streamed to Google's speech servers for transcription. Safari (iOS/macOS) does on-device recognition. The original ADR draft claimed "no audio leaves the device"; this was wrong for Chromium browsers and has been corrected. For rationale-box use this is acceptable (the rationale text we capture from typing is already sent to our own servers and to Claude), but it should be surfaced if/when we add voice to more sensitive inputs.

Implemented as a shared <RationaleInput> component and a useSpeechRecognition hook so all call sites share behavior with no per-page logic. The hook uses continuous: false with auto-restart on onend — each utterance gets a fresh short-lived connection to the speech service, which is far more reliable than continuous: true's long-lived stream (which produces frequent network errors).

Rationale

  • Zero cost to us: Web Speech API is free for the developer — Google subsidizes the speech service in Chrome/Edge, Apple subsidizes it in Safari. We pay nothing per minute.
  • Low latency interim results: The API streams interim transcripts as the player speaks, so the box updates live.
  • Covers the playtest devices: Chrome, Edge, Safari (including iOS) all support the API. Firefox is the only notable gap; it affects less than 5% of our playtest cohort.
  • Component boundary future-proofs the upgrade: If we later want a single first-party transcription path (Whisper/Deepgram) for consistency or Firefox support, only useSpeechRecognition needs to change — all seven call sites stay untouched.
  • Append-at-cursor semantics: Dictation inserts at the cursor position, not at the end, so players can dictate into the middle of typed text naturally.
  • ADR-027 footprint: No new content guardrail surface is created — voice maps directly to text we'd already capture. The Chromium audio path goes to Google's speech service rather than our own pipeline; same Google-eyes-on-data posture as any user of Google Docs voice typing.

Alternatives Considered

  • Whisper / Deepgram via /api/transcribe route: Better accuracy and punctuation, Firefox support, but adds ~$0.006/min per session, requires recording + upload UX, and adds 1–2 second latency for final results. Not warranted until Web Speech quality complaints arise.
  • Per-page implementation: Avoided in favor of a shared component — avoids the pattern of seven divergent mic implementations drifting over time.
  • Opt-in via settings toggle: Considered but rejected. The mic button is already opt-in (you have to click it); hiding it behind a settings toggle adds friction with no benefit, and the button disappears automatically on unsupported browsers so there's no display noise.

Discussion

The main debate was cost tier: start with Web Speech (path 1) vs. going straight to Whisper (path 2) for higher accuracy. The deciding factor was that Web Speech quality is "good enough for thinking out loud" — rationale boxes don't require perfect punctuation or speaker-independent accuracy. A player dictating "I picked this because it keeps the team together even if it slows us down" doesn't need comma placement. If Whisper becomes warranted, the upgrade is a hook swap with no call-site changes — that made path 1 the lower-risk starting point.

The cursor-position anchor was a non-trivial design choice: capturing selectionStart/selectionEnd at the moment the mic button is clicked (not at onresult) ensures that a player who types, then dictates, then types more gets natural append behavior throughout.

Consequences

  • Mic button appears in Chrome/Edge/Safari, hidden in Firefox and SSR — no functional regression on any platform
  • All seven rationale inputs now use <RationaleInput> — future changes to rationale UX (character counts, submit-on-enter, etc.) have one place to update
  • 60-second auto-stop prevents dangling recognizers if a player forgets to stop
  • Errors (not-allowed, network, audio-capture) surface as actionable inline tooltips and the mic button remains clickable for retry — first-attempt failures (e.g. a flaky network blip) don't strand the player
  • no-speech is treated as benign (silence between utterances) — the recognizer auto-restarts via continuous: false + onend loop, so brief pauses don't end dictation
  • Chrome/Edge users have audio streamed to Google's speech servers; if we later need a first-party transcription path this is a hook swap, no call-site changes

Key files: - src/hooks/useSpeechRecognition.ts — hook: browser detection, start/stop, cursor-aware append - src/components/ui/RationaleInput.tsx — component: mic button, multiline/singleline, error display - src/types/speech-recognition.d.ts — Web Speech API type declarations (not yet in TS DOM lib) - src/app/solo/page.tsx — two call sites replaced (noneRationale, main rationale) - src/app/play/[sessionId]/page.tsx — five call sites replaced (noneRationale, main rationale, peer prediction rationale, trust condition, revised rationale)