ADR-019: Speed round question quality pass

Status: Accepted Date: 2026-04-08 Context: Playtesting revealed that several speed round questions produce low-signal responses because the answer genuinely depends on context that isn't given (e.g., "Should you open-source your program or keep it proprietary?" — the right answer depends on stage, product type, and competitive landscape). Other questions have a socially obvious "right" answer that nobody would honestly disagree with (e.g., "Would you take credit for a colleague's work?"). A third set were near-duplicates measuring the same signal. All three categories reduce the quality of the player decision profile the speed round is designed to build.

Decision

Perform a quality pass on all 200 seed questions with three interventions:

1. Context-starved questions (18 found): Add a grounding clause that narrows the scenario enough for a real opinion — org size, concrete stakes, a timeline, or a specific role. Keep total text under ~120 characters to fit the card UI comfortably (3 lines at text-lg in a max-w-md container).

2. Socially obvious questions (5 found): Replace with genuine dilemmas where both sides are defensible. The key test: would a reasonable, ethical person pick either option? If one side is "take credit for a colleague's work" or "cheat to win," the question fails.

3. Near-duplicates (7 found): Deactivate the weaker version via is_active = false rather than DELETE, preserving any existing response data that references the question ID.

Additionally, 4 questions were migrated from the binary swipe mechanic to scratch_card (4 options) because the scenario needed more than two positions to produce useful signal. The third/fourth options map to distinct driver signals, so every answer teaches something — unlike a "not sure" or "depends" option which would be low-signal noise.

Rationale

• Grounding context over abstract dilemmas: Speed round questions are answered in seconds with no rationale text. The question itself must carry enough context for a gut-level decision. Abstract questions like "How much of your strategy do you share publicly?" don't activate decision-making instincts — they activate "well, it depends" thinking, which is useless at speed.
• Every option must be a real position: A "not sure" / "maybe" / "depends" third option was considered and rejected — it's low signal and gives players an escape hatch that undermines the whole point of the speed round (forcing fast gut calls). Instead, where a third option is needed, it should be a distinct strategic position (e.g., "Open-source the core, monetize the tooling") that maps to a different driver signal.
• Deactivate over delete: Existing speed_round_responses rows reference question IDs via foreign key. Soft-deactivation preserves data integrity and allows reactivation if needed.
• Character budget: The swipe card renders question text at text-lg (~18px) in a px-6 container on a max-w-md (448px) card inside h-[65dvh]. At ~38 chars/line, 120 characters = ~3 lines, which fits comfortably. The longest existing production questions are ~112 chars. We target ≤120 for rewrites.

Alternatives Considered

• Add a "Depends" swipe-up option to all questions: Rejected. This would be the most common answer for context-starved questions, producing a profile full of "depends" signals with no directional information. The problem isn't missing options — it's missing context.
• Generate questions dynamically with AI based on player's industry/role: Considered for the future but premature now. The intake profile doesn't yet capture enough context to personalize scenarios, and LLM-generated questions would need the same quality criteria enforced at generation time.
• Delete duplicate questions entirely: Rejected due to FK constraints on speed_round_responses. Soft-deactivation is safer and reversible.

Discussion

The trigger was a playtester encountering "Making your core approach public would grow the community but weaken your competitive position" and not knowing how to answer because it genuinely depends on the org's stage and competitive landscape. This led to an audit of all 200 questions.

The main debate was around whether to add a third option vs. adding context. Adding context won because: (a) many of the problematic questions work fine as binary choices once the scenario is concrete enough, (b) a blanket third option creates an escape hatch that undermines the speed round's design intent, and (c) where a third option genuinely adds signal (like the open-source question), migrating to scratch_card with 4 distinct positions is better than a mushy middle.

The character budget constraint forced useful discipline — grounding context had to be ruthlessly efficient. "Your 3-person startup built a dev tool" adds 40 characters but completely changes the decision calculus compared to the abstract original.

Consequences

• 194 active questions remain (down from 200), still well above the 10-question session size
• Questions changed mid-flight may affect longitudinal comparisons for the ~18+5 rewritten questions — existing responses reference the old text but the question ID and tension pair remain the same
• The quality criteria established here (grounding context, no socially obvious answers, no pure self-report) should apply to any future question additions
• Future work: consider a question review checklist or linting step when adding new seed questions

Key files: - migrations/030_speed_round_question_quality.sql — the full migration (deactivations, rewrites, one insert) - migrations/013_speed_round.sql — original seed data for reference - src/components/speed-round/mechanics/SwipeQuestion.tsx — card UI that constrains text length