Skip to content

ADR-028: Implicit Name-Bias Detection (Signal 13)

Status: Accepted Date: 2026-04-20 Context: AI-generated scenarios include named characters with diverse demographic associations. These names may subtly influence player decisions — a player might consistently side with or against characters whose names suggest certain backgrounds. We want to detect whether this is signal (consistent, directional bias) or noise (random variation), without changing the behavior we're measuring.

Decision

Add a 13th behavioral signal (name_bias_signal) that tracks correlation between character name demographics and player option selection. The signal is strictly back-end — it is never surfaced to users, never used for matchmaking, and never shared.

Character names are tagged at generation time with three demographic dimensions: - Name origin (linguistic/cultural origin of the name itself): anglo, east_asian, south_asian, latin, west_african, east_african, middle_eastern, slavic, nordic, ambiguous - Formality: formal (Dr. Harlow), informal (Jude), institutional (Big Russ) - Gender presentation: masc, fem, ambiguous

Each character is linked to the option they advocate for in the scenario, enabling correlation analysis.

Rationale

  • Observer effect: Surfacing bias detection would cause players to compensate, destroying the signal. This is consistent with Sync's core philosophy (ADR/identity-vs-process): we measure how people actually decide, not how they think they should decide.
  • Same-call extraction: Character name metadata is extracted in the same Claude call that generates the scenario, avoiding a second API call and ensuring the extraction is grounded in generation context.
  • Separate table over JSONB column: Character names are stored in scenario_character_names rather than a JSONB column on scenarios to keep scenario reads lean and make analytical joins cleaner.
  • New signal vs. folding into existing: Name bias is orthogonal to attention distribution (Signal 4 — what players notice) and occasion noise (Signal 12 — random variability). It detects systematic directional bias, which warrants its own signal.

Alternatives Considered

  • Surface the signal to users: Rejected. Would trigger observer effect and eliminate the signal. If the signal ever proves significant, it should inform scenario design (name selection) rather than label players.
  • Post-processing NLP extraction: Rejected. Adds latency, cost, and could miss the implicit demographic associations Claude already knows about the names it chose.
  • Fold into occasion noise (Signal 12): Rejected. Occasion noise measures random within-person variability. Name bias is a directional hypothesis — a player consistently favoring one demographic is not noise.
  • Use explicit racial categories: Rejected. The taxonomy describes name linguistic origins, not character or player race. This is more factual and less reductive.

Discussion

The key tension: is it ethical to track this at all? The reasoning is that the signal exists whether or not we measure it. By measuring it, we can (a) confirm it's noise for most players, which validates our scenario design, or (b) detect it in aggregate, which would inform how we choose names to minimize unintended influence. We are not labeling individuals — we're instrumenting the game to understand how naming choices affect decision patterns.

The minimum threshold for meaningful signal is total_scenarios_with_names >= 20 and signal_strength >= 0.15. Below this, any deviation from base rate is likely noise.

If the signal is ever surfaced, it should be aggregate and opt-in ("across all players, decisions shifted X% when...") rather than individual ("you appear biased"). The former is social science; the latter is an accusation from a game.

Consequences

  • Every AI-generated scenario now includes characterNames metadata in the generation response
  • The scenario_character_names table stores per-character demographic data linked to scenarios
  • updatePlayerProfile() performs one additional indexed DB read per game to fetch character names
  • The signal naturally populates only for new AI-generated scenarios — no backfill needed
  • If the hypothesis is disproven (no meaningful signal across players), the column can be ignored at zero cost

Key files: - migrations/072_name_bias_signal.sql — table + column definition - src/lib/ai/scenario-generator.ts — CharacterNameMeta type, prompt extension, validation - src/lib/ai/player-model.ts — updateNameBiasSignal() function and integration - src/types/database.ts — NameBiasSignal, ScenarioCharacterName types - src/app/api/sessions/start/route.ts — character name persistence on inline generation - src/app/api/scenarios/generate/route.ts — character name persistence on saveToDb