ADR-024: Judgment Layer Enhancement — 12 Behavioral Signals¶

Status: Accepted Date: 2026-04-12 Context: Sync captures WHAT drives decisions (8 primary drivers with EMA-weighted distributions) and WHEN behavior shifts (10 context triggers). But the judgment layer — the ability to model HOW someone thinks, not just what they choose — has gaps. A design review compared Sync's approach against document-based AI systems (Ryan Sarver's AI Chief of Staff, Meta's "Second Brain," Block's hierarchy-replacement model) and identified that Sync's forced-tradeoff approach generates richer judgment signal than any of these, but isn't fully exploiting the data it already collects. This ADR establishes 12 behavioral signals grounded in decision science research (Tetlock's superforecasting, Kahneman's Noise, Klein's naturalistic decision-making) that deepen the behavioral model without adding manual maintenance burden.

At a glance

What it decides: The behavioral model carries 12 judgment signals — how someone reasons, how well-calibrated and consistent they are, how they update — not just which driver they pick. They're collected now (cheaply) and pruned later once the data shows which ones predict.

Three data sources, zero new API calls — some signals extend the existing classify-driver prompt (reasoning style, attention), some are deterministic computations on data already collected (timing, dissent, persuadability), and one (self-awareness) adds only a quick-tap UI strip.
Biases are never labeled — overconfidence, anchoring, sunk-cost etc. emerge as patterns across the 12 signals rather than as a separate named taxonomy; there's no bias-free baseline to score against.
The signals compose into the trust thesis — self-awareness + calibration → meta-cognitive trustworthiness; + low noise + consistency → predictability; + rationale-choice match → trust.
"Collect now, separate signal from noise later" — the marginal cost of an extra nullable column is near-zero; the cost of missing historical data we later need is high. Watch: which combinations actually carry predictive weight.

flowchart LR
    P["classify-driver prompt"] --> S1["Reasoning style (1)"]
    P --> S4["Attention model (4)"]
    P --> S10["Rationale-choice match (10)"]
    D["Existing data<br/>(timing, choices, predictions)"] --> S5["Calibration (5)"]
    D --> S6["Timing (6)"]
    D --> S7["Persuadability (7)"]
    D --> S9["Dissent (9)"]
    D --> S11["Learning rate (11)"]
    D --> S12["Occasion noise (12)"]
    U["Quick-tap + milestone UI"] --> S2["Self-awareness (2)"]
    S2 --> T["Meta-cognitive trustworthiness → predictability → trust"]
    S5 --> T
    S12 --> T
    S10 --> T

The 12 signals draw from three sources — the existing classifier prompt, deterministic computation on data already collected, and one low-friction UI add — and a subset composes into the trust chain.

Decision¶

Add 12 new judgment signals to the player behavioral model, organized in 5 implementation phases. Zero new Claude API calls — all signals either extend the existing classify-driver prompt or are deterministic computations on existing data.

The 12 Signals¶

#	Signal	What It Captures	Source Discipline	Data Source
1	Reasoning Style	HOW they reason (6 modes: principled deductive, consequentialist, analogical, eliminative, intuitive, systems thinking)	Cognitive science, Klein RPD	Extend classify-driver prompt
2	Self-Awareness	Gap between self-model and behavioral model	Meta-cognition	Quick-tap reactions + milestone self-predictions
3	Arc Trajectory	How values shift under escalating pressure across campaign chapters	Behavioral economics	Profile snapshots per chapter
4	Attention Model	What scenario details they notice vs. ignore (6 categories: financial, interpersonal, political, temporal, ethical, technical)	Cognitive psychology	Extend classify-driver prompt
5	Confidence Calibration	Does self-reported certainty correlate with actual consistency	Tetlock (calibration/resolution)	decision_strength vs. prediction accuracy
6	Decision Timing Signature	Speed/deliberation patterns and variance	Klein (RPD, expert recognition)	Existing decision_time_ms
7	Persuadability Profile	Stand rate, who influences them, in what contexts	Social psychology	Existing post_discussion_actions
8	Peer Read Confidence	Can they accurately judge others' decisions	Theory of mind	New column on peer_prediction_edges
9	Dissent Profile	None-of-the-above rate + consensus deviation	Group dynamics	Existing choices data
10	Rationale-Choice Expression Match	Do stated reasons match revealed preferences	Self-deception research	Existing classify-driver response
11	Learning Rate	Rate of improvement in prediction accuracy over time	Tetlock (perpetual beta)	Existing prediction history
12	Occasion Noise	Within-person variability across repeated/similar scenarios	Kahneman (Noise)	Existing scenario repeat data

Self-Awareness Signal Design (Signal 2)¶

Two collection mechanisms:

Quick-tap reaction (every game, optional): Three icons beneath AI reasoning on the reveal screen: - ✓ Spot on — AI reasoning matched how they think - ✗ Not quite — AI reasoning didn't reflect their thinking - 💡 Interesting — AI showed them something new

Labels are intentionally internationally neutral — no idioms, no sports metaphors, no culturally-loaded emoji.

Milestone self-prediction (solo only, at games 10/25/50 and arc completion): User predicts their own driver profile before seeing the actual model. The gap between self-prediction and behavioral model = self-awareness score. This only appears in solo games to avoid blocking multiplayer flows.

Bias Philosophy¶

Cognitive biases (Kahneman/Tversky) are NOT labeled explicitly. They emerge as patterns across the 12 signal dimensions:

Bias	Detected Via
Overconfidence	Signal 5 (Confidence Calibration)
Anchoring	Signal 4 (Attention Model)
Confirmation bias	Signal 7 (Persuadability — do they ever update?)
Status quo bias	Signal 9 (Dissent — do they default to "safe" options?)
Sunk cost fallacy	Signal 3 (Arc Trajectory — do they double down on earlier choices?)
Framing effects	Signal 12 (Occasion Noise — same situation, different frame, different choice?)
Authority bias	Signals 7+8 (do they defer to high-sync peers regardless of context?)
Loss aversion	Signal 1 (Reasoning Style — consequentialists who always weight downside heavier)

The system detects the behavioral fingerprint of biases without needing a bias taxonomy. This avoids the problems of explicit labeling: redundancy with existing signals, implying biases should be "fixed" when they may be adaptive, and the philosophical quicksand of opposing biases (loss aversion vs. recklessness, deliberation vs. analysis paralysis).

Trust Thesis Chain¶

These signals compose into a chain that maps directly to the Beacon trust model:

Self-Awareness (2) + Confidence Calibration (5) → Meta-cognitive Trustworthiness Someone who knows their own biases and accounts for them is more trustworthy than someone who's unaware of their patterns.

Meta-cognitive Trustworthiness + Occasion Noise (12) + Consistency → Predictability Low noise + well-calibrated + consistent framework = reliably predictable decision-maker.

Predictability + Rationale-Choice Expression Match (10) → Trust A predictable person whose stated reasoning matches their revealed preferences is trustworthy. A predictable person who performs values they don't actually follow is a different kind of risk. (Note: signal 10 was originally named "Rationale-Choice Alignment (authenticity)"; renamed to a valuation-neutral name per ADR-078 — the low-match pattern reads as inauthenticity or epistemic humility depending on framing, so the name shouldn't pre-commit either. The internal storage key remains authenticity.)

Rationale¶

Why 12 signals, not fewer¶

Each signal captures a genuinely distinct dimension of judgment that doesn't decompose into combinations of the others. We considered additional signals (emotional regulation, cognitive load effects, moral foundations profiles) and rejected them as either unmeasurable without biometric data, requiring game design changes, or redundant with existing signals.

Why not label biases explicitly¶

Three reasons: (1) Each named bias would decompose into a combination of existing signals — overconfidence IS confidence calibration, anchoring IS attention pattern. Adding bias labels creates a redundant layer. (2) The "opposite" of most biases is another bias — there's no bias-free baseline to measure against. (3) The insight value comes from the behavioral pattern, not the label. "Your attention narrows to political dynamics under authority threat and your confidence drops" is more actionable than "you have anchoring bias."

Why zero new API calls¶

The classify-driver endpoint already receives the full scenario context, chosen option, and rationale text. Asking Claude to also classify reasoning mode and attention focus in the same response adds ~50 output tokens per game at zero additional latency cost. The remaining 10 signals are either deterministic computations on existing data (5 are completely free) or require only minor UI additions.

Why "collect now, separate signal from noise later"¶

With limited players and games, we cannot yet determine which signals carry the most predictive weight. The cost of collecting all 12 is negligible (no new API calls, one migration of nullable columns). The cost of NOT collecting a signal that later proves valuable is high — we'd need historical data that doesn't exist. Err on the side of collection; prune during analysis.

Why the quick-tap design for self-awareness¶

Alternatives considered: - Dwell time on AI reasoning: Rejected — reasoning is displayed inline (no "open" signal), dwell could mean reading or distraction, ambiguous data pretending to be behavioral signal. - Post-game survey: Rejected — too much friction, survey fatigue, self-report bias on every game. - Quick-tap + milestone self-prediction: Accepted — quick-tap is so low-friction that responses are gut reactions (closer to honest than considered), milestones are infrequent enough to not burden the flow, and the gap between self-prediction and model produces the actual self-awareness metric.

Why persuadability is dual-signal¶

High persuadability could indicate either lacking conviction (bad) or intellectual humility (good). The distinction comes from combining with other signals: high persuadability + overconfident calibration = caves under pressure; high persuadability + well-calibrated low-strength initials = rational updating on evidence. Collecting the raw signal and interpreting via combination is more honest than pre-judging persuadability as positive or negative.

Alternatives Considered¶

Explicit bias detection and labeling: Add a "bias profile" with named biases and severity scores. Rejected — redundant with signal combinations, philosophically problematic (no bias-free baseline), and risks implying users should eliminate biases rather than become aware of them.
Full self-assessment surveys at regular intervals: Have users complete periodic questionnaires about their decision-making style. Rejected — self-report data is exactly what document-based systems already capture. Sync's differentiator is behavioral data from forced tradeoffs, not self-attestation.
Biometric integration (heart rate, galvanic skin response): Would provide direct emotional state measurement. Rejected — requires hardware, excludes mobile/web players, and decision timing variance (Signal 6) already serves as a proxy for arousal/deliberation states.
Fewer signals, deeper analysis: Implement only the 5 highest-confidence signals and invest more in prediction integration. Rejected per "collect now, separate later" philosophy — the marginal cost of additional JSONB columns is near-zero, while the opportunity cost of missing data is high.

Discussion¶

This ADR was triggered by a design review comparing Sync's judgment capture against three external AI systems: Ryan Sarver's AI Chief of Staff (document-based memory with manual curation), Meta's "Second Brain" (searchable filing cabinet of user-authored content), and Block's hierarchy-replacement model (transaction data for routine decisions). All three systems know the user through what the user has fed them. Sync's forced-tradeoff approach generates behavioral data that the user cannot curate, which theoretically produces a more authentic judgment model.

The review confirmed this advantage is real but identified that Sync wasn't fully exploiting its own data. Five signals were identified initially: reasoning style, post-reveal reaction, arc trajectory, attention model, and confidence calibration. Through iterative discussion, this expanded to 12 as we realized that several judgment dimensions were already being collected (decision timing, persuadability, dissent, repeat-scenario consistency) but not aggregated at the profile level.

The bias discussion was particularly productive. The initial instinct was to add bias detection as a separate signal, but this was rejected when we realized that every named bias decomposes into patterns across existing signals. The deeper insight: the "highest version of yourself" isn't bias-free (impossible) — it's bias-aware. Sync doesn't measure whether you're biased. It measures whether your biases are stable, known, and accounted for. This connects directly to the trust thesis: Self-Awareness → Predictability → Trust.

The self-awareness signal (Signal 2) went through three design iterations: (1) post-reveal reaction with three options every game — rejected as too much friction, (2) dwell time measurement — rejected as ambiguous, (3) quick-tap buttons (optional, every game) + milestone self-prediction (solo only, at game thresholds) — accepted as the right balance of signal quality and UX burden.

Grounding in established research (Tetlock's calibration/resolution metrics, Kahneman's noise framework, Klein's recognition-primed decision model) provided both validation and new signals. Learning Rate (Signal 11) maps directly to Tetlock's finding that "perpetual beta" — commitment to self-improvement — is 3x more predictive of forecasting ability than raw intelligence. Occasion Noise (Signal 12) maps to Kahneman's finding that within-person variability is often a larger source of judgment error than bias.

Consequences¶

The player behavioral model expands from 3 profile dimensions (category_patterns, context_rules, option_preferences) to 15 (adding reasoning style, attention, calibration, timing, learning rate, occasion noise, self-awareness, persuadability, peer reading, dissent, authenticity, plus the existing 3)
player_decision_profiles table gains 11 new JSONB columns, all nullable with defaults — fully backward compatible
The classify-driver prompt expands to return reasoning_mode, reasoning_mode_confidence, and attention_focus alongside the existing driver + triggers — no new API calls
results table gains post_reveal_reaction and post_reveal_note columns for the self-awareness quick-tap
peer_prediction_edges gains predictor_confidence for peer read calibration
campaigns gains behavioral_trajectory for arc-level tracking
sync_score_dimensions gains calibration_score with initial weight 0.00 (tunable via sync_dimension_config)
The solo reveal screen gains a compact quick-tap reaction strip (optional, non-blocking)
The multiplayer peer prediction step gains a confidence selector
formatProfileForPrompt expands to include all new signal sections, giving the AI twin significantly richer context for predictions
migrateProfile in player-model.ts must handle missing fields gracefully for existing profiles
Future work: analyze which signal combinations are most predictive and weight accordingly; surface bias-adjacent insights on the insights page; integrate into delegation map confidence thresholds

Key files: - src/types/database.ts — All new types (ReasoningMode, AttentionCategory, CalibrationStats, BehavioralTrajectory, etc.) - src/lib/ai/player-model.ts — updatePlayerProfile, predictFromProfile, formatProfileForPrompt, migrateProfile - src/app/api/ai/classify-driver/route.ts — buildClassificationPrompt + response handling for reasoning mode and attention - src/lib/ai/learning-notes.ts — LearningNoteContext + prompt extension for post-reveal reaction - src/lib/sync/multi-dimensional-score.ts — Calibration dimension - src/app/api/campaigns/[id]/advance/route.ts — Trajectory computation - src/app/api/results/[id]/reaction/route.ts — New endpoint for quick-tap reactions - New migration file — All schema additions