ADR-010: Decision Principles Synthesis Layer — Superseded¶

Status: Superseded (by agent-connector summary pipeline; see 2026-05-06 update below) Date: 2026-03-16 (original) / 2026-05-06 (status update) Context: Considered adding a persistent "decision principles" layer that synthesizes all player rationales every 5 games into 3-5 durable principles (e.g., "prioritizes relationship preservation over optimal outcomes under time pressure"). This would serve as the DTA's identity layer, replacing the initial intake prior as it ages out.

2026-05-06 status update¶

Closing this as Superseded, not Accepted-and-built. The narrative-synthesis job ADR-010 proposed has been done in a different shape by the agent-connector summary pipeline — but it intentionally routes around ADR-010's core question rather than answering it.

What shipped instead (src/lib/agent/summary.ts, agent-connector-brief.md §10.4):

LLM (Haiku 4.5) generates summary_text (2–3 sentences) and agent_context_block (~80–120 word system-prompt paragraph) on player_decision_profiles.
Regenerates every 3rd game, debounced 10 minutes, gated on model_confidence ≥ 0.15.
Drops into agent system prompts as second-person context for downstream LLM consumers.

Why this isn't ADR-010: the shipped pipeline feeds the LLM the distilled structured profile (src/lib/agent/distill.ts) — 12 behavioral signals + category tendencies + context rules — not raw rationales. The LLM does prose translation, not pattern discovery. ADR-010 deferred specifically because pattern discovery from rationales was building on the weaker signal (rule engine 70.2% vs. Claude 29.8% in simulated disagreements). The shipped pipeline sidesteps that objection by making the LLM consume the strong signal as input.

Where ADR-010's underlying question now lives: the rule-vs-Claude head-to-head comparison was never productionized — the in-game prediction loop short-circuited Claude when the rule engine was confident, so head-to-head data was never captured on the cases the rule was most opinionated about. Fixed 2026-05-06 in src/app/api/ai/multiplayer-predict/route.ts (always call both) and src/app/api/sessions/complete/route.ts (persist ai_rule_prediction + ai_claude_prediction to results). Going forward every multiplayer round captures both predictions; the comparison can finally be tested without the selection bias that shaped the original 70.2/29.8 simulation finding.

Early real-user signal (n=16 with both predictions populated, solo flow only, pre-fix): Claude won 3/4 disagreements; rule won 0/4. Sample is too small to be conclusive but is directionally opposite to the 2026-03 simulation finding. Worth tracking once the multiplayer fix accumulates data.

The principles-as-UI-surface variant (player-facing list of 3–5 durable principles they can read and dispute) is a distinct unshipped feature, not what was built. If someone wants to revisit it later, file a fresh ADR — don't reopen this one.

Trigger for future revisit: if rule-vs-Claude head-to-head data shows the rule engine winning ≥55% of disagreements after ≥30 real-user disagreement events, the original premise (behavioral patterns are the stronger signal) is vindicated and a behavioral-only identity layer becomes worth considering. If Claude is winning, no further work needed — the structured-signal-fed LLM summary is the right architecture.

Original ADR (2026-03-16)¶

Decision (original)¶

Hold until we have accuracy data from the now-working rule engine. Revisit after 10+ games with the rule engine live.

Rationale¶

Four factors argued for deferral:

Rules beat Claude in disagreements. Simulation testing showed the rule-based prediction engine was correct 70.2% of the time in disagreement cases vs Claude's 29.8%. A principles layer built from Claude's learning notes would build on the weaker signal.
Zero real-world data. The rule engine was broken in production until Week 10 (RLS bug — player_decision_profiles updates were silently failing). We have no data on how the two-stage system performs for real users with functioning profiles.
Flat driver distributions cap accuracy. CoachJ's governance drivers were nearly flat (autonomy 31%, pragmatism 27%, transparency 22%, team_harmony 20%). When a player doesn't have strong dominant patterns, no system improvement will dramatically increase prediction accuracy. Principles might not help where behavioral patterns are inherently ambiguous.
Input quality just improved. Rationale integration, comparative conviction, and keyword noise fixes all shipped in Week 10. Future learning notes will be built from higher-quality data, and we don't know if that changes the behavior-vs-notes equation.

Key Metric for Revisiting¶

If Claude starts winning disagreements at 45-50%+ (up from 29.8%), narrative context adds real value and principles are worth building. If rules still dominate disagreements after 10+ real-user games, behavioral patterns alone are sufficient and the principles layer is unnecessary complexity.

Alternatives Considered¶

Build it now and A/B test: More data-driven but adds engineering cost for a feature that may not improve predictions.
Lightweight version (just extract top-3 rationale themes): Lower effort but still consumes tokens for synthesis without evidence it helps.
Wait for digital twin infrastructure: Principles would be part of the DTA's "playbook" — building them now without the playbook architecture means rework later.

Consequences¶

The DTA's "identity" remains purely behavioral (driver distributions + context rules + keywords) until this is revisited.
Learning notes accumulate but are only used as prompt context (last 50 games), not synthesized into durable abstractions.
The rule engine's accuracy is the gating metric — tracking it is essential.

How We Got Here¶

The idea of a "decision principles" layer came from observing how Claude's memory system works in consumer products (Claude Projects, ChatGPT custom instructions). After enough interactions, these systems build up a model of who the user is — not just behavioral patterns but synthesized principles like "this person values directness over diplomacy" or "prioritizes long-term team health over short-term wins."

The proposal: every 5 games, synthesize all the player's rationales into 3-5 durable principles. These principles would serve as the DTA's "identity layer" — a stable understanding of who the player is that doesn't shift with every game. As the intake profile ages out (speed round responses weighted at 0.4x become noise after 50+ full games), the principles would replace it as the DTA's prior.

The investigation started optimistically but four findings converged to argue for deferral:

Finding 1: The rule engine was secretly winning. Simulation data showed that when the rule engine and Claude disagreed about a prediction, the rule engine's behavioral patterns were correct 70.2% of the time. Principles would be built from Claude's narrative reasoning — the weaker signal. Building an identity layer on the less accurate system felt backwards.

Finding 2: We were flying blind. The RLS bug (Week 10) meant player_decision_profiles had never actually updated in production. Every user had games_analyzed: 0. The rule engine had never fired in a real game. All performance data was from simulations — no real-world validation. Building a new layer on top of an untested foundation was premature.

Finding 3: CoachJ's profile was nearly flat. The first real profile data showed governance drivers almost evenly distributed (pragmatism 43%, caution 19%, principle 18%, team_harmony 20%). When a player's patterns are genuinely diffuse, no amount of narrative synthesis will create clarity that doesn't exist in the behavior. Principles would either be misleadingly specific ("pragmatism-driven") or uselessly vague ("context-dependent").

Finding 4: Input quality was about to change. The same week we considered building principles, we shipped: rationale integration (learning notes now see what the player actually said), comparative conviction (decision strength reflects how clear the choice was, not how good it was), and keyword noise fixes (rationale words no longer pollute attractors). Future learning notes would be built from fundamentally better data. Building principles from the old, noisy data and then rebuilding from the improved data was wasted work.

The deferral metric (Claude winning disagreements at 45-50%+) was chosen because it directly tests whether narrative context adds prediction value over behavioral patterns alone. If it does, principles are worth the investment. If not, the behavioral system is sufficient and principles are intellectual overhead.

This ADR connects to ADR-011: the delegation map concept may ultimately supersede the need for a principles layer entirely, since per-context confidence provides a more operational form of "identity" than synthesized principles.

Key files: - src/lib/ai/learning-notes.ts — current learning note generation - src/lib/ai/player-model.ts — current behavioral profile (what would be augmented)