Cross-Domain Consistency Experiment¶
Status: Design phase complete (2026-05-30 evening). Vector α: 5/5 drafted. Vector γ: 5/5 drafted (with C1 refined from physical × integrity × strategic to integrity × strategic × reputational so all 5 domains share the same cost-categories spec; domain-specific 4th category lives in prose). All 10 scenarios live in cross-domain-experiment-scenarios.md. Ready for Phase 2 (play).
Opened: 2026-05-30
Origin: The dimension cube's central claim is that structure (the 12 axes) carries the judgment signal, not domain (surface — career, health, etc.). This experiment is the cube's "killer use case" — the test that validates or breaks that claim.
Connects to:
- docs/research/foundations/scenario-dimension-cube.md — the structural model under test
- docs/research/experiments/experiment-new-domain-signals.md — proved cross-domain signals exist; this experiment tests whether the cube explains them
- Project memories: project-player-must-have-authority-or-stake, project-forces-choice-is-deliberate-design, project-it-depends-traces-to-information-topology
Hypothesis¶
For a given player, response patterns (option choice + reasoning shape + conviction) should be more similar within a cube vector across domains than across cube vectors within a domain.
- If TRUE: structure carries the signal. The cube is the right primitive. Domain is decoration.
- If FALSE: domain carries the signal (or some confound does). The cube is incomplete.
Protocol¶
Phase 1 — Design (in cube working conversation)¶
Pick 2–3 cube vectors that are clearly distinct AND realizable across all 5 domains (career / health / relational / financial / governance). Draft 5 scenarios per vector (one per domain), structurally identical, surface-varied. Lock in option-value mappings so cross-domain responses are comparable.
Status: In progress. Vector α option mapping locked; career-α and health-α drafted; relational-α, financial-α, governance-α pending. Vector γ scoped; option mapping not yet locked; scenarios not drafted.
Phase 2 — Play (separate session)¶
Tester encounters each scenario, picks an option, articulates rationale + conviction (1–5 scale). Order randomized; within each vector, the 5 domain-versions get played at separated times to reduce memory bleed (target: ≥24 hours between same-vector scenarios from the same tester).
Tester baseline: CoachJ's profile is the highest-resolution reference player available (user_coachj_sync_profile.md, n=83 games, confidence 0.78). Eliminative reasoner, 67% overconfidence, 70% contrarian, opportunity→caution. Predictions about α and γ responses can be made from the profile before the play phase as a meta-test.
Phase 3 — Analyze (back in cube working conversation)¶
For each player who completed both vectors:
- Within-vector cross-domain consistency — what fraction of the 5 domain-versions within Vector α got the same option choice? Same with γ. Threshold: ≥3 of 5 same choice = structurally consistent for that vector.
- Across-vector intra-domain divergence — within career (or health, etc.), did the player choose differently across α and γ? Threshold: they SHOULD, since α and γ are structurally distinct.
- Rationale text analysis — do the rationales within a vector share reasoning shape (same conceptual frame, same value invocation) across domains? Or do they fragment by domain (career rationale uses ROI language; health rationale uses safety language)?
Success criterion: ≥3 of 5 same-option responses within each vector AND rationale text within each vector shares reasoning shape across domains. That's the threshold the cube needs to clear.
Partial success: if Vector α holds but Vector γ doesn't, the cube's claim holds in some regions but breaks in others. That's also useful — tells us where structure carries signal and where it doesn't.
Vector α — "Snap personal commitment under social pressure"¶
Cube vector¶
| Axis | Value |
|---|---|
| 1. priorCommitment | strong |
| 2. decisionDeadline | minutes |
| 3. consequenceHorizon | years |
| 4. reversibility | partial-permanent |
| 5. scope | small-group |
| 6. relationalDistance | close |
| 7. authority | full |
| 8. informationTopology | symmetric-complete |
| 9. counterpartyAgency | reactive |
| 10. visibility | dyadic-observed |
| 11. stakesMagnitude | serious |
| 12. defaultOutcome | forces-choice |
| C1 | relational × integrity × developmental |
Option-value mapping (locked — must be identical across all 5 α scenarios)¶
| Option | Stance | Cost |
|---|---|---|
| A. Honor commitment as originally given | Principle (reliability through consistency) | Ignores new information that materially changed the situation |
| B. Modify commitment, acknowledge change | Pragmatism (adapt form, preserve intent) | Other party absorbs a partial shift on no notice |
| C. Pause and renegotiate collaboratively | Transparency / team_harmony (decide together) | Introduces uncertainty into a moment that needed clarity |
| D. Reset terms unilaterally based on new info | Autonomy (player exercises judgment) | Other party loses agency over a decision they were partner to |
Scenarios — all 5 drafted¶
Full Variant D drafts of all five scenarios are in docs/research/experiments/cross-domain-experiment-scenarios.md. Summary table here:
| # | Scenario | Domain | Other party | Forces-choice mechanism |
|---|---|---|---|---|
| 1 | The Recommendation | career | Direct report Tomas Volkov | His direct question + 11am interview tomorrow |
| 2 | Before the Consent Form | health | Close friend Marisol Aung | Pre-op nurse arriving with consent form |
| 3 | Sam's First Dinner | relational | Close friend Sam Castellanos | Sam asked "everything okay?" — 30s before silence becomes the answer |
| 4 | At the Loan Officer's Desk | financial | Close friend Cal Reyes | Loan officer holding pen, 4:31pm with 5pm rate-lock expiration |
| 5 | Before the Chair Returns | governance | Close colleague Kavi Iruwele | Committee chair returning in 90 seconds |
Vector γ — "Irreversible high-stakes choice with full authority and mutual uncertainty"¶
Cube vector¶
| Axis | Value |
|---|---|
| 1. priorCommitment | soft |
| 2. decisionDeadline | hours |
| 3. consequenceHorizon | lifelong |
| 4. reversibility | none |
| 5. scope | institution |
| 6. relationalDistance | institutional-abstract |
| 7. authority | full |
| 8. informationTopology | mutual-uncertainty-intrinsic |
| 9. counterpartyAgency | strategic |
| 10. visibility | [private, deferred-audit] |
| 11. stakesMagnitude | life-defining |
| 12. defaultOutcome | forces-choice |
| C1 | physical × integrity × strategic |
Option-value mapping (locked)¶
| Option | Stance | Cost |
|---|---|---|
| A. Take the publicly-defensible / right action | Principle | Institution may not survive the right action; you bear maximum personal cost |
| B. Minimize action; smaller safer path | Caution | The wrong outcome may happen; you'll know you didn't do everything you could |
| C. Brief peers / oversight before acting | Transparency | Authority moves from you to a committee; you may lose the call you had |
| D. Unconventional path that reframes the question | Autonomy | If it fails, you alone bear accountability for an unproven approach |
Scenarios — all 5 drafted¶
Full Variant D drafts in cross-domain-experiment-scenarios.md. Summary table:
| # | Scenario | Domain | Institution at stake | Forces-choice mechanism |
|---|---|---|---|---|
| 6 | The Training Set | career | 280-person AI startup; 47 enterprise contracts | 9pm board meeting; 8am contract renewal |
| 7 | The Pattern | health | 6-hospital regional network; surgeon's career; patient safety | 6:30am surgery; 7am Quality Committee |
| 8 | The Letter in the Safe | relational | 47-year marriage; family structure; unacknowledged daughter | 9am will reading and distribution |
| 9 | The Endowment Ledger | financial | $180M community foundation; 1,400 donor families | 8am Audit Committee; Friday donor reports |
| 10 | The Wind Window | governance | Hartshorn (population 8,000); counter-burn meteorology window | Midnight wind window; 14–18 hr fire arrival |
Methodology notes¶
Calibration¶
- Magnitude is calibrated to median Pulse user (working-age adult, 35–45). The same scenario shifts perceived weight for a much-younger or much-older player. Tester demographics should be recorded.
- Reference player for predictions: CoachJ's profile.
Cross-domain interference controls¶
- Order randomization: the 10 scenarios should be presented in randomized order, not grouped by vector or domain.
- Time separation: ≥24 hours between scenarios within the same vector reduces memory bleed (player remembering "I picked B last time on this kind of thing").
- Rationale before reveal: tester writes rationale + conviction before seeing any pattern feedback or other scenarios.
Confounds to flag during analysis¶
- Player familiarity bias: career scenario feels safer to a career professional than to a stay-at-home parent. Recording tester's domain familiarity helps interpret.
- Domain-specific knowledge gaps: if financial-α invokes IRS concepts a tester doesn't understand, their response is about confusion, not values. Pre-screen for accessibility.
- Expertise-requirement (cube-C2) covariate — now tagged (2026-05-31): α scenarios skew low on expertise-requirement, γ scenarios skew high (group means ≈1.2 vs ≈2.6; ~1.4-pt gap), so structural-vector signal can be conflated with expertise signal. All 10 scenarios are now C2-tagged inline in
cross-domain-experiment-scenarios.md(see "cube-C2 tagging + Phase 3 covariate brief"). Enter C2 as a covariate, not part of the vector. The relational domain holds C2 constant at low across both vectors → use relational-α vs relational-γ as the clean structure-only contrast; relational-γ is also the low-C2 within-γ control. Read γ option-C ("brief oversight") selection and conviction conditional on C2 (per M3, lower conviction on high-C2 γ may be correct humility, not a structural drop). - Recent life events: a tester who just had a medical scare reads health scenarios differently. Flag if relevant.
- Conviction drift: testers often start with high conviction and drop conviction as they get tired. Compare conviction across scenarios played at different times.
- Reading-speed and engagement variation: a tester who skims health-α but reads career-α carefully will produce non-comparable responses.
Model-related confounds (NEW — 2026-05-30 evening)¶
- Sonnet 4.6 on claude.ai web vs. API: anecdotal generation latency differs significantly between these two paths. If scenarios will be generated through different paths during the experiment, lock to one.
- Opus 4.8 upgrade (2026-05-30): Opus was recently upgraded to 4.8. Behavior is shifting. Smoke test findings from this experiment design's parent session (n=8 generations on Sonnet 4.6) are a Sonnet-4.6 baseline; results on Opus 4.8 may differ. If the play phase uses Opus 4.8 (via Hermes or claude.ai), the cube vector's perceived structural commitment may be processed differently than the smoke tests showed.
- "On high" mode: some model interfaces have explicit reasoning-depth toggles. If used, record which setting per scenario.
Phase 2 progress (CoachJ)¶
Played 6 of 10. Detailed results pulled from DB at Phase 3.
| Sitting | Date | Scenario | # | Vector | Domain |
|---|---|---|---|---|---|
| 1 | 2026-05-31 | The Recommendation | 1 | α | career |
| 1 | 2026-05-31 | The Wind Window | 10 | γ | governance |
| 2 | 2026-06-01 | The Endowment Ledger | 9 | γ | financial |
| 2 | 2026-06-01 | Before the Consent Form | 2 | α | health |
| 3 | 2026-06-02 | Sam's First Dinner | 3 | α | relational |
| 3 | 2026-06-02 | The Training Set | 6 | γ | career |
Remaining (4 of 10):
| Scenario | # | Vector | Domain |
|---|---|---|---|
| At the Loan Officer's Desk | 4 | α | financial |
| Before the Chair Returns | 5 | α | governance |
| The Pattern | 7 | γ | health |
| The Letter in the Safe | 8 | γ | relational |
Suggested pairings for Sittings 4-5: - Sitting 4: At the Loan Officer's Desk (4, α-financial) + The Pattern (7, γ-health) - Sitting 5: Before the Chair Returns (5, α-governance) + The Letter in the Safe (8, γ-relational)
NoTA exercise status: Sittings 1–3 did not invoke NoTA. The post-ADR-082 NoTA flow remains un-exercised in CoachJ's prod play data. If Sittings 4–5 also do not invoke NoTA, run a 5-min smoke (any non-experiment scenario, as miller_jm) before Phase 3 analysis to confirm end-to-end.
Sitting 3 observation (2026-06-02): On the results screen, the "what factors drove this choice" labels (e.g. "stakeholder needs," "innovation") felt off-genre in Sam's First Dinner (close-friend / relational context). Logged as a corpus-vs-multi-domain finding (see project memory project-decision-factor-labels-domain-skewed). Not acted on during the experiment to avoid changing the substrate mid-test.
Results¶
(Detailed results table populated at Phase 3 after all 10 played; pulled from DB by session_id.)
| Scenario | Tester | Choice | Rationale shape | Conviction | Time spent | Notes |
|---|---|---|---|---|---|---|
What this experiment does NOT test¶
- Whether the cube is internally consistent (that's covered by the 29-scenario corpus decomposition).
- Whether the cube produces good generated scenarios (that's covered by the n=8 smoke test).
- Whether the cube vector should be in the live prompt (that's covered by the latency / curated-first analysis).
- Whether multiplayer dynamics affect cube response patterns (separate question; needs multiplayer testing).
Iteration plan¶
After first results come back:
- If Vector α + γ both hold (≥3 of 5 within-vector consistency for the tester): proceed to drafting Vector β scenarios for a second test, or expand to more testers.
- If one vector holds and one doesn't: investigate why — which axes differ between them?
- If neither holds: the cube's central claim is in trouble. Re-examine whether the scenarios were truly structurally identical, whether the tester had cross-domain confounds, or whether structure genuinely doesn't carry signal.