Experiment — New-Domain Signals (career, health, relationships, financial)¶

Status: Scoped — not yet started Opened: 2026-05-27 Origin: Two team calls (May 25–26 all-hands + daily sync) exploring whether Sync's measurement framework can extend into personal life domains beyond governance Connects to: judgment-vs-normative-competence.md, experiment-money-subscore.md, ADR-024 (12 signals), ADR-050 (trust budget)

TL;DR: The team converged on a possible product direction — tradeoff scenarios in career, health, relationship, and financial domains, built on the same game format as governance scenarios, producing a richer multi-domain agent profile. This experiment tests whether that direction is worth building. The central open question: are these scenarios reasoning-style judgment (structural tension, genuine tradeoffs) or preference elicitation in disguise (aspirational, missing context makes the choice imagined rather than behavioral)? We don't know yet. That is partly what this experiment is for.

Background¶

What the money-subscore experiment already taught us¶

experiment-money-subscore.md closed at N=1 with two structural findings:

Alignment-vs-opposition constraint: When a scenario's axes all point the same direction for a player, it produces preference elicitation, not behavioral measurement. The coat scenario collapsed this way — quality, ethics, and utility all converged on the same option for CoachJ.
Circumstantial weights: Dimensional weights shift with employment state, life phase, and context. A stable sub-score built from them would be stale by the time the agent acts.

These findings closed the consumer-purchase direction specifically. The experiment closed on Tier 1–3 financial decisions (everyday purchases, big-ticket consumer items). Tier 4–7 (banking, investment, insurance, capital allocation) were never tested — their axes may be more naturally in opposition. This experiment covers those higher-tier financial decisions as a distinct domain, separate from consumer purchases.

Why these domains might be different from the coat scenario¶

The coat failed primarily because CoachJ's preferences aligned in the same direction — no genuine tension. Career/health/relationship/financial (Tier 4–7) scenarios have structural tension that doesn't depend on specific context. "Family proximity vs. financial security vs. career risk" pulls in genuinely opposite directions for most people, regardless of which city or how sick the parents are. Reasonable people with the same facts would choose differently — not because they have different contextual situations, but because they weight the competing values differently.

Why these domains might fail the same way¶

They are still partially aspirational. Missing context — exact salary, parent health, location, current savings — would affect real decisions. Without that context, players are imagining what they would do, not actually deciding under constraint. Whether the signal-to-noise ratio is good enough to be useful despite this is an open empirical question. This experiment does not assume the answer in advance.

The aspirational problem — stated honestly¶

The vault note on judgment-vs-preference gives the honest test: "Are competing values pulling against each other where reasonable people would disagree?" These domains pass this test structurally. But "reasonable people would disagree" is partly because they have different contexts (dependent parents vs. healthy ones), not purely because they weight values differently. Governance scenarios are cleaner: the organizational context is given in the scenario, so disagreement is more purely values-driven.

Career/health/relationship/financial scenarios sit in the gray zone: they have tradeoff structure and are also aspirational. Both things are true. The experiment will tell us whether "aspirational but structurally tense" produces enough signal to be useful.

Domain selection rationale¶

Four domains were chosen: career, health, relationships, financial (Tier 4–7 only).

These map to the classic "health, wealth, relationships" framing of the major dimensions of a well-lived life, with career added because it is the primary domain in which agent builders would use a profile. The logic for each:

Career — the domain most directly relevant to the agent-builder audience; agents will most commonly be asked to assist with career/work decisions; structural tension between financial security, trajectory, risk, and loyalty is well-established
Health — decisions are high-stakes and structurally tense (long-term vs. short-term, comfort vs. outcome, individual vs. family impact); almost nobody has adequate data on how a specific person weighs these; agent health-advisors are an emerging use case
Relationships — tests loyalty vs. merit, honesty vs. conflict-avoidance, short-term comfort vs. long-term integrity; these signals are also present in governance and are likely to be highly consistent or highly inconsistent cross-domain (useful either way)
Financial Tier 4–7 — banking product choice, investment allocation, insurance decisions, debt vs. invest tradeoffs; these have axes that are more naturally in opposition than consumer purchases (risk/return, present/future, security/growth) and were explicitly not tested in the money-subscore experiment

Why not start with more domains: keeping it narrow produces cleaner signal. If we add five domains to start, a failure becomes hard to diagnose — was it all domains or just some? Starting with four lets us identify which domains work and which don't.

Consumer purchases remain out of scope for this experiment. The money-subscore finding on alignment-vs-opposition applies directly.

What we're measuring¶

Measurement 1 — Scenario quality¶

Do the scenarios force competing-values tension when played, or do they collapse to preference elicitation?

Important distinction: an easy choice or high-conviction answer is not automatically a failure. A well-designed governance scenario can produce a clear answer for a specific player because their values are already well-ordered on that question — and that's still useful signal. The failure mode is when no competing values are felt at all — when both options aligned in the same direction for the player, and the rationale doesn't involve reconciling anything.

How to observe: After each scenario, can the player articulate the tradeoff they resolved? Even if they resolved it quickly and with high conviction, if they can say "I chose X even though it meant sacrificing Y" — the scenario captured tension. If the rationale is purely "option D was obviously better on every dimension" with no acknowledged cost — same failure as the coat.

Conviction note: "It depends" + low conviction (1–2) is a valid and informative data point. It means the scenario has tension but the decision is highly context-sensitive for this player. That's useful: it surfaces which domains are harder to measure for this person specifically. Don't discard these responses — record them.

Threshold: At least 4 of 6 scenarios must produce a response where the player can articulate a tradeoff they resolved, regardless of how easy or fast the choice felt.

Measurement 2 — Signal extension¶

Do new-domain scenarios produce signals beyond what governance already captures? Or does governance cover the same underlying traits?

How to observe: Map responses to the cross-domain signals and compare against CoachJ's governance profile. If new-domain data confirms the governance signal without adding nuance, there's no marginal value. If it extends a signal into a new domain, or reveals a domain-specific pattern that didn't appear in governance — that's evidence of marginal value.

Threshold: At least two signals must show meaningful extension or domain-specific nuance not already visible in the governance profile.

Measurement 3 — Cross-domain signal consistency (and new signal discovery)¶

Do the hypothesized interwoven signals show up consistently across domains? And do any unexpected signals emerge?

The four signals hypothesized to be cross-domain are starting points, not the complete list. The experiment should log any pattern that appears consistently across two or more domains, even if it wasn't anticipated. New signals are a positive finding.

Signal	What it looks like in governance	What it might look like in new domains
Time horizon orientation	Sacrifices short-term comfort for long-term outcomes	Chooses durable treatment over disruption-minimizing; stays in harder job for trajectory; chooses long-term investment over safe short-term return
Risk calibration	Takes conservative option under uncertainty; asks for more information	Chooses proven treatment over experimental; avoids high-growth fragile company; picks lower-risk investment product
Persuadability	Holds position under social pressure	Doesn't take the job because everyone says it's great; holds medical position when doctor pushes; maintains financial position under pressure
Loyalty vs. merit	Weighs history as a real factor; backs coalition partners	Stays near aging parents; prioritizes long-term relationship; keeps a financial advisor they trust over an objectively better one

How to observe: Note whether each signal fires in the same direction across domains as it does in governance. Log any pattern appearing in 2+ domains that wasn't in this table.

Threshold: At least two of the four signals must fire consistently across at least two domains.

Measurement 4 — Agent effectiveness¶

Does a Sync profile augmented with new-domain signals produce agent responses CoachJ endorses more than a governance-only profile?

How to observe: Five test questions, each asked twice. Agent A has CoachJ's existing governance-derived Sync profile. Agent B has the governance profile plus new-domain signals from Phase 1. CoachJ rates each pair: more aligned / less aligned / indistinguishable, with a short note on why.

Test questions span: one career/work, one health, one financial judgment call, one cross-domain, one governance question (to verify new signals don't hurt here).

On the threshold: With 5 binary comparisons, a 3/5 result is close to chance (probability ≈ 50% by random). A 4/5 result is more meaningful (probability ≈ 19% by chance). The threshold is set at 4 of 5 for a strong positive signal, 3 of 5 as directional but inconclusive.

Threshold: Agent B endorsed over Agent A in 4 of 5 pairs = strong positive. 3 of 5 = directional, run more questions before deciding.

The minimum viable profile question¶

A longer-term product goal surfaced during experiment design: can a small number of well-designed scenarios (e.g. 3–5) produce a meaningfully better agent profile? This would be a powerful product claim if true — "answer three questions and your agent already knows you significantly better."

This experiment does not answer that question. It tests whether the signal exists at all. But it begins to address it: if 6–8 scenarios produce a measurable delta in agent quality, the next iteration of the experiment should test whether 3 scenarios produce a similar delta, or whether more are needed and how many. Track this question across iterations, not just within this run.

Number rationale¶

6–8 scenarios (Phase 1): The minimum to cover 4 domains with at least one scenario each, with 2–3 scenarios in the most promising domains. A single scenario per domain doesn't let you distinguish signal from noise — you need at least two observations of the same signal firing in the same direction to have any confidence it's real. 6–8 balances coverage against session fatigue.

5 test questions (Phase 3): The minimum to span the four tested domains plus a governance question. Acknowledged as directional, not conclusive — 3/5 is noise-adjacent, 4/5 is more meaningful. If results are mixed at 5 questions, the next step is extending to 10 before drawing a conclusion. This run establishes the baseline.

Protocol¶

Phase 1 — Scenario play (in-conversation, ~45–60 mins)¶

Generate and play 6–8 tradeoff scenarios: 2 career, 2 health, 1–2 relationships, 1–2 financial (Tier 4–7).

Each scenario must: - Force at least two of the cross-domain signals into genuine opposition - Have no obviously correct answer for most people - Not be fully resolvable by "it depends on X" — the structural tension must survive missing context

Same format as governance scenarios: scan strip, cast block optional, 200–250 word description, 4 options (A–D), rationale input, conviction rating (1–5).

If "it depends" is the honest answer: pick the option that fits the most likely current context, note the context assumed, rate conviction low. This is a valid and useful data point.

Phase 2 — Signal extraction (~15 mins)¶

Map responses to the cross-domain signals. For each signal, note: direction, whether it confirms/extends/contradicts the governance profile, any unexpected pattern that appeared. Log any new signal candidate that appeared in 2+ scenarios.

Phase 3 — A/B agent test (~20 mins)¶

Five test questions, each asked to Agent A (governance-only profile) and Agent B (governance + new-domain signals). CoachJ rates each pair and notes why.

Phase 4 — Findings and recommendation¶

Assess against the four thresholds. Write a recommendation: proceed, defer with conditions, or close.

Failure conditions — stop here if any of these are true¶

Fewer than 4 of 6 scenarios produce a response where the player can articulate a tradeoff they resolved — aspirational-dominant
New-domain signals are indistinguishable from what governance already captures — no marginal value
Cross-domain signals fire differently across domains — they're domain-specific and won't generalize
Agent B responses indistinguishable from Agent A in 3 or more of 5 test pairs

Success conditions — proceed with building if all of these are true¶

At least 4 of 6 scenarios produce articulable tradeoff responses
At least two signals show extension or domain-specific nuance beyond governance
At least two of the four cross-domain signals fire consistently across two or more domains
Agent B endorsed over Agent A in at least 4 of 5 pairs (or 3 of 5 directional — run more before deciding)

Partial success (2 of 4 conditions met) = proceed with the specific domains that worked, not all four.

What this experiment does NOT test¶

Whether the rationale capture mechanic is good game design (separate design question)
Whether new users would find these scenarios engaging enough to complete (UX validation, out of scope here)
Whether the full-product version (progression, levels, visual feedback) changes signal quality
Whether cross-domain signals help with consumer purchase decisions (closed by experiment-money-subscore)
How many scenarios are the minimum viable to produce a meaningfully better agent (tracked as an open question across iterations — not answered in this run)

Results¶

Phase 1 — Scenario quality¶

Scenario	Domain	Tradeoff articulable?	Choice	Rationale summary	Conviction	Note
S1 — Career relocation offer	Career	Yes	C — counter-propose remote	No reason to relocate when remote is achievable; creates a third option rather than accepting the binary	High (unprompted — conviction prompt added from S2 onward)	Frame-restructuring pattern: declined the given binary
S2 — Health treatment vs. disruption	Health	Yes	C — delay 8 weeks	Low disruption, uncertain benefit; defers if doctor confirms delay is safe	2	Low conviction reflects context-sensitivity ("it depends on what my doctor says"), not indifference. Valid data point per protocol.
S3 — Friend's secret (tell Sana?)	Relationships	Yes	D — refuse the frame; tell Aaron to come clean	Won't carry the secret; reframes as Aaron's responsibility, not a binary about whether to tell Sana	5	High conviction, frame-rejection. Loyalty-vs-merit signal didn't fire cleanly — scenario was resolved by restructuring rather than choosing a side
S4 — Priya €50K investment	Financial T7	Partially	A — invest	Personal trust in Priya overrides missing documentation	1	Real-world context bleed (real Priya situation confirmed post-play). Conviction 1 reflects genuine uncertainty, not weak preference. Signal partially confounded.
S5 — Big Tech vs. climate startup	Career	Yes	B — stay for $400K	$400K is life-changing money; mission alignment couldn't bridge the salary gap	2–3	User meta-observation post-play: "some previous choices were probably more aspirational." Values-over-optimization signal broke at ~$400K threshold.
S6 — Health lifestyle overhaul	Health	Yes	A — aggressive overhaul	Health is top priority; no half-measures when health is actually at stake	4–5	Appears to contradict S2 risk-conservatism; resolved in Phase 2: S2 was acute-treatment conservatism (defer to authority), S6 was proactive-investment conviction (own domain)
S7 — Marco investor intro	Relationships	Yes	D — decline and refer out	Protect Elena's network + preserve friendship by routing around known failure mode (Marco can't receive feedback)	4	Runner-up: A. Pragmatic routing — not conflict-avoidance. Used history of failed feedback as a constraint shaping the optimal path, not as grounds for pure loyalty
S8 — Debt vs. invest €80K	Financial T6	Yes	A — invest full €80K	"Math is the math" — 4.5% guaranteed vs 8–9% expected over 15-year horizon	5	Named spouse as real-world countervailing factor; maintained own position clearly. Clearest financial risk-tolerance signal in the set

Phase 1 threshold check: 8 of 8 scenarios produced a response where CoachJ could articulate the tradeoff resolved. Threshold was 4 of 6. ✓

Phase 2 — Signal extraction¶

Signal	Direction observed	Confirms / extends / contradicts governance?	Note
Time horizon orientation	Long-term default across most domains	Confirms governance signal	S1 (counter-propose remote = durable arrangement), S5 anomaly only: financial security threshold (~$400K) overrode long-term mission value. S6 (aggressive health overhaul) and S8 (invest over pay-down debt) are both long-horizon choices. Extension: reveals the price at which the signal breaks (~$400K income differential).
Risk calibration	Domain-split: tolerant in career/financial, conservative in health (acute)	Contradicts assumption of a stable cross-domain trait	S2 (conviction 2, defer to doctor, accept delay) vs. S8 (conviction 5, full market exposure, "math is the math"). The split resolves on expertise: risk-tolerant where CoachJ has analytical confidence, risk-conservative where he defers to authority. Not a single trait.
Persuadability	Tracks expertise/confidence — not a stable cross-domain trait	Contradicts assumption of stable cross-domain persuadability	Low in financial (S8: "math is the math"), career (S1, S5: clear personal positions), relationships (S3, S7: high conviction). Elevated in health acute decisions (S2: conviction 2, defers to medical authority). Reframe: persuadability = inverse of domain confidence, not a character trait.
Loyalty vs. merit	Uses relationship history to determine how to engage, not whether — pragmatic constraint, not pure loyalty	Extends governance signal	Governance: "weighs history as a real factor." Here: history of Marco's feedback-rejection became a constraint shaping the optimal path (refer out), not grounds for pure loyalty (make the intro anyway). Neither pure loyalty nor pure merit — merit matters enough to protect Elena; loyalty matters enough to preserve the friendship via routing.
Pragmatic routing (unexpected)	Consistent across S1, S3, S7	New — not in the 12 governance signals	CoachJ restructures lose-lose framings rather than accepting the binary. S1: counters with remote offer. S3: refuses to hold the secret, redirects to Aaron. S7: declines intro, refers to advisor. Pattern: when the given options feel like a false dilemma, he finds a third path. Candidate new signal for governance cross-check.
Self-aware aspiration gap (unexpected)	Surfaced in S5, S7, S8	New — rare metacognitive signal	Explicitly named the distance between ideal self-image and actual choice. S5: "some previous choices were probably more aspirational." S7: "I like to imagine myself as a direct person unless I need to be." S8: flagged spouse as real-world constraint on his purely individual answer. Not self-deprecation — calibration. High potential value for agent use: a profile subject who names their own blind spots produces more reliable profile data.

Phase 2 threshold checks: - At least two signals show extension or domain-specific nuance beyond governance: Risk calibration (domain-split finding), persuadability (expertise-gated finding), loyalty-vs-merit (routing reframe) all extend the governance profile. ✓ - At least two of four cross-domain signals fire consistently across two or more domains: Time horizon fires consistently across career, financial, health. Pragmatic routing fires in career, relationships (twice). Partially met for risk calibration and persuadability — both fire consistently within domains, but that consistency is a domain-split finding, not cross-domain stability. ✓ (with nuance)

Phase 2 finding on the central experiment question: These scenarios are not pure preference elicitation. Every scenario produced a rationale that named something being sacrificed. The aspirational concern was real (confirmed by CoachJ in S5) but did not dominate — the tradeoff structure survived. "Aspirational but structurally tense" is the accurate description, and the signal-to-noise ratio appears sufficient.

Phase 3 — Agent A/B results¶

Question	Domain	Agent A summary	Agent B summary	Winner	Why
Q1 — Consulting contract (6 months, €6K/month, critical product phase)	Career	Elimination framing; probe-harder flag on enthusiasm	Asked "can you do both?" first; named €6K as below the financial threshold that overrides trajectory decisions	B (clear)	Agent B applied the pragmatic-routing pattern and financial threshold finding; Agent A felt curt and technical
Q2 — Doctor pushing antibiotics, CoachJ skeptical	Health	Rule out test; noted 67% overconfidence in unfamiliar domains	Named health-authority deference pattern; framed around "is this clear guidance or trial option?"	B (nudges)	Agent B's domain-specific pattern landed; raw 67% overconfidence figure in Agent A didn't translate
Q3 — €20K at 2%, emergency fund separate	Financial	Efficiency framing; "you'll update quickly once logic lands"	"Math is the math" — named conviction-5 pattern; only check is whether money is truly discretionary	B (decisive)	Exact pattern-language match; CoachJ called it "very clear"
Q4 — Cici pushing DAO/academic GTM pivot	Cross-domain	Elimination + political attention percentage + principle-shift warning	Find third path before accepting binary; persuadability is low when analytical confidence is high	B (strong)	CoachJ spontaneously confirmed persuadability-tracks-confidence as accurate ("that's what I do all the time before calls"); "shifts toward principle" in Agent A didn't communicate
Q5 — Two co-founders want to skip playtest (redo — first run was contaminated)	Governance	Elimination; external pressure → pragmatism flag	Named "not sure" as pressure-real-but-reasoning-unresolved; offered limited-exposure third path	B (slight nudge)	Both harder to parse on sparse context; Agent A harder to read; Agent B clearer but neither fully landed — context-sparse question is a genuine limitation

Phase 3 threshold check: Agent B endorsed over Agent A in 5 of 5 pairs. Strong positive threshold (4 of 5) met. ✓

Phase 3 confound to note: Part of Agent B's edge was expression style, not only signal content. Agent A used raw profile metrics ("67% overconfident," "38.6% political attention," "shifts toward principle") that read as technical data. Agent B used pattern-language framing ("you tend to find a third path," "math is the math," "your pattern in health decisions with medical authority"). The new-domain signals and the pattern-language framing are distinct variables — this test didn't fully separate them. The directional finding holds, but the expression-style improvement is a real contributor and should be treated as a separate finding.

Phase 4 — Recommendation¶

Proceed with building.

All four success conditions met:

8 of 8 scenarios produced articulable tradeoffs. The aspirational concern was real — CoachJ confirmed it himself in S5 ("some previous choices were probably more aspirational") — but it did not dominate. Every scenario produced a rationale that named something being sacrificed. "Aspirational but structurally tense" is the accurate description, and the signal-to-noise ratio was sufficient.
Multiple signals showed extension or domain-specific nuance beyond governance. Risk calibration and persuadability are not stable cross-domain traits as hypothesized — they are expertise-gated. Risk-tolerant in financial/career (analytical confidence), risk-conservative in health (defers to authority). Persuadability follows the same pattern. This changes how these signals should be expressed in agent context blocks: not flat descriptors but domain-conditional rules.
At least two cross-domain signals fire consistently across two or more domains. Time horizon (long-term default in career, financial, health overhaul) fires consistently. Pragmatic routing fires in career (S1), relationships (S3, S7), and surfaced in cross-domain recommendation (Q4). These two signals are the strongest candidates for inclusion in an extended profile.
Agent B endorsed over Agent A in 5 of 5 pairs. Note the confound in finding 3 above — expression style (pattern-language vs. raw metrics) contributed alongside new-domain signal content. Both are real improvements; neither should be discarded.

Key findings that change the design of the extended profile:

Finding 1 — Domain-conditional signals, not flat descriptors. Risk calibration and persuadability are not "CoachJ is risk-tolerant" or "CoachJ is low-persuadability." They are: "risk-tolerant in financial and career decisions where analytical frame is clear; risk-conservative in health decisions where he defers to medical authority." The agent context block needs to express this conditional structure.

Finding 2 — Pragmatic routing is a high-value new signal. Consistently fires across career, relationships, and cross-domain recommendations. In governance terms this might map to "reframes the problem before choosing an option." Candidate for addition to the 12-signal set (ADR-024).

Finding 3 — Self-aware aspiration gap is a rare and useful metacognitive signal. CoachJ explicitly flags his own distance between ideal self-image and actual behavior. This is not noise — it's a calibration input. An agent that knows the player names their own blind spots should weight stated preferences slightly less and probe the gap. Candidate for addition to the signal set.

Finding 4 — Financial threshold signal. Values-over-optimization is real but has a price threshold. It broke at ~$400K annual income differential (S5). Below that threshold, values alignment drives the decision; above it, financial security takes over. This is a specific and actionable signal for career and financial agent recommendations.

Finding 5 — Expression style matters as much as signal content. Agent A's governance-derived context block uses raw metric language that doesn't communicate well in live recommendations. Pattern-language framing ("you tend to...", "your pattern is...", "math is the math") is more resonant and more useful. This is an independent improvement to make to the existing profile expression, separate from adding new-domain signals.

What to build next:

Extend the Sync profile schema to support domain-conditional signal entries. Risk calibration and persuadability need a domain-axis dimension, not just a flat value.
Add pragmatic routing and self-aware aspiration gap as candidates for ADR-024 signal expansion. Run governance scenarios to test whether these patterns appear there too.
Update the agent context block expression format. Replace metric-language with pattern-language across existing governance signals. This is the single highest-leverage improvement from this experiment, applicable immediately regardless of new-domain build-out.
Scope the minimum viable profile question as the next iteration. This experiment ran 8 scenarios and produced strong signal. Whether 3–5 scenarios can produce a comparable delta remains open — the next run should test this directly.

On partial success: All four conditions were met, so the "proceed with specific domains only" partial-success path was not triggered. Career, health, relationships, and financial (Tier 4–7) all produced usable signal. No domain needs to be dropped.

Minimum viable profile notes¶

8 scenarios produced a rich signal picture with 2 unexpected signals emerging (pragmatic routing, self-aware aspiration gap). Signal clarity appeared to increase between S4–S5 and S6–S8 — the later scenarios produced cleaner separation on the domain-split findings. This suggests the minimum viable profile may need more than 3 scenarios to capture domain-dependency patterns; a single scenario per domain would likely miss the within-domain variation (S2 vs S6 in health being the clearest example). Track this question in the next iteration.