Design Brief: Open Questions from March 23 All-Hands¶
Date: 2026-03-29 Source: Beacon All-Hands Weekly — March 23, 2026 Participants: Jonathan (CoachJ), James, Amin, 08 (Razvan)
This brief captures 9 open design questions that emerged from the all-hands. Updated 2026-03-31: Items 2, 3, 5, 6, 7 have been shipped. Item 1 has a standalone design brief. Remaining open items: 4 (monitor), 8 (deferred), 9 (deferred).
Cluster A: Identity & Measurement Scope¶
These questions affect what the game measures and who the DTA represents.
1. Personal vs. Business Scenarios¶
The observation: CoachJ's profile says he's pragmatic and risk-averse — but only in business contexts. In personal life, he explores legal gray areas, takes investment risks, and is far less cautious. The game currently generates only organizational/workplace scenarios.
The question: Should the scenario generator include personal and relational dilemmas alongside business ones?
Trade-offs: - Yes, include personal: Creates a fuller picture of the person. The DTA could eventually represent the whole person, not just the work-persona. Useful if Beacon expands beyond DAO governance. - No, keep business-only: The DTA is explicitly a governance proxy (per ADR-013). Personal decision-making is a different domain. Mixing them muddies the profile — "risk-tolerant in investing" doesn't mean "risk-tolerant in DAO treasury management." - Middle ground: Add personal scenarios as a separate optional track that feeds a parallel profile. Players can see their "work self" vs. "personal self" divergence without contaminating the governance profile.
Depends on: Nothing — can be decided independently.
Who should weigh in: Full team. This is a product scope decision.
2. Scenario Presupposition Problem¶
The observation: The Innovation Team Revolt scenario assigned CoachJ "a risk-averse micromanagement style" as part of the role setup. He said: "I wouldn't have that, but here we are." The scenario's framing forced him into a personality that isn't his.
The question: Should scenarios describe situations and let the player's response reveal their style, rather than assigning personality traits?
Trade-offs: - Ban personality presuppositions: Scenarios describe the situation, the stakeholders, and the decision — never the player's management style or personality. The player's choice reveals their style. This is cleaner diagnostically. - Allow them selectively: Some scenarios are specifically designed to test how you respond when you're cast in an uncomfortable role. "What would you do if you were the micromanager?" could reveal interesting things. But this should be the exception, not the default, and should be flagged explicitly. - Compromise: The scenario prompt already has strong option-label rules. Add a similar rule: "Do NOT describe the player's personality, management style, or emotional state. Describe only the situation, the stakeholders, and the decision to be made."
Recommended action: Prompt update — low effort, high signal improvement. This is similar to the fix we already made for AI-era realism.
Depends on: Nothing.
3. The "None of the Above" Problem¶
The observation: CoachJ encountered a remote work scenario where every option felt wrong. He's 100% pro-remote and none of the four options reflected that. He picked the least-bad option and caveated in his rationale: "Really though, I would say let's all go remote."
The question: Should there be a structured "None of the above — here's what I'd actually do" option?
Trade-offs: - Add it: Captures authentic preferences when the option space is wrong. The free-text response becomes high-value training data for the DTA. Prevents forced choices from polluting the profile. - Don't add it: Everyone picks "None" to avoid committing. The whole diagnostic value of the game depends on forcing a choice among constrained options. Real governance has constrained option sets too — you can't always propose a fifth option. - Middle ground: Allow "None" but require a structured response: "What would you actually do?" + "Which of the four options is closest to what you'd do?" This captures the authentic preference while still getting a ranked signal from the constrained set.
Risk to watch: If "None" is too easy to reach for, it becomes an escape hatch from hard decisions. The game's value is in the forcing function. Maybe limit to 1 "None" per 5 games, or only in solo mode.
Depends on: Nothing — but implementation should be informed by ADR-015 (conviction). A "None" with a strong rationale is high-signal; a lazy "None" is noise.
Cluster B: Trust & Social Dynamics¶
These affect how the multiplayer game captures interpersonal dynamics.
4. Self-Reporting Bias Mitigation¶
The observation: James raised that survey self-reports don't match actual behavior. The "performative higher self" problem (ADR-013/015) is a specific instance — CoachJ chose against his instinct to present his aspirational self.
The question: Beyond conviction weighting (ADR-015), are there other design levers to mitigate self-reporting bias?
Existing mitigations: - Speed round captures faster, more reflexive responses at 0.4x weight - Decision_strength tracks behavioral hesitation (switching options, deliberation time) - Rationale analysis can detect hedging language - EMA smoothing filters session-to-session noise
Possible additional levers: - Implicit behavioral signals: Track which option the player hovers over first, how long they spend reading each option, whether they scroll back to re-read. These are less gameable than the final choice. - Consistency checks: If a player's solo choices and multiplayer choices diverge significantly in the same category, that's a signal of social desirability bias in multiplayer. - Scenario pairs: Present the same core dilemma twice (months apart, different framing) and measure consistency. High consistency = authentic signal. Low consistency = context-dependent or performative.
Depends on: ADR-015 (conviction weighting). This cluster of mitigations works together.
Recommended action: Monitor for now. The existing mitigations may be sufficient. Revisit after 100+ games of aggregate data when we can measure actual divergence rates.
5. Retrospective Phase Design¶
The observation: James argued that in agile practice, retrospectives are where the deepest learning happens — understanding why people thought what they thought. He suggested the post-decision reveal and discussion phase may be more valuable than the decision itself for trust-building.
The question: How much design investment should the multiplayer debrief phase get?
Current state: After reveal, players see results and can discuss informally (voice/chat outside the game). No structured in-game discussion or follow-up.
Possible investments: - Structured rationale sharing: Each player's rationale is displayed in the reveal phase (currently only shown for the current player's own rationale) - Reaction prompts: "Whose rationale surprised you?" / "Whose reasoning would you want to hear more about?" - The Stand/Revise/Delegate mechanic (ADR-016): This IS the structured retrospective — it captures what changed after discussion - Recording integration: If the discussion happens on a call, offer a way to attach the recording/transcript to the session for DTA ingestion
Depends on: ADR-016 (delegation as game action). The Stand/Revise/Delegate mechanic is the core of this.
Recommended action: Implement ADR-016 first. That gives the debrief phase its structural backbone. Then iterate on additional features (rationale sharing, reaction prompts) based on usage patterns.
6. Scenario Tag Taxonomy Expansion¶
The observation: The Innovation Team Revolt was tagged "Team Dynamics" but the discussion revealed it touched competitive market strategy, organizational structure, innovation philosophy, and self-preservation. Amin's rationale was about industry-wide competitive dynamics (the Google/ChatGPT precedent), not team dynamics.
The question: Should the game expand beyond the current 4 categories (governance, resource-allocation, team-dynamics, values-culture) and allow multi-tagging?
Current state: Each scenario has exactly one category. The 4 categories are used for: category balancing (ADR-005), profile driver distributions, and the delegation map context space (ADR-011).
Options:
- Expand to 6-8 categories: Add explicit categories like competitive-strategy, innovation-adoption, crisis-response, people-management. More granular but increases the cold-start problem (need more games per category).
- Multi-tag within existing 4: A scenario can be tagged team-dynamics AND values-culture. The driver distributions update under both categories. Preserves the existing 4-category structure while allowing richer classification.
- Keep 4 categories, use triggers for granularity: The existing trigger system (crisis_mentioned, team_conflict, resource_constraint, etc.) already adds a second axis. Instead of expanding categories, expand the trigger taxonomy. The Innovation Team Revolt would be team-dynamics + triggers: competitive_pressure, innovation_challenge, authority_threat.
Recommended action: Option 3 — expand triggers, not categories. This is the approach ADR-014 (relational proximity) already takes. The trigger taxonomy is the right place for granularity because triggers compose naturally and don't require rebalancing the category system.
Depends on: ADR-014 (relational proximity adds a new trigger type, validating this approach).
Cluster C: AI & Prediction Quality¶
These affect how the AI generates scenarios and uses predictions.
7. AI Prediction as Mirror (Results Screen Enhancement)¶
The observation: When the AI predicted CoachJ incorrectly in the offshore scenario, the misprediction was more valuable than a correct prediction would have been. Seeing what the AI expected him to do created self-awareness about the gap between his pattern and his actual choice.
The question: Should the results screen highlight prediction failures and explain why the AI expected something different?
Current state: The results screen shows whether the DTA predicted correctly and its reasoning. But incorrect predictions get the same visual treatment as correct ones — just a different label.
Possible enhancements: - "Pattern divergence" callout: When the AI is wrong, show a brief explanation: "Your DTA expected [X] because in past [category] scenarios, you've favored [driver]. This choice suggests [Y]." - Learning note surfaced to player: The learning notes already extract patterns and flag contradictions. Surface the relevant learning note to the player after a misprediction. - "What changed?" prompt: After a misprediction, ask the player: "What was different about this scenario that made you choose differently?" This free-text response becomes extremely high-value training data.
Depends on: Nothing technically, but philosophically connects to ADR-015 (conviction). The "What changed?" prompt is most valuable when the player has high conviction on a divergent choice.
Recommended action: Medium priority. The insight is valuable but the implementation requires changes to the learning notes pipeline and results UI. Good candidate for a future sprint.
8. DTA Reputation Layer¶
The observation: Amin asked whether the game could show how a player's DTA has been perceived by others — e.g., "This person has been rated as reliable in [subject area] by 3 peers across 5 sessions."
The question: Should the trust/prediction phase show aggregate reputation data from prior sessions?
Trade-offs: - Show it: Gives players more information for trust decisions. Creates a positive feedback loop where consistent players build visible reputation. - Don't show it: Anchoring bias — if I see that "Amin is rated highly in competitive strategy by 3 people," I'll just delegate to him without forming my own judgment. The game's value is in forcing independent assessment first. - Show it after, not during: Display reputation data in the post-reveal phase so it informs future sessions but doesn't anchor current decisions.
Depends on: Enough multiplayer data to be meaningful. Currently 4 players with a handful of sessions. This needs 10+ sessions with 4+ players each to have useful aggregate data.
Recommended action: Defer. Build the data first by playing more multiplayer sessions. Revisit when there's enough aggregate trust/readability data to display meaningfully.
9. Scaling / Fast-Track for New Players¶
The observation: CoachJ has played 37 games and is only at 37% sync score. "A CEO is not going to play 37 games." 08 suggested a "fast track." CoachJ noted that the data from many games could eventually reveal which scenarios are most diagnostic.
The question: How do we get a meaningful profile from fewer games?
Approaches: - Item-response theory (IRT): Treat scenarios like test items. After enough aggregate data, identify which scenarios have the highest discriminating power — they most effectively differentiate between decision-making profiles. A fast-track battery of 5-10 high-discrimination scenarios could approximate the signal of 30+ random scenarios. - Adaptive scenario selection: Instead of random or category-balanced selection, choose the next scenario based on which choice would maximally differentiate the player's profile from other possible profiles. Similar to how GRE-adaptive picks questions based on your current estimated ability. - Intake quiz enrichment: The existing intake quiz captures demographics and sector. Enrich it with 3-5 forced-choice values questions (not full scenarios) that seed the profile with priors. The game then confirms/refutes these priors. - Transfer learning from similar profiles: If a new player's first 3 choices match an existing profile cluster, borrow confidence from that cluster's deeper history. Risky — could anchor on false similarity.
Depends on: Aggregate data from many players. This is a Phase 2+ concern.
Recommended action: Keep playing to build the dataset. The current founding team's 100+ games will become the calibration corpus. After opening the waitlist, implement adaptive scenario selection as the first scaling optimization.
Decision Priority Matrix¶
| # | Item | Effort | Impact | Dependencies | Recommendation |
|---|---|---|---|---|---|
| 2 | Scenario presupposition fix | Low | High | None | Do now (prompt update) |
| 5 | Retrospective phase (ADR-016) | High | High | ADR-016 | Next sprint |
| 7 | AI prediction as mirror | Medium | Medium | None | Future sprint |
| 1 | Personal vs business scenarios | Medium | Medium | Scope decision | Discuss with team |
| 3 | None of the above | Medium | Medium | ADR-015 | Discuss with team |
| 6 | Tag taxonomy expansion | Low | Medium | ADR-014 | Use trigger expansion approach |
| 4 | Self-reporting bias | Low | Low | ADR-015 | Monitor, revisit later |
| 8 | DTA reputation layer | High | Medium | Data volume | Defer |
| 9 | Scaling / fast-track | High | High | Data volume | Defer (Phase 2+) |