← All memos
May 28, 2026platformplatformOpen

Mart spec deltas (missing metrics, kind field errors) and a firewalled name collision

Expects responseYes
Tagsmart, contract, spec, firewall

Mart spec deltas (missing metrics, kind field errors) and a firewalled name collision

Why

Them OS finished the ADR-0044 Mart Consumer API arc in May 2026: the lib/martClient substrate, seven Tier 2 section pass-through capabilities (<section>.read_metric.v1), and the first two Tier 3 authored interpretation capabilities (portfolio.org_market_health_brief.v1 and growth.intake_health_summary.v1) are live. During the wiring eval we hand-extracted the live spec into contracts/mart/metrics.json (122 metrics across 7 sections) and walked it row by row against docs/mart-requirements-them-os-forecasting.md (the requirements contract Them OS originally handed the mart team). Three classes of delta surfaced. The first two are spec gaps the mart owners can close at their cadence; the third is a name collision that, left unresolved, will keep producing slow confusion in code review and capability authoring.

We are filing these now rather than waiting for them to bite, because two of the deltas (missing coaching utilization metrics, the firewalled name collision) will block the next round of Tier 3 capability authoring once we start work on the coaching brief and on per-section synthesis interpretations.

What we found

Class 1: metrics the requirements doc names that the spec does not expose

Eight from Coaching (§6 of the requirements doc):

coach_utilization_completed
coach_utilization_scheduled
coach_utilization_rate
coach_idle_hours
coach_idle_hours_by_capability
coach_quality_score
coach_ramp_time_days
coach_tenure_distribution

Four from Delivery (§5 of the requirements doc, including coach_satisfaction_of_platform which the doc places in Delivery despite the coach_ prefix):

customer_satisfaction_of_coach
coach_satisfaction_of_platform
avg_lessons_per_active_student
time_to_first_delivery_lesson_days

The Coaching utilization gap is the most operationally costly. The mart exposes coach_capacity_hours, coach_completed_hours, and coach_scheduled_hours, but Them OS cannot construct any of the three utilization views without doing the ratio in-process, which makes the basis-point precision and the null semantics inconsistent with the rest of the rate-shaped metrics. Per the requirements doc, utilization is a section-owned derived metric (Coaching computes it), not a consumer-side derivation.

Class 2: suspected kind field declarations that disagree with the metric

A consumer-side audit of contracts/mart/metrics.json flagged the following mismatches between the declared kind field and what the metric is actually shaped like (per its id, description, grain, and the requirements doc). High confidence cases (id plus description disagree unambiguously):

growth/ad_impressions                          declared cents, is count
growth/ad_clicks                               declared cents, is count
growth/lead_volume                             declared cents, is count
growth/lead_intake_count                       declared cents, is count
sales/intake_count                             declared rate,  is count
sales/intake_count_per_source_class            declared rate,  is count
sales/qualified_count                          declared rate,  is count
revenue/unpaid_orders_count                    declared cents, is count
delivery/lesson_continuation_rate_window_weeks declared count, likely duration_weeks

Our heuristic flagged sixteen total; we filtered down to the nine above by reading each flagged metric's description. The others (e.g. coaching/coach_revenue_share_pct, finance/burn_rate_per_week, revenue/discount_applied_value) are correctly declared and the heuristic produced false positives on them. If your own audit surfaces other adjacent cases when you check these nine, please include them in the same correction batch.

These mismatches do not break the envelope contract: the value field carries whatever number the mart computes regardless of the declared kind. But the kind field is what consumers route on to decide whether to render the value as basis points, cents, or counts in interpretation outputs, and to drive the right aggregation rule when rolling up across grains. Wrong kind declarations will eventually produce wrong UI labels or wrong rollups.

Class 3: the firewalled field name now means two different things

In contracts/mart/metrics.json, each metric carries a per-metric boolean firewalled: <bool>. The intent, as we read it, is "this metric's computation is internal to its section and the underlying inputs should not be exposed across the section boundary." Today exactly one metric in the spec has firewalled: true (revenue/revenue_per_completed_lesson), so the field is near-vestigial.

ADR-0044 Rule 5 (the Mart Consumer API architecture decision in Them OS) also uses the word "firewalled" with a different and load-bearing meaning: a section's output is firewalled when no shared customer / lesson / source / coach key is reachable across the five operational sections (Growth, Sales, Delivery, Coaching, Revenue). This is the architectural firewall enforced by the requirements doc §1 and §2.5, and it operates at the section API surface, not at the individual metric level.

The collision: when a capability author or reviewer reads "firewalled" in either the spec or in ADR-0044, they have to context-switch to figure out which of the two concepts is in play. We had to disambiguate this in code review once during the lib/martClient build. It will keep happening, and the cost compounds as more Tier 3 capabilities accrete.

Asks

Three asks, in order of bite:

First, please add the twelve missing metrics named above to the spec, at the grain and null-semantics the requirements doc declares for each. The Coaching utilization triplet is the most urgent because it blocks the coaching Tier 3 work we want to author next. If any of the twelve has been intentionally omitted, please reply with the reason so we can either accept the omission and remove the requirement, or surface a substitute.

Second, please audit the nine high-confidence kind field cases and either correct the declarations in the spec or reply with the reason the current declaration is right and our reading is wrong. We are not asking for a sweep of all 122 metrics, only these nine.

Third, please rename the per-metric firewalled boolean to disambiguate it from the section-level architectural firewall concept. We suggest compute_is_internal (literally what the field means today) or hides_inputs_from_consumers (what its effect is). We have no preference between the two; the rename itself is what removes the collision.

There is no deadline on any of these. We are not blocked today; we are surfacing them now so they can land at your normal cadence rather than under pressure when the next consumer need fires.

References

  • adrs/0044-mart-consumer-as-substrate.md (the Mart Consumer API architecture decision in Them OS)
  • docs/mart-requirements-them-os-forecasting.md (the requirements doc this memo cross-references; §5 Delivery and §6 Coaching for the missing-metrics section)
  • contracts/mart/metrics.json (the consumer-side spec extract; 122 metrics across 7 sections; the git commit IS the spec version per ADR-0044 Rule 3)
  • domain/PROGRESS.md rows P4.7 and P4.9 (the Them OS closure notes that surfaced these deltas)

Thread (4 memos)

May 28platformBoth items landed; coach_utilization_rate alias is in the registry and scripts/export-mart-metrics.ts ships the generated contracts/mart/metrics.jsonMay 28platformReply, the extractor is the bug; accepting the coach_utilization_rate alias and yes on the offered exporterMay 28platformReply on mart spec deltas, the extraction is the source of three of the three classes (one real missing metric remains)

View source on GitHub