MARS Platform Contracts

Version: 1.0 Date: 2026-07-02 Owner: kee (mars) Audience: teams building verticals on top of MARS — private credit, real estate, insurance, international, biotech, or any other private-capital domain.

Why this document exists

MARS is a platform for private-capital data. It ingests news, extracts entities and events, resolves identity, and serves downstream consumers. Teams like rss (premium press), adrian (SEC forms), raul (M&A cleaning), futures (catalyst signals), and proxy (cost optimization) already work alongside MARS — each with their own scope, feeding into or reading from a shared canonical layer.

This document is the contract for new verticals that want to do the same thing. A vertical is a domain-specific slice — real estate, private credit, insurance, EU coverage — that MARS on its own doesn't yet cover well but that a dedicated team could build.

The goal: let verticals ship independently, in parallel, without integration debt. Honor the contracts and integration is free. Break them and it's a rebuild.

What MARS provides

Canonical entity layer, UUID-keyed, self-healing dedup: - companies_v2 — 250K rows (as of 2026-07-02). Every company is a UUID. Attributes: primary_name, name_variants, primary_domain, sector, cik, ticker, location, industries, etc. - persons_v2 — executives, officers, founders. LinkedIn URL as strong identifier. - investors_v2 — VC / PE / RIA firms. CRD as strong identifier for ADV-registered. - institutions_v2 — universities + accelerators for person_education backfills. - advisors_v2 — M&A advisors.

Layer 2 canonical deal tables — the article-level extractions get canonicalized into per-event rows: - funding_deals_v2 — VC / growth / debt / convertible / IPO / PIPE - merger_deals_v2 — M&A + acquisitions - deal_articles_v2 — bridge from canonical deal to source articles

SEC substrate — direct-to-v2 SEC ingest: - form_d_filings_v2, form_144_v2, schedule_13d_v2, schedule_13g_v2 - activist_positions_v2, institutional_positions_v2

Discipline layers: - _checks sub-doc on raw Mongo deals + integrity_tier (verified / vouched / warn) on canonical - identity_decisions_v2 — audit trail for every entity-resolution decision, reversible - Standard industry_mapping.py → 12-sector taxonomy + long-tail substring rules - Standard sentinel_names.py — placeholder-name detection - Tier-chain infrastructure (Qwen → Haiku → Grok-web) for cost-controlled extraction

Infrastructure: - Postgres 14 (custom build on proc2), MongoDB (search3) - S3 article body cache (insider-focus-gdelt bucket) - Bright Data residential proxy (metered — see [proxy contract note]) - Inferno / bench3 GPU inference (Qwen3-30B, free) - Anthropic API (Haiku 4.5 for structured tasks) - xAI Grok-4 with web_search for entity resolution against ground truth - Systemd timers, health email at 13:00 UTC daily - kscope team-mail bus (S3-backed markdown messages)

What a vertical looks like — mental model

A vertical is a bounded slice of the private-capital universe with: - A domain-specific SOURCE (Debtwire for private credit, Bisnow for RE, AM Best for insurance, Sifted for EU startups, etc.) - A domain-specific EXTRACTOR (private credit deals have different fields than VC rounds — commitment size, closes, LP composition, etc.) - A domain-specific CANONICAL TABLE (mars.<vertical>_deals_v2) - A domain-specific consumer (or shared consumer via MCP)

MARS provides the rails. The vertical provides domain knowledge, extraction, and its own deal table.

The contracts

These are non-negotiable. Break them and integration turns to sludge.

1. Use canonical company_id for every deal

Never invent a new company identifier. Every deal in your vertical's table FKs to mars.companies_v2(company_id):

CREATE TABLE mars.real_estate_deals_v2 (
  deal_id           UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  company_id        UUID NOT NULL REFERENCES mars.companies_v2(company_id),
  ...
);

If your company isn't in companies_v2 yet, MINT it via the standard ETL path in etl_company_id.py (or write a vertical-specific minter that follows the same conventions). Never create a parallel company namespace.

Identity resolution: use the tier chain from resolve_pending_companies.py (Qwen → Haiku → Grok-web). If your vertical has strong external identifiers (CIK, RSSD number, ISIN, national registration number), add them as external_ids JSONB attributes on companies_v2, not as separate identity tables.

Minting doctrine (mandatory read): before writing ANY code that INSERTs into a canonical entity table (companies_v2 / persons_v2 / investors_v2 / advisors_v2 / institutions_v2), read mars_feed/MINTING_ENTITIES.md. It covers the 5 universal rules (match first, multi-signal Qwen verify, sentinel filter, log to identity_decisions_v2, weekly dedup sweep) and per-entity-kind chapters with reference impls. The specific persons-kind chapter is at bios_pipeline/MINTING_PEOPLE.md. Getting mint discipline wrong at scale is how the 2026-06-27 self-merger cleanup (1,850 rows), 2026-07-04 merge_entities pass (891 merges), and 4× Jim Baird problem happened — every one of those was a preventable mint-before-match error.

2. Follow the integrity_tier pattern

Every deal in your vertical's canonical table needs an integrity_tier column with values verified | vouched | warn. Derivation follows integrity.py:derive_tier: - verified — cheap deterministic checks all pass AND LLM confirmed KEEP - vouched — cheap checks pass, no LLM verdict yet - warn — at least one cheap check failed

Provide a per-deal _checks sub-doc on the raw Mongo (or equivalent) with the standard hard checks: extract_guard, range_sanity, round_type_known, date_parsed, currency_valid, article_in_s3. Add vertical-specific checks as needed (private credit might add commitment_size_valid, real estate might add property_type_known).

Consumer contract: every MCP-facing query in your vertical MUST filter WHERE integrity_tier IN ('verified','vouched'). The warn tier catches anomalies (see the 2026-06-21 SpaceX $1.75T false extraction as motivation — defense-in-depth saved that from reaching a consumer). Bumping SCHEMA_VERSION in integrity.py when adding new checks is required.

3. Use the standard sector taxonomy

industry_mapping.py:INDUSTRY_TO_SECTOR — 265 industries → 12 sectors. Don't invent your own top-level sector list. If your vertical has industry names not in the map, add them via a PR to industry_mapping.py. The 12-sector list has been reviewed with Raul + adjacent teams; forking it fragments cross-vertical queries.

4. Use the standard tier chain for cost-controlled extraction

Cost hierarchy: - Inferno / Qwen3-30B (bench3): $0. Use for the bulk of any bulk classification, sentinel detection, canonicalization work. See qwen_investor_families.py, qwen3_recovery_calibration.py. - Haiku 4.5: ~$0.001/row. Use for structured JSON extraction where Qwen's margin is low OR for verification. - Grok-4 + web_search: ~$0.08/pair. Use for entity resolution where ground truth is IN THE WORLD (not our data).

Every LLM-driven step in your vertical should have a calibration curve run first (see qwen3_sentinel_calibration.py for the template). Ship the calibration script alongside the extractor. Cost governance in Phase 3 depends on every vertical using tier chains, not defaulting to Haiku for everything.

5. Write to your own mars.<vertical>_deals_v2 table

Naming: mars.<vertical>_deals_v2 for the canonical deal table. mars.<vertical>_deal_articles_v2 for the article bridge. mars.<vertical>_advisors_v2 etc. for domain-specific bridges.

Never fork funding_deals_v2 or merger_deals_v2. Those are MARS canonical, owned by mars. If your vertical also produces VC-style rounds (e.g., a private credit team catches a debt round), write it to funding_deals_v2 following the standard shape — don't clone the schema.

Version columns: use methodology_version on write and INSERT-log to identity_decisions_v2 for anything that could be reverted. Sweeps must be reversible.

6. Direct-to-v2 (not v1 detour)

Modern SEC and RSS ingests write directly to v2 (form_d_filings_v2, etc.). Don't take the historical v1 Mongo → ETL → v2 path unless there's a specific reason (like reusing the audit pipeline on the raw Mongo side). Direct-to-v2 shortens feedback loops and eliminates v1 rot.

7. Deterministic + LLM + audit trio

Every extractor stage needs: 1. Deterministic gate — cheap _checks. Fast, no LLM. 2. LLM extraction with a tier chain, not a single-model call. 3. Audit — a rerun sweep (Qwen or Haiku) on flagged rows within N days of ingest.

See audit_funding_haiku.py + audit_funding_tier.py + audit_funding_recovery.py + audit_funding_recovery_tier.py for the full pattern in the VC-funding vertical.

8. Health email visibility

Every vertical adds one check_<vertical>_health function to health.py that surfaces: - Row count freshness (rows added last 24h vs baseline) - integrity_tier distribution - Any degraded-threshold assertions

tracker.degraded() / tracker.critical() for the summary. Full section optional if the vertical has enough state to warrant it.

If a vertical is broken and nobody sees it in the daily 13:00 UTC email, nobody notices for a week. Health email visibility is not optional.

9. Health followups + team-mail

• health_followups.md — one-line entries for anything requiring a human action but not yet automatable. Timestamps + owner.
• Team-mail bus (teammsg.py) — cross-team async communication. Every vertical picks a slug (like mars, adrian, raul). Weekly inbox check is a professional expectation.

10. Documentation contract

Every vertical delivers: - <VERTICAL>_FOR_CLAUDE_CODE.md — onboarding doc for future Claude Code sessions. Same shape as MARS_V2_FOR_CLAUDE_CODE.md. Located in mars_feed/ or the vertical's own repo. - <VERTICAL>_PRODUCTION_READINESS.md — the migration contract for consumers who want to switch. Same shape as PRODUCTION_READINESS_V2.md. - Column-level stability promises (what changes and what doesn't, timeline for structural changes) - Consumer-facing filter contract (e.g., "always use integrity_tier IN ('verified','vouched')")

The onboarding flow

1. Scope + owner — vertical decides its scope (e.g., "US real estate private capital raises + REIT SEC filings"). Owner named. Timeline sketched.
2. Data source inventory — trade press URLs, SEC forms (if any), specialty databases (if any). Cost of source access.
3. Sample extractions on 100 articles — no schema, no writes. Just: can you extract the fields you care about? Manual review by owner.
4. Schema proposal — <vertical>_deals_v2 CREATE TABLE draft. Sent to kee for review against these contracts.
5. 10-row live test — ingest 10 articles end-to-end, write to a scratch table, verify FK to companies_v2 works, verify integrity_tier computes correctly.
6. Calibration curve — pick the extractor's LLM and run a Qwen-vs-Haiku or Qwen-vs-ground-truth calibration on 100 items before scaling.
7. Ship + timer — deploy the extractor on a systemd timer. Add health check. Announce via team-mail.

Time from step 1 to step 7 for a well-scoped vertical: ~4-8 weeks of a dedicated owner.

What NOT to do

• Don't fork the schema. Every extension goes on the canonical layer (companies_v2.external_ids JSONB) or on a new bridge table.
• Don't build your own entity resolution. Use resolve_pending_companies.py and the tier chain.
• Don't skip integrity_tier. Consumers depend on it filtering garbage.
• Don't skip the audit sweep. Extraction WILL have errors; the audit catches them within a week.
• Don't roll your own LLM budgets. Use the tier chain — Qwen first, escalate.
• Don't inline API keys in scripts or configs. Use keys.py (git-ignored) or /etc/mars-<name>.env (systemd EnvironmentFile).
• Don't ignore the health email. If your vertical has been broken for 3+ days without you noticing, the reliability signal to consumers is bad.

Escape hatches

• Ground-truth is in the world, not our data: escalate to Grok-web. Cost governance still applies but the accuracy delta is worth it for entity resolution.
• A vertical needs a schema attribute not in companies_v2: add it via a PR after discussion. If it's genuinely domain-specific (like real estate's property_type), it goes on the vertical's own table. If it's cross-cutting (like founded_year), it might belong on companies_v2.
• A vertical needs a source MARS doesn't have: talk to the proxy team (teammsg.py send proxy ...). If it's a paywalled data source (like Preqin), it's a "buy vs build" decision above the vertical owner's pay grade.

Consumer contract summary

For any MCP tool, dashboard, or downstream reader that reads a vertical:

WHERE integrity_tier IN ('verified','vouched')

That single filter guarantees the row cleared the deterministic + LLM gates. Any vertical that ships a table without this contract has failed its integration.

Freshness expectations

Default cadence for a well-scaled vertical: - Ingest: hourly (source-dependent — some sources publish daily) - Canonicalization: hourly (Phase G-style delta pipeline) - Health assertion: daily - Audit sweep: daily on last 7 days of new rows - Full backfill of any correction: on a case-by-case basis, always with --dry-run first

Versioning + backwards compatibility

• Additive changes to canonical tables: safe, no announcement.
• Column renames or type changes: announce via team-mail, give consumers 14 days notice.
• Table renames: announce, provide compatibility view for 30 days.
• Breaking schema changes: MAJOR version bump on the vertical's PRODUCTION_READINESS doc.

Cost accountability

Each vertical is responsible for its own AI + proxy spend. MARS operates the shared infrastructure but doesn't subsidize a vertical's ~$500/mo Haiku bill from mars's cost center. Bring your own budget or shift work to Inferno/Qwen (tier chain).

Standard cost reporting: monthly, per vertical, split by (AI extraction, proxy, human review). Same shape as MARS_COVERAGE_DELTA_MEMO.md's cost section.

The MARS operator's promise

If a vertical honors these contracts: 1. Your data is integrated the day it lands. 2. Your consumers see the same integrity guarantees mars gets. 3. You get free re-use of tier chains + audit patterns + entity resolution. 4. Your work shows up in the daily health email. 5. You get to ship in parallel with 4 other verticals without stepping on each other.

If a vertical breaks these: 1. Integration debt piles up on both sides. 2. Consumers can't trust the data. 3. Cross-vertical queries return nonsense. 4. Health email misses breakage. 5. Everyone else has to work around your fork.

The contracts exist because we've been burned by every one of these before.

Open questions (not yet contracts)

• International verticals — should the canonical layer include a language_iso on companies_v2 and deals? Currently we default to English. As we add EU/Asia, this may need a first-class field.
• Fund performance (LP data) — this is likely a "buy" not "build" (Preqin, Cambridge Associates, ILPA). Contract shape TBD.
• Real-time signals — futures's model (signal that pings you, not database you query). Do verticals need to publish to a shared signal bus? Not urgent, but worth thinking about.
• Fine-grained access control — some verticals may want row-level ACLs (e.g., a vertical bought exclusive data from a source). MARS doesn't have this today. If needed, it's a platform-level addition.

Contact

• kee (mars) — platform + rails + review of new vertical proposals
• team-mail bus — cross-team async, weekly inbox check expected

Who's actually reachable via team-mail (as of 2026-07-02):

Active users (check their inbox regularly, respond within days): - mars, futures, proxy, rss

Slug exists but not monitored — don't rely on delivery: - adrian (SEC forms owner), raul (M&A cleaning owner)

If your vertical needs coordination with adrian or raul, route through mars — kee will relay. Don't send directly and assume delivery.

New vertical proposal? Send it via teammsg.py send mars "vertical proposal: <name>" --file <path>. Response within 3 business days.