Welcome to the MARS mandate

Living document. Last updated 2026-08-04. Latest version at mars_feed/WELCOME_TO_THE_MANDATE.md (source of truth) and kee.staticpipe.com/mars/welcome-to-the-mandate.html (shareable render).

For: teams that have been running independently and now need to integrate with the MARS platform. If your work touches a company, person, deal, filing, or any structured entity — this doc applies to you.

How to read this: the contracts section is non-negotiable. Every new vertical accepts them by sending the acceptance note on team-mail. Everything else is context you need to be productive.

1. What MARS is now

MARS is the shared platform underneath every kscope data product. Verticals (reit / bios / flare / credit / lp_data / insurance / signals / prospects / insider / donor / sedar / sections / crown / ...) all write to the same canonical entity substrate under mars.*_v2 in Postgres.

• One companies_v2. One canonical row per real company, UUID company_id. Every vertical writes here through a single write gate — the mars.submit_* SQL functions; a direct INSERT is refused.
• One persons_v2. Same shape for real people.
• One investors_v2, institutions_v2, funds_v2, advisors_v2. Same shape for the other entity kinds.
• Deal / event tables are per-vertical (funding_deals_v2, merger_deals_v2, real_estate_deals_v2, private_credit_deals_v2, thermal_events_v2, form_4_transactions_v2, signals_v2, ...) but every row FKs back to the shared entity canonicals.

The moat is the entity graph across all verticals. Not any single vertical. The reason cross-vertical queries work — "who worked at Merrill now runs an RIA managing > $1B AUM," "was the target of this merger raised on private credit first," "insider sold days after Form D close" — is that every vertical writes to the same company_id / person_id.

2. The 10 platform contracts

Non-negotiable. Full text in mars_feed/MARS_PLATFORM_CONTRACTS.md. Summary:

1. Canonical company_id. Mint via the write-gate functions (mars.submit_company / submit_person / …) — never write companies_v2 directly. Discover the verbs with mars.gate_help(...) (see §3). (The old mars_entity_mint HTTP API is being retired.)
2. integrity_tier on every deal / event row. verified / vouched / warn. Consumers filter IN ('verified','vouched').
3. Standard sector taxonomy. Use the same industry buckets as funding_stage_mapping.py / industry_mapping.py.
4. Tier chain for LLM cost. Qwen first (~$0), Haiku or gpt-oss as escalation, Grok/Brave only for world knowledge. Detail below.
5. Own your *_v2 deal/event table. Namespaced (real_estate_deals_v2, credit_bankruptcy_filings_v2, etc.). Not on the shared canonicals.
6. Direct-to-v2. No detour through v1 Mongo. Every write goes to mars.*_v2 in Postgres.
7. Deterministic + LLM + audit trio. Deterministic checks first (regex, whitelist, name-match), LLM for the gray zone, audit script that spot-checks after landing.
8. Health email visibility. Your vertical writes to the daily 13:00 UTC health.py email. If it's broken, the health check must fire.
9. Team-mail communication. All cross-team coordination via s3://kscope-team-mail/{slug}/inbox/. No Slack, no direct email for coordination.
10. Documentation deliverables. Every vertical has a CONTRACT.md + SCHEMA.md + SOURCES.md + README.md + CLAUDE.md (ops guide). See bios_pipeline/ or reit_pipeline/ for the shape.

Full text: https://kee.staticpipe.com/mars/platform-contracts.html (source: mars_feed/MARS_PLATFORM_CONTRACTS.md).

How you accept: send an acceptance note from your team-mail slug to mars. That's the handshake.

3. Minting entities — MINTING_ENTITIES.md summary

The most common bug in the mars ecosystem is a vertical writing a duplicate entity because it didn't check for matches first. The MINTING_ENTITIES.md doctrine (in mars_feed/) covers all 5 canonical tables. Full details there; the 5-rule discipline:

1. Strong-ID match first. CIK, CRD, ticker, LinkedIn URL, ISIN. If a strong ID matches, USE THAT ROW. Never mint a duplicate.
2. Name-variant lookup. companies_v2.name_variants[] is populated with historical aliases (Facebook → Meta). Check GIN-indexed @> lookup BEFORE trigram match.
3. Name + candidate + Qwen verify. Trigram fuzzy match with pg_trgm, then Qwen classifier when candidates are close.
4. Sentinel filter at write. is_sentinel_name / is_role_not_name / is_placeholder_crd — never mint "Unknown Target" / "TBD" / "N/A" / "CEO" as a name.
5. Log every mint to identity_decisions_v2. Decision + methodology + metadata. Reversible.

Full text: https://kee.staticpipe.com/mars/minting-entities.html (source: mars_feed/MINTING_ENTITIES.md).

For persons specifically, see bios_pipeline/MINTING_PEOPLE.md — 10 persons-side rules with drop-in code.

Write through the gate verbs, don't hand-write minting logic. Every canonical write is a SQL function you call over a normal postgres connection (as your <vertical>_writer role) — mars.submit_company / submit_person / etc. Same logic every time, same guards, same identity_decisions_v2 audit trail.

The old mars_entity_mint HTTP API (mint.staticpipe.com) and the from mars_feed.mint import mars_entity_mint Python shim are being RETIRED — do not build against them. The gate functions below are the path, and they are what a direct INSERT into a canonical is refused in favour of.

Discover the write verbs from the database itself — no doc hunt. The gate is self-describing. Connected to mars, query:

SELECT * FROM mars.gate_verbs_v;               -- every verb: args, purpose, who-can-call
SELECT * FROM mars.gate_help('mint a person'); -- natural-language search

Every canonical write is a verb (submit_company / submit_person / submit_funding_deal / enrich_entity / set_entity_flag / submit_merge_batch / submit_split / …); the catalog tells you the payload shape, the guards, and which one to reach for. mars.preview_submission(kind, payload) answers would_mint WITHOUT writing anything — use it to decide before you submit, so rows that can't resolve never enter the review queue.

4. The Qwen / Serper / Grok line (2026-08 doctrine)

The rule: Qwen is reliable whenever the answer is IN THE TEXT you give it. Grok / Serper are only needed when the decision requires WORLD KNOWLEDGE.

Concretely:

The trap Qwen falls into: same-brand-different-vehicle. "Blackstone Real Estate Partners" vs "Blackstone Europe Real Estate" reads as the same to Qwen because both strings contain "Blackstone Real Estate." The answer requires WORLD KNOWLEDGE (which fund vehicle exists, which is a regional sub-fund) that isn't in the strings you passed.

Fix pattern: for entity resolution where only the NAME links two rows, use Grok-web OR the newer Serper+Qwen tier. Never Qwen alone.

Serper+Qwen tier chain (2026-08-02, replaced Brave after A/B): 1. Serper (Google Search API) returns candidate URLs, $0.001/query 2. Qwen reads the top-3 URL summaries to verify 3. ~20× cheaper than Grok-web for the same accuracy 4. Endpoint: bios owns search_providers.py module — ask bios team 5. See ref-post-spider-crawl-stack memory for the full stack (Serper + requests + Qwen + Firecrawl) that replaced mars_spider 2026-08-02

Note: Brave was retired after bios's A/B measured Serper winning on downstream people-extracted count (96 vs 90) even though URL counts were similar (43 vs 46). Older refs to Brave still exist in memory / inferno-tier docs; treat them as historical.

Cheap extraction + expensive verification (2026-07-29, bios doctrine): for any extraction+judgement pipeline, use the CHEAP model for EXTRACTION and the EXPENSIVE model as the VERIFIER. bios validated over 8 random-100 rounds on their relationship-graph work: raising the verifier (30b → 235b) lifted BOTH precision AND recall. Raising the extractor did nothing at 25× the cost. Apply this everywhere: qwen3-30b for extraction, qwen3-235b (or Grok-web where world knowledge required) for verify. Same "iterate 100-sample spot-checks, add a deterministic guard per failure mode" loop used everywhere.

5. Consumer contract — the 3 filters

Every consumer query MUST apply these 3 filters (or explicitly document why not):

WHERE integrity_tier IN ('verified','vouched')  -- not 'warn'
  AND amount_usd_converted IS NOT NULL           -- for FX comparability
  AND NOT is_placeholder                         -- on investors_v2 side

Additional performance notes:

• Use lower(col) LIKE lower(:pat), never ILIKE (130ms → 17ms; the trigram GIN doesn't kick in for ILIKE).
• Use WITH found AS MATERIALIZED (... LIMIT 1) to avoid the GIN+LIMIT-1 seq-scan trap on small tables.
• Person queries should hit bios_current_firm_v2, not persons_v2 directly (planner mis-estimates the methodology filter 38×).

6. Kbrain — Adrian's knowledge vault

URL: https://api-dev.kscope.io/kbrain/mcp (MCP endpoint).

What kbrain is for: durable reference. Entity models, source-of-truth documentation, doctrines, API contracts, concepts. Things that will still be true 6 months from now.

What kbrain is NOT for: transient state. Task lists. Progress trackers. Current-status pages. Things that decay and mislead future readers.

MARS's kbrain footprint: 6 notes live there today (created by mars on adrian's kbrain): - Projects/MARS.md — top-level project doc - Reference/MARS entity graph v2.md — the canonical shape - Reference/MARS data sources.md — where each vertical sources from - Reference/mars_entity_mint API.md — entity-write contract (now the write-gate SQL verbs; the HTTP API is retiring) - Concepts/MARS query patterns.md — consumer patterns - Concepts/Consumer contract filter.md — the 3-filter rule above

Sync obligation: when substrate / contract / sources / mint changes, mars updates kbrain. Contract change → kbrain edit within same session.

For your team: if you want your team's docs in kbrain, submit them via team-mail to mars — we'll seed on your behalf. Adrian owns the vault so all writes go through us.

7. Team-mail bus

The async coordination channel.

• Bucket: s3://kscope-team-mail/
• Your inbox: s3://kscope-team-mail/{your-slug}/inbox/
• Helper: mars_feed/teammsg.py — copy to your repo or use directly
• Send: python3 teammsg.py send {recipient-slug} "subject" --file body.md
• Read: aws s3 ls s3://kscope-team-mail/{your-slug}/inbox/ --recursive

Active slugs (2026-07-28): mars · mars-ui · futures · rss · bios · reit · credit · insurance · intl · lp · signals · prospects · insider · donor · flare · sedar · sections · inferno · crown · instagram · argos · power-sec.

Establishing a new slug: first inbound message to s3://kscope-team-mail/{slug}/inbox/ creates it.

8. New tools you should know about (2026-07 → 2026-08)

• The write gate (SQL verbs) — the canonical write path. Connect to mars as your <vertical>_writer role and call mars.submit_company / submit_person / submit_funding_deal / enrich_entity / … Discover them and get per-verb help with mars.gate_verbs_v + mars.gate_help(query); mars.preview_submission(kind, payload) returns would_mint without writing. This REPLACES the retiring mars_entity_mint HTTP API (mint.staticpipe.com) — do not build against that.
• Inferno HTTP API — https://inferno.staticpipe.com/jobs. Batch Qwen at ~$0.0002/item. Auto-shard 100 items/shard. Body limit ~1MB (~5-6K items per submission).
• Serper Search API — via bios's search_providers.py module. Cheap entity-in-the-world resolution (see Qwen line above). Retired Brave.
• Post-spider crawl stack (bios-owned, offered platform-wide) — Serper discovery + requests fast fetch + Qwen extract + Firecrawl for 403/202/JS tail. Real cost $0.0014/person. Ask bios for modules — don't re-derive. See ref-post-spider-crawl-stack memory.
• merge_entities.py — atomic entity merger with dynamic FK discovery from pg_catalog. Handles all 5 canonicals + JSONB additive merge. Never write your own merge code.
• Weekly linkage sweep — mars-linkage-sweep.timer (Sun 09:00 UTC). 7 deterministic FK re-sweeps that catch write-time misses. Free.
• Refresh MV chain — mars-refresh-views.timer (daily 06:15 UTC). Add your MVs / serving tables via refresh_materialized_views.py.
• News landing zone + signals-as-extractor (2026-07-31) — additive shared parquet drop at s3://kscope-news-raw/{producer}/{YYYY-MM-DD}/ {batch}.parquet for cross-vertical extraction. See separate section below.

8a. News landing zone — an extraction OPTION for text-source verticals

What it is. Shared parquet corpus at s3://kscope-news-raw/{producer}/{YYYY-MM-DD}/{batch}.parquet with a pinned 10-column schema (docid / published_at / title / body / source_url / producer / language / classify_scores / extras / etc.). Spec: mars_feed/docs/NEWS_LANDING_ZONE_SPEC.md.

Current producers (2026-08-04): - benzinga/ — mars-owned (from Benzinga SQS) - gdelt/ — mars-owned (from GDELT poller) - rss/ — rss-owned (~1,200-1,500/hr from RSS feeds) - 8k/ — 8k-owned (SEC 8-K filings emitted as article-like text)

Current consumer: signals reads all producers for cross-vertical extraction into lawsuit_events_v2 / bankruptcy_events_v2 / appointment_events_v2 / etc.

The option for your vertical: if your data is article-like text and you don't want to own extraction, package it as parquet with the 10- column schema, drop in s3://kscope-news-raw/{your-slug}/, and signals extracts events from it into their tables. This is what 8k did — they emit SEC 8-K filing text into the landing zone, signals extracts bankruptcy / appointment / guidance events from it. 8k doesn't own event extraction; they own the SOURCE.

When this fits: - Your data is article-like text (press releases, filings, transcripts, court records, PR wire) - Your source contains events across multiple categories (bankruptcy, appointment, litigation, etc.) — signals can extract many kinds from one text - You want to focus on being the AUTHORITATIVE SOURCE for your text stream rather than the extractor - Cross-vertical extraction (signals's job) beats bespoke extraction (your job) for the shape of your data

When it does NOT fit: - You own DOMAIN-SPECIFIC extraction that requires expertise signals doesn't have (agreements linking parties, sedar QP resolution, flare thermal-asset registration — these are vertical-owned extraction) - Your data is structured (SEC XBRL, LEI, structured registry rows) that needs bespoke parsing, not text extraction - You need extraction quality that only comes from tight coupling between source knowledge and extraction logic

Systemd requirement if you write to the landing zone: KillSignal=15 (SIGTERM) or the shutdown-flush signal handler never fires. See mars's Benzinga writer for reference.

Ping signals (slug signals) with a scope + first-write plan if you want this path. They'll tell you which producer slug to use and whether your text needs any pre-classification before drop.

9. Extraction traps to avoid (2026-07 learnings)

Things that have bitten us hard. If your vertical touches any of these patterns, read the corresponding memory / doctrine before shipping.

• Write-drop trap: LLM extracts a field, Python whitelist DROPS it before writing to Postgres. 10-day zero-coverage silent bite on valuation columns 07-27. Always grep for the write-path copy list when adding a field to an LLM prompt.
• JSONB additive-merge trap: merge_entities.py used COALESCE(keep, drop) on JSONB, silently dropping additive keys. Fixed 07-28. Any merge tool MUST handle JSONB additively.
• Franken-person taxonomy: placeholder-CRD (1234567), firm-CRD attributed to individual, common-name over-merge (John Smith 30 companies). See ref-franken-person-taxonomy memory. Fix in MINTING_PEOPLE Rule 1a + 2 + 3.
• Qwen fails fluently on world-knowledge: same-brand-different-vehicle is rife in PE / RE / infra. Use Grok-web or Brave+Qwen for entity-in- the-world tasks.
• Health check must detect a dead pipeline: internal consistency checks (MV vs its own base tables) stay green forever on a graph ingesting NOTHING. Check newest-row-age from SOURCE, not just from your MV. Bios check_ingest caught this class 07-14.
• Legacy substrate inheritance: when your vertical takes over legacy rows, retag doesn't surface them; the tier gate holds. But BEFORE backfilling bridges (person_work_history_v2 etc.), vet the legacy — linkages leak past the tier gate. See ref-legacy-substrate-inheritance-doctrine.
• nohup / setsid blocked here. Use run_in_background if you're scripting via Claude. If shell, use systemd.
• PyMuPDF not thread-safe. Use processes.
• Any INSERT INTO mars.*_v2 needs the sentinel guard. Every mint path. extract_2b_bios guarded, link_employers was NOT — re-leaked in a loop. Grep every INSERT.

10. Vertical status map (2026-07-29)

10a. Linking doctrine (2026-07-29)

bios is IN CHARGE of linking across the platform. mars ASSISTS. Other verticals just LOAD DATA.

Why: bios has proven domain knowledge — 9 regex guards each earned by a random-100 failure mode, cheap-extract + expensive-verify pattern, quote-grounding discipline, 70-77% compounding read on when to resolve. That knowledge transfers well as a team doing the work — not as a contract for people to follow.

What this means for your vertical:

• Do NOT build a per-vertical link resolver. No fuzzy matching, no LLM verifier calls on names, no cross-source stitching. If you need "resolve donor name X to person_id" or "cluster owner names Y across property records," route to bios via team-mail — don't build it.
• DO land raw data via MINTING_ENTITIES.md discipline: strong-ID first, sentinel filter, log to identity_decisions_v2 with your methodology tag. Bios's resolver picks up your person_id=NULL / company_id=NULL rows on their cadence.
• mars-core is different: mars can still do linking work that leans on infrastructure we built + understand (existing runners pending_resolver_runner, merge_entities.py, mars-linkage-sweep.timer). mars does NOT build NEW edge-discovery pipelines. Direction on linking comes from bios.

Anti-pattern (permanently closed): every vertical rolls its own edge resolver. That's how we ended up with the franken-mess we spent months cleaning out of companies_v2 + persons_v2.

11. How to integrate — new-vertical checklist

1. Read this doc end to end.
2. Read mars_feed/MARS_PLATFORM_CONTRACTS.md in full.
3. Read mars_feed/MINTING_ENTITIES.md in full.
4. Send acceptance from your slug on team-mail bus to mars. Format: "we've read the mandate + contracts + minting doctrine; here's what we're doing; here's our proposed table shape; here's our target ship date."
5. Decide your extraction shape — three paths: - (a) Direct-to-v2 vertical extraction — you own the whole pipeline from source → extraction → canonical writes. Reference impls: reit, bios, flare, credit, sedar, sections. Right when extraction needs vertical-specific expertise. - (b) Landing-zone producer — you package article-like text as parquet, drop in s3://kscope-news-raw/{your-slug}/, signals extracts events into their tables. Reference impl: 8k. Right when your data is article-like text with multi-category events. - (c) Hybrid — landing zone for some sources + direct-to-v2 for others. Nothing forbids picking both.
6. Get pointed at existing substrate you'll write to — or scope a new *_v2 table under your slug's methodology tag prefix (e.g., power-sec-*).
7. Health check contract: your vertical adds a check_{vertical} function to mars_feed/health.py that fires on real signals (freshness, coverage, integrity_tier mix). Route it through the daily 13:00 UTC email.
8. Documentation: create {your_pipeline}/CONTRACT.md, SCHEMA.md, SOURCES.md, README.md, CLAUDE.md. Copy shape from reit_pipeline/ or bios_pipeline/.

12. Living document — how this updates

Last update: 2026-08-04. Changes since last edit: - Section 4: retitled Brave → Serper. Brave retired after bios's A/B — Serper won on downstream people-extracted count. Table + tier chain description updated. Brave references treated as historical. - Section 8: added post-spider crawl stack pointer (Serper + requests + Qwen + Firecrawl, bios-owned modules offered platform-wide) with $0.0014/person real cost. - Section 8a NEW: News landing zone + signals-as-extractor. Extraction OPTION for text-source verticals — package article-like text as parquet under your slug in s3://kscope-news-raw/, signals extracts events into their tables. 8k is the reference impl. Includes when-it-fits + when-it-doesn't guidance. - Section 11: extraction shape decision (a/b/c) now step 5 — direct-to-v2 vs landing-zone-producer vs hybrid.

Last update: 2026-07-29. Changes: - Section 4: added "cheap extraction + expensive verification" doctrine from bios's relationship-graph work (raise verifier not extractor) - Section 10: added deeds vertical (public-records wealth signals, scaffolded 07-29) - Section 10a NEW: linking doctrine — bios owns linking, mars assists, other verticals just load data - Section 12 (this): change log

This doc is the running doctrine for platform-wide learnings. When something changes:

• Contract change → update MARS_PLATFORM_CONTRACTS.md + append to section 2 here.
• New extraction trap discovered → append to section 9.
• New tool shipped → append to section 8.
• New vertical goes LIVE → update section 10.
• New model / Qwen-line refinement → update section 4.
• Kbrain sync if section 6 changes.

The mars team maintains this doc. Contributions welcome — send an edit-suggestion on team-mail and we'll integrate.

Appendix — reading list order

If you're onboarding fresh, this is the order (all available as hosted HTML for teams without repo access):

1. This doc: kee.staticpipe.com/mars/welcome-to-the-mandate.html
2. Platform contracts: kee.staticpipe.com/mars/platform-contracts.html
3. Minting entities: kee.staticpipe.com/mars/minting-entities.html
4. bios_pipeline/MINTING_PEOPLE.md (if your vertical mints persons — ping bios team for hosted URL if needed)
5. mars_feed/mcp_specs/ (if you're a consumer)
6. mars_feed/PRODUCTION_READINESS_V2.md (if you're building a consumer product)

That's ~2 hours of reading. Everything else is available on team-mail or by asking mars directly.

— mars