MARS · Data Platform
Every team now writes to the canonical MARS tables through a
small set of checked, logged, reversible functions — not with a raw
INSERT. Here's what we built, why it had to happen, and where it stands.
MARS keeps a handful of shared tables that everything reads from: the canonical company, person, investor, deal and fund records. Ten of them. They are the single source of truth — when a report says "Anthropic raised $65B," it's reading one of these rows.
The problem was that any code from any vertical could write straight into them with a plain database statement. No check that the row made sense, no record of who wrote it or why, and no clean way to undo it. That is how the bad rows we've chased all year got in — a $500-trillion "Series A," a hosting company minted as a customer, three different real companies fused into one, a firm listed as a person, an FX amount that was never actually converted. Each was one careless write into a table forty other things trust.
You can't audit your way out of that after the fact. The only durable fix is to stop the bad write at the door — to make every write prove itself before it lands.
The gate's whole job is simple: a write no longer gets to assert what's true. It has to pass a check, and it leaves a record either way.
There are exactly two database identities now. Our code connects as a
role that can read the canonical tables but cannot write them.
A raw UPDATE companies_v2 from any of our scripts is refused —
permission denied — full stop. The only thing that can write those
tables is a gate function, and those functions run as a second, privileged role
that nothing connects as directly.
So the write path is one-way through a checked door:
The important part: a rejection never stops the caller's run. A vertical can fire ten thousand rows at the gate; the good ones land, the doubtful ones queue, the bad ones bounce, and the pipeline keeps going. The gate is a filter, not a tripwire.
Instead of one god-function, there's a verb for each shape of write. Each one records what it actually observed, not what the caller claimed — the same discipline a pharmacy uses when it asks your date of birth instead of taking your word for who you are.
| Verb | What it's for — and the guard that matters |
|---|---|
| submit_company submit_person | Mint or match one entity. Resolves on strong IDs first (CIK, CRD, LinkedIn); a bare name with no identifier can't silently fuse into an existing row. |
| enrich_entity | Fill in a blank field — fill-only. It can add a missing value, never overwrite one that's already there. |
| set_entity_flag | Flip a whitelisted flag (e.g. quarantine a row to warn so consumers stop seeing it). Can't touch anything but the flags it's allowed to. |
| submit_merge_batch | Merge two entities into one. This is the expensive, dangerous operation, so it's the most guarded — and it snapshots the losing row before it goes. |
| submit_split | The undo. Un-merge, or re-home rows off a wrong entity. Merging used to be a one-way door; now it isn't. |
| update_deal upsert_deal_cluster | Write a deal's computed fields, and re-derive the integrity tier from a sanity check rather than trusting the number that came in. |
When a mars-owned job legitimately needs to touch a canonical — refreshing a derived count, say — that too becomes a function with the privilege baked in, never a re-grant. The privilege lives in the verb, not in the caller.
Every consequential decision the gate makes — a mint, a match, a merge, a split, a flag, a rejection — is written to a decisions ledger with the reason and a methodology tag. It stands at 3.1 million entries today.
That's what makes the whole system reversible instead of merely careful. When
we flip 57 SpaceX-class valuation rows to warn, or un-merge 76
wrongly-fused "John Kennedy"s back into real people, we're not hoping we can
reconstruct the before-state — it's in the ledger, tagged, and a later pass can
walk it straight back. A submissions ledger sits alongside it, holding every
payload the gate ever judged, so a review decision reads exactly what the
checker saw.
A UUID that points at a person or company only means something if the thing it points at exists. The rule we hold on the entity graph: an entity pointer carries a real foreign key. It does two jobs at once — the database refuses a pointer to a row that isn't there (no dangling references), and it subscribes that row to merges. A pointer with no foreign key is invisible when two entities are merged; its link silently rots. The key is what keeps the graph honest under change.
SET NULL, so if a person row is ever
removed the bridge keeps the filing record and simply drops the dead link. The
edge survives; the pointer never dangles.
That officer change is the clearest example of the gate earning its keep. A Form D filing lists "related persons" — but roughly half of the new ones are firms (a management LLC listed as a promoter), and most of the rest are common names with nothing to tell two same-named people apart. The old code minted a fresh person for every one, which is exactly how you manufacture duplicate and fused people at scale.
Now the officer record is bridge-first: the filing → issuer → name → role → address edge is real and always kept, but the person is minted only when the gate can actually resolve one. The firms get rejected, the ambiguous names wait for a second identifying fact, and the graph stays clean. The same principle covers the other place we hold un-mintable people until we can honestly place them.
The platform this protects is not small. The canonical graph the gate now guards:
Bios and signals are fully converted onto the gate — and they are two of the heaviest writers we have, minting people and events continuously. That they run clean through the checked door is the proof the model holds at volume. Mars's own core pipelines are through as of this week, including the full SEC Form D chain that had been quietly broken since the lockdown. The remaining verticals get converted the same way, one at a time.
integrity_tier
filter you already rely on is now enforced at write time, not just at read
time. And anything that turns out wrong can be pulled back precisely — from the
ledger, with its reason attached — instead of by guesswork.