An SEC filing goes in. A structured row comes out, plus one row per named party. Between those two points sit nine steps, and only one of them is a language model — the other eight are deterministic, and most of the work is deciding what to send the model and what to believe when it answers.
Regex first, model once, then guards. Regex decides whether a document is an agreement at all, guesses its type, and detects blank templates. Qwen reads roughly 6,000–15,000 characters of a document that averages 32,000 and answers a fixed JSON schema. Then eight deterministic checks decide which of its answers we keep.
One agreement costs $0.0004 and takes about three seconds.
Three different populations get confused easily, so plainly:
| population | what it is | size |
|---|---|---|
| scan log | filings checked for an EX-2.1, by the one exhaustive sweep we have run | 321,147 |
| M&A slice | exhaustive over that sweep — every EX-2.1 it found | 6,209 |
| general slice | a random sample of EX-10 material contracts, drawn across 2024–2026 | 9,795 |
The 9,795 is a sample, not a census. It is roughly 7% of the ~141,000 EX-10 agreements filed since 2022, scattered across three years rather than covering any of them. Every coverage figure below is therefore a sample statistic, not a corpus fact — and the sample is weighted by a family rotation, so it over-represents commercial and credit relative to what a year of filings actually contains.
1 · deterministic · boto3 + S3
One gzipped JSON per accession from s3://sec4-parse-raw/, with a
header and a docs[] array carrying every exhibit's
full text inline. The header gives us the filer CIK and filing date; both end up
on the output row.
2 · deterministic · textify.py
Exhibit text is raw HTML. Block-aware conversion, because paragraph boundaries matter to every later step — heading detection, span selection and the quote checker all read the flattened text.
3 · deterministic · identify.py
Two filters. is_agreement_exhibit keeps EX-10 material
contracts. is_excluded_exhibit drops company-wide plans and
policies — an equity incentive plan has no counterparty, so it is not an
agreement.
That second filter is subtle. An award under a plan is an agreement; the plan is not, and both say “Participant” on every page. The guard reads the title only, because reading the body let four plans through.
4 · deterministic · identify.py
classify_agreement runs ~58 title patterns and returns a type or
unknown. is_template_document asks a separate
question: is this a blank form of exhibit, filed to disclose
terms rather than record a deal?
14% of admitted documents are templates. Their numbers are literally underscores — Grant Date: __________. They are kept and flagged, never silently dropped, because a consumer counting signings needs to exclude them and a consumer studying terms does not.
5 · deterministic · spans.py
The median agreement is 32,000 characters; the median document sent is about 6,000. Three pieces, and the third only sometimes:
$10,050,000 at character 14,296 of 130,535;Both windows are conditional, and that is the whole finding. A
paired experiment over 116 documents and three seeds: reading 5.6× more of
the document moved term from 31 to 54 filled and moved nothing
else. Appending a window unconditionally made every credit field
worse.
More text pays only where the fact is not already in front of the model.
6 · the only model call · Qwen via OpenRouter
qwen3-30b-a3b-instruct-2507, one call, fixed JSON schema. Every
agreement answers the same universal core:
agreement_type obligation parties[] + roles filer_role
value_direction amount + currency + amount_basis effective_date
term quotes[] unstated[]Four families add a block: credit asks borrower, lender, facility, maturity, covenants; commercial asks counterparty, which side the filer is on, subject matter, exclusivity; employment and equity similarly.
unstated is unusual and deliberate: the model lists what the
document does not say, so a downstream reader knows the limits of the
row rather than assuming completeness. inferno keeps it as an
anti-confabulation guard for their report writer.
A second call fires on about 45% of documents, and only those: when the regex could not guess a type, the first call cannot know which family block to ask for, so the block is requested afterwards once the model has named the type. The universal core is taken from the first call and the merge refuses any key it already holds, so the second call cannot move the type, the obligation or the amount.
The obvious alternative — offer all four blocks in one call and let the model choose — was tested and rejected. It works, but it anchors: against a control of the same prompt run twice (31 of 32 identical types), offering the blocks changed the type on 18 of 32 and the reporting family on 4. An equity block turns warrant into equity_agreement.
7 · deterministic · typemap.py
Regex and model both named the document. reconcile() folds the
model's free text onto the taxonomy and returns a verdict, so
“renamed what we already knew” is separable from “found
something we gave up on”:
| verdict | meaning |
|---|---|
| agree | both landed on the same type |
| renamed | same answer, different spelling — award, grant and agreement are noise |
| refined | the model named a subtype; an asset purchase is a purchase |
| recovered | the regex punted and the model found a real type |
| conflict | both committed and disagree — usually a regex bug |
| unmapped | the model named something the taxonomy cannot express |
8 · deterministic · guard.py, dates.py, party_router.py
Eight checks. Every one exists because the model got that specific thing wrong at least once:
| check | what it caught |
|---|---|
| value_is_literal | a fabricated $3,000,000 — no digits, no words, no currency symbol anywhere in the span |
| value_is_currency | 4,998,838,436 shares becoming the largest “deal” in the corpus |
| detect_amount_basis | a $300-per-hour rate sharing a column with a $4.686B facility — 24% of amounts are rates |
| dates.parse_verbose | 2023-01-__, which passed a length-and-dash check and killed a 246k-row backfill |
| quote_is_verbatim | paraphrase sold as quotation — ~10%, tolerant of curly quotes and ellipsis |
| is_amendment_document | 28% of documents modify an earlier one; counting them inflates signing flow |
| classify_party | General Partner typing as a person — 3.1% of party names are bare role words |
| normalize_basis | the model paraphrasing its own enum |
A number the model returns that does not appear literally in the text we sent is dropped, not stored. That rule is the reason the warn tier exists.
9 · deterministic · staging.py, promote.py
The row lands in mars.agreement_extraction_staging_v2 —
ours, reviewable, holding the span we sent and every guard note. Promotion into
the shared table is a separate, deliberate step.
Two rows in two tables, plus the staging record they came from. A real example, promoted and readable through the consumer's own filter today:
VirTra, Inc. party_type=company role=Borrower
UMB BANK, n.a. party_type=company role=LenderBare role words — Borrower with no
name attached — are dropped rather than stored. A party that names nobody
is not a party. company_id is deliberately left NULL for now:
resolution goes through mars's mint gate and gets its own review.
| tier | rule |
|---|---|
| verified | no guard failure, every quote verbatim, two or more named parties |
| vouched | extraction sound, something thin |
| warn | a hard guard failed, or no obligation at all |
WHERE integrity_tier IN ('verified','vouched')
AND is_template = FALSE -- exclude blank forms
AND is_amendment = FALSE -- ONLY when counting signing flow
AND amount_basis = 'total' -- ONLY when comparing deal sizesAcross 8,620 executed documents in the sample. The thin fields are thin because the documents are, not because extraction fails — that distinction was measured, not assumed.
| field | filled | note |
|---|---|---|
| obligation | 100% | |
| two or more named parties | 99% | |
| effective date | 88% | |
| every quote verbatim | 82% | the rest carry at least one paraphrase |
| amount | 48% | most agreements state no total |
| term | 44% | was 31% before the termination window |
Equity awards sit at 10% for amount, and that is ground truth: of 37 executed awards, two state a share count. The exhibit governs vesting and forfeiture; the quantity lives in the grant notice. Award size comes from Form 4, not from us.
Not yet a pipeline. There is no watermark and no timer. Documents arrive by sampling, not by ingest, and the backlog grows ~260 exhibit-bearing filings a day. The health check measures exactly this: 86.6% of the last 30 days' filings are scanned, and that number sheds ~3.5 points a day with nothing running.
The family block is missing on 45% of the rows that should have
it. Found while writing this page. build_prompt selects
the per-family block from the regex guess — so when the regex
punts, no block is sent, and recovered means precisely
“the regex punted.” The VirTra note above is
verified, clean, and has empty
family_fields: we never asked for borrower, lender or facility. It
got them into the obligation sentence anyway, which is why it went unnoticed.
2,013 of 4,442 rows are affected.