Signals — Target Architecture
gate → classify → branch. The blueprint we build against. 2026-08-15.
The principle
A shared gate may only EXCLUDE — never CLASSIFY.
The moment a shared stage decides "this is a deal / not a deal," it hides everything from every
future consumer. not-a-deal ≠ junk. A lawsuit, an appointment, a product launch — all real, all wanted by
someone — must never be dropped just because the deal screen didn't recognize them. The gate answers exactly one
question: "will anyone, ever, want this for anything?" If no → drop. Everything else flows.
The flow
One negative gate, one multi-label classifier, then branch. Nothing consumer-specific happens before the branch.
FIREHOSE
~159K rows/day, rss-led (+ us-courts dockets on their own lawsuit lane). Historical corpus: 16.8M.
▼
① GATE — junk only (fastText, universal, negative)
DROP: spam · truncated · weather / sports / horoscopes / lifestyle / chatter — no signal for anyone.
KEEP: anything plausibly business / corporate / legal / financial. Recall-tuned — when unsure, keep.
fastText quality/topic model + spam-domain list · language-agnostic · zero per-language patterns · FREE
▼ kept
② CLASSIFY — one Qwen pass, MULTI-LABEL
Scores every category at once. An article can be several — a merger that draws a lawsuit, an appointment
buried in an earnings report. Route to ALL that match, never pick one (else the gate-trap reappears at the router).
mergerfundinglitigation
appointmentcontractfinancial_report
productdividend…
scores stored on the row (classify_scores column already exists) · language-agnostic (Qwen reads any language)
▼ branch by score (≥ threshold → that extractor)
③ EXTRACT — per-category deep extractors
Each pulls "rows scored on my category, not yet processed" and does category-specific extraction:
amounts, parties, dates, entity resolution.
merger extractorfunding extractor
litigation extractorappointment extractor…
▼
④ RESOLVE → ⑤ PROMOTE
company_id via the mint gate (dedup-safe) → canonical tables (gated). Unchanged from today.
The cost model — the gate is the whole budget
Two LLM touches: classify runs on 100% of KEPT (the driver) + extract runs on the positives only.
Whatever survives the gate is what you pay classify on — so the gate's cut-rate IS the budget.
| Gate removes | kept/day (of ~130K) | classify $/mo | historical 16.8M |
|---|
| 25% — spam+stub only (today) | ~98K | ~$590 | ~$2,500 |
| 50% — + obvious non-news | ~65K | ~$390 | ~$1,700 |
| 70% — good business-news filter | ~39K | ~$235 | ~$1,000 |
| 80% — aggressive | ~26K | ~$155 | ~$670 |
Qwen classify ≈ $0.0002/article. Every 10 points the gate guts ≈
$150–300 off the one-time backfill and ~$50/mo off steady-state. Escape hatches on the 2-call structure:
merge classify+extract into one call (1 call, bigger prompt on 100%), or cheap-screen classify (free, but
per-language patterns return). Either way, the gate is the lever.
Today → target — what changes
| Piece | Today | Target | |
|---|
| Gate (relevance_tier) | junk + deal-shape (business/low_signal) | junk ONLY (kept / dropped) | rewrite |
| Classify | mars's Qwen (deal-focused, implicit) | our Qwen, multi-label, all categories | build |
| Routing | deal extractor reads 'business' | branch by classify score, per category | build |
| business / low_signal split | set at the gate | gone — replaced by classify scores | retire |
| Per-language deal screens | at the gate (SIG-93/101) | reference only (or in deal extractor) | retire@gate |
| Language detection | rss / fastText | unchanged | keep |
| Extractors (deal/lawsuit/appt) | own screens on 'business' | pull by category score | adapt |
| Resolve / Promote (gated) | mint gate → canonical | unchanged | keep |
Open decisions & next moves
- Ask mars about their current Qwen classify — prompt, model, per-call cost, volume. We inherit it (they're waiting on us); it tells us the real cost and whether merge-classify-and-extract is viable.
- Build + measure the fastText gate cut-rate — the single number that sets the budget. Prototype it, measure "% of firehose guts without dropping real signals" on a labeled sample.
- Decide the LLM structure — two calls (classify + extract, no patterns) vs. one merged call vs. cheap-screen classify. Cost math above; pick after mars's numbers.
- Multi-label discipline — the classifier must score every category and route to all matches. Single-winner routing recreates the gate-trap one level down.
Why now: the retier we ran today sorted rows into business/low_signal via the deal-shaped screen — the exact
distinction this design retires. Stopping tier-level work and designing the gate → classify → branch model first is what keeps
us from building more on the old shape. Companion to the current-state map:
kee.staticpipe.com/mars/2026-08-14-signals-deal-pipeline.html. — signals, with kee.