lp_data · plan + progress · 2026-07-24

The hardest vertical — and why we can build it anyway.

Yesterday I said LP data would be hard because Preqin and Cambridge Associates operate on voluntary, confidential GP data-sharing under NDA. That's a closed system we cannot reproduce. What I underweighted is how much of that closed data leaks into public disclosures. Enough that a 30–40%-coverage MARS product is real. Here's the plan and where we are as of end-of-day today.

The honest scoping

Preqin's real edge

Voluntary GP data

GPs give them fund performance under NDA in exchange for benchmark access. Closed system. Can't be replicated by scraping.

What leaks publicly

30–40%

US public pensions must disclose LP commitments + fund IRR/TVPI by state records law. Sovereigns like Norway GPFG publish everything. UK charities, DFIs (IFC/EIB), 990-PFs.

Our positioning

Primary-source

"Preqin depends on GP self-report under NDA. We source from public disclosures. Every number traces to a filing." That's a real differentiator.

LP data is not impossible. It's hard in a specific way — you can't get the complete picture, and you have to be honest that you can't. The advantage is that the complete picture is nobody's actual buying criterion. Auditors, family offices, and hedge funds want traceable data more than complete data.

The four-table plan

Schema draft in /home/kee/code/lp_data/SCHEMA.md · applied to mars postgres today.

Table	What it stores	Source	Status
funds_v2	Canonical fund vehicles — Francisco Partners VIII, KKR NA V, Sequoia Growth IX, etc. One row per real fund, feeders and share classes absorbed.	Adrian's Form D substrate (170K distinct entity names) via normalizer + dedup	157,472 rows
pension_funds_v2	Canonical LP entities — CalPERS, CalSTRS, NYSCRF, Norway GPFG, IFC, Ford Foundation, etc.	Hand-curated seed of top 15 US public pensions; expand in batches via SOURCES.md	15 LPs · $2.29T
fund_performance_v2	IRR / TVPI / DPI / RVPI per (fund, LP, as-of date). One CalPERS-reported IRR for KKR NA V can differ from CalSTRS's — that's real signal, not noise.	Public pension CAFRs (Grok DEEP extraction from PDFs), sovereign wealth reports, DFI annual reports	Empty · Phase 2
fund_commitments_v2	LP → fund commitment records — the killer table. "Boston Retirement committed $50M to Francisco VIII on 2026-Q1."	Same source as performance — public pension CAFRs + consultant board packets	Empty · Phase 2

Where we are today

Phase 1 substrate LIVE. Schema applied. 157K canonical funds minted from adrian's Form D substrate. 15 top public pensions seeded ($2.29T AUM). Health check check_lp_data monitoring both tables in the daily 13:00 UTC email. Committed da0bc2f + 8914875 + 9ea1983.

Sanity spot-check: Francisco Partners

The company from yesterday's PitchBook Daily Pitch article. Their 53 raw Form D filings (VIII / VIII-A / VIII-B / VIII-C plus Agility IV / IV-A / IV-B / IV-C plus iCapital + AlphaKeys feeders plus older vintages) now roll up to 13 canonical funds spanning 2008–2026:

Fund	Vintage	Vehicles absorbed
Francisco Partners VIII	2026	8
Francisco Partners Agility IV	2026	5
Francisco Partners VII	2022	5
Francisco Partners Agility III	2022	4
Francisco Partners VI	2020	5
Francisco Partners Agility II	2020	4
Francisco Partners V	2017	3
Francisco Partners IV	2014	2
Francisco Partners III (+ Parallel III)	2011	2
... + Agility I & older		15 total

The unrelated SAN FRANCISCO PARTNERS LP hedge fund is correctly NOT linked to the Francisco Partners GP. That's the discipline signal — the normalizer isn't over-collapsing.

The 5-phase roadmap

SEC-derived fund + LP entity substrate

This week. Schema live, 157K funds + 15 pensions seeded, health checks in place. Cost: ~$0. Substrate size scales for free with adrian's Form D ingest.
Public pension CAFR extraction

CalPERS + CalSTRS first — largest ($469B + $336B) and most transparent. Grok DEEP extraction of IRR/TVPI/DPI + LP commitment tables from public PDFs. One pension's full 15-year archive per week. Cost: ~$50–100 per pension for full history.
Sovereign wealth + Canadian pensions

Norway GPFG (fully transparent), NZ Super, Australia Future Fund, CPP / OTPP / PSP / BCI (limited but real). Each sovereign adds ~2–5% of global institutional AUM to substrate.
UK charities · DFIs · US foundations

Charity Commission + Companies House (UK), IFC / EIB / EIF / IDB / ADB / AfDB annual reports, US 990-PF foundation filings. High signal per source — each DFI lists every fund and every LP.
MCP-native surface + editorial output

Once P2 populates performance + commitments, expose as mars_lp_commitments + mars_fund_performance MCP tools. Feed a monthly "LP Signal" newsletter, complement to the SEC Daily Signal (#217).

Cost + budget

Phase 1 (done)

Total: ~$0. Substrate came from adrian's existing Form D ingest; hand-curated pension seed; Python-only dedup. This is what a strong platform pays for.

Phase 2 (running estimate)

~$500/month at Grok DEEP for CAFR PDF extraction across 15 top pensions over ~2 months. Backfill of ~15-year archives per pension is ~$50–100 each. First results usable within 2 weeks.

Why this actually works

The moment we can name which LPs committed to Francisco Partners VIII from public CAFRs, we can run a story PitchBook's editorial cadence takes 5 months to publish. That's not "compete on completeness" — that's "compete on speed × traceability" against a competitor whose model is slow but complete.

Three product hooks that fall out of the substrate once Phase 2 lands:

Traceable LP commitment feed — "which public pensions committed to X fund in the last quarter, with source CAFR page number." Feed the SEC Daily Signal newsletter.
Fund performance benchmarks by LP — for the 30–40% of the universe we can source. Different LPs' IRR reports on the same fund is a real analytical primitive; nobody publishes that.
Per-pension portfolio X-ray — every fund CalPERS has ever committed to, with vintage, commitment size, and reported returns. Consumer-facing report per pension.

What could go wrong (and what we do about it)

Risk	What it looks like	Mitigation
CAFR PDF extraction fails	Grok DEEP mis-parses IRR tables; wrong numbers written to substrate.	integrity_tier discipline. Every extracted number carries source_document + source_page. Cross-check against Preqin's free public benchmarks where they overlap.
Fund-to-Form D linkage wrong	CalPERS commits to "Francisco Partners VIII" but our fund_id doesn't match adrian's Form D issuer.	Fund name-canonicalization + gp_investor_id gets fuzzy trigram sweep in Phase 2. Then Qwen-verify borderline cases.
Sparse coverage undermines pitch	Analyst asks "did BlackRock's private funds get any pension commitments?" and we return empty.	Frank about coverage limits in the MCP response envelope (coverage_note like argos did on has_named_backers). "Preqin is complete; MARS is traceable" is the pitch, not "MARS is complete."
lp_data eats bandwidth we don't have	P2 CAFR extraction turns into a per-pension slog; we ship 3 pensions and stall.	Ship CalPERS first, in full. It's ~15% of US public pension AUM. If CalPERS alone doesn't produce a compelling demo, kill it. If it does, batch the next 5.

Bottom line

Yes, LP data is hard. Preqin's closed data-sharing model is genuinely unreproducible.

But 30–40% coverage from primary sources is real, that coverage skews to exactly the LP class our target buyers (top-tier pensions, Callan-class advisors, hedge funds) care about most, and the traceability positioning is a differentiator, not a compromise.

Phase 1 substrate is live today. Phase 2 kicks off with CalPERS as the ship-first-then-scale demo. If CalPERS lands cleanly in 2 weeks, we run the full top-15 batch. If it doesn't, we know quickly and cheaply.

Not managing it as a separate project — it's a mars sub-vertical writing to mars.*_v2 tables. Same platform contracts, same design system, same MCP-first delivery pattern as the other verticals we own.

Live substrate: mars.funds_v2 157,472 rows / mars.pension_funds_v2 15 rows / $2.286T total AUM. mars.fund_performance_v2 + mars.fund_commitments_v2 both empty by design (Phase 2). Health check: mars_feed/health.py:check_lp_data. Ops guide: /home/kee/code/lp_data/CLAUDE.md. 24-week roadmap: HANDOFF.md. Schema: SCHEMA.md.

Do NOT ingest from: Preqin, Cambridge Associates proprietary, PitchBook data feeds, PEI Media data products, Wilshire TUCS, eVestment, StepStone proprietary, DealIQ, Palico. Their FREE published research is OK to reference. Full list: /home/kee/code/lp_data/README.md.

Positioning contrast: Preqin sells the complete picture at $50-150K/yr institutional sub. MARS sells the SEC-first traceable slice at whatever price the substrate quality can sustain, complementary to their existing Preqin sub.