How I built a paid-grade data product on 100% free public data

A market-data terminal runs five figures a year, per seat. A lot of what people rent it for — sovereign fundamentals, central-bank credibility, currency stress in the emerging and frontier markets the terminals cover badly — is built on data that is completely public. The World Bank, the IMF, the BIS, FRED, the OECD, and the central banks themselves all publish it. For free. In APIs.

So the gap between "free public data" and "a data product a research desk would actually pay for" is not the data, and it isn't the model. It's four unglamorous things: point-in-time correctness, source reliability, licensing diligence, and cost discipline. Get them wrong and you have a demo that looks great in a screenshot and lies the moment someone leans on it. Get them right and you have something people can trust with real decisions.

This is a teardown of one I built solo — EM Macro Watch, live at quantamentry.com — and specifically of those four things, because they're exactly the parts I get hired to do for clients.

What the thing actually is

EM Macro Watch scores the credibility of 171 countries' central banks and macro policy, daily, on a 0–100 scale. The headline number is a weighted blend of seven sub-dimensions: a credibility gap, a behind-the-curve/Taylor-rule signal, communication stance (how hawkish or dovish the bank is talking), geopolitical stress, growth, liquidity, and governance. It's macro and sovereign, not stock-level — the coverage that terminals treat as an afterthought is the whole product here.

Underneath, it's two codebases:

• A Python scoring platform — a Dagster pipeline (35 assets, 7 of them daily-partitioned and independently backfillable), a FastAPI read API, and PostgreSQL — running on a self-hosted Docker Swarm. A daily_credibility_job runs at midnight UTC.
• A Next.js freemium front end where you sign in with a magic link, build a watchlist, and read the scores. The free tier is hard-capped at a three-country watchlist; the code literally reads FREE_WATCHLIST_LIMIT = 3.

The 171 countries are tiered by how much clean data exists for them — 27 "gold," roughly 53 "silver," and about 92 "bronze." That tiering isn't cosmetic; it decides how much work each country is worth, which is the first hint that the interesting problems here are about data quality, not features.

Ten external sources feed it, and every single one is free and public: World Bank WDI, IMF (WEO, IFS, BOP), ILO, BIS, FRED, OECD, GDELT, the ECB, and UN Comtrade. The whole thing sits on roughly 469,000 point-in-time observations across 14 tables. No paid data feed anywhere in it.

Now the four things that make it a product instead of a notebook.

1. Point-in-time correctness, or your backtest is quietly lying

This is the one that separates people who have shipped a data product from people who have shipped a dashboard.

Macro data gets revised. A country's Q1 GDP print you saw in April is not the number that's in the database today — it's been revised twice since. If you score history using the values as they read now, every backtest you run is contaminated with information you couldn't possibly have had at the time. The strategy looks brilliant because it's cheating, and you don't find out until real money is on it.

So nothing in the scoring engine is allowed to ask "what is this value?" Every query asks "what did we know, on the day we're scoring?"

# Wrong: silently reads whatever the value happens to be today
value = latest_observation(country, "CPI_YOY")

# Right: as-of the date we're actually scoring
value = get_pit_observation(country, "CPI_YOY", as_of=target_date)

as_of=target_date runs through the entire codebase; date.today() is effectively banned inside scoring. Making that real needed a schema change: a migration renamed the old ingestion_ts column to first_seen_at, so the database keeps a true vintage archive — when did we first actually see this value — kept deliberately separate from the estimated release_date the source claims it was published. Those two dates are not the same, and treating them as the same is how look-ahead bias sneaks back in.

None of this shows up in a demo. All of it shows up the first time a client asks "would this have worked in 2018?" and you need the answer to be honest.

2. Source reliability is an engineering problem, not luck

Ten public sources means ten things that go down, change shape, or quietly omit the country you need — usually the interesting one.

The default instinct is to let a pipeline fail loudly when a source is missing. That's correct for your own data. It's wrong for a flaky upstream you don't control. The IMF's IFS endpoint is frequently unreachable; if fetching it raised a hard dagster.Failure, the failure would cascade and skip every downstream scoring asset for the night — 171 countries dark because one government API had a bad evening.

So that source is allowed to fail soft. When it's down, the asset logs an error, stamps imf_ifs_outage=True on its metadata, and a documented fallback chain keeps scoring alive on the best data available:

CPI_YOY  →  CPI_HEADLINE  →  WEO_INFLATION

Crucially, the outage is surfaced, not swallowed — it shows up in a public /api/status.json so anyone can see the system is running on a fallback today. Degrading gracefully and lying about it are different things; the first is engineering, the second is how you lose trust exactly once.

The other half of reliability was covering what the free sources don't. The ECB's reference feed is a clean, free source of FX rates — but it omits a lot of the currencies that matter most for this product. So the platform runs custom scrapers against seven official central-bank FX feeds — Ukraine, Georgia, Armenia, Kazakhstan, Colombia, Peru, Uzbekistan — to fill the gaps. That's the unglamorous middle of every data product: the last 10% of coverage is 90% of the bespoke code, and there's no library for it.

3. Licensing diligence: the homework almost nobody does

"It's public data" is not "you can redistribute it." Those are completely different questions, and the difference is the kind of thing that turns a product into a legal problem after it has customers.

So before any of this went live, I did the boring part properly. There's a docs/legal/ folder with a per-source dossier — BIS, IMF, OECD, GDELT, ILO, FRED, Comtrade, World Bank, and the central-bank statements — plus a summary-for-counsel.md. Each dossier rates the pre-counsel risk and, more importantly, maps exactly which kind of API response counts as redistribution: is this a raw observation lifted straight from the source, or a derived value we computed?

That distinction is then enforced in code, not just in a document. A RESTRICTED_SOURCES set — the raw BIS and IMF series with the tightest terms — is SQL-filtered out of the public API. Customers only ever receive derived values: scores, blends, transformations. The public endpoints will never hand back a paywalled single-source raw series as passthrough, because the query that would do it can't.

This is the least fun section to write and the most valuable one to have. When a data product gets acquired, or audited, or sued, this is the folder that decides how the day goes. Building it into the schema — so compliance is a property of the system rather than a promise — is the move a client is really buying when they hire someone who's done this before.

4. Cost discipline you design in, not bolt on

The moment you put an LLM in a daily pipeline, you've built a machine that spends money 365 days a year whether or not anyone's looking. Cost discipline that's a policy gets ignored; cost discipline that's in the control flow can't be.

Here, LLM usage is deliberately small and gated. Claude — reached through a self-hosted, OpenAI-compatible LiteLLM proxy — does two jobs: claude-haiku classifies central-bank statements as hawkish or dovish, and Claude Sonnet writes the daily prose country summaries. Both are boxed in:

• They skip weekdays. The LLM assets run weekends-only, behind an explicit date.today().weekday() < 5 guard. Central-bank stance doesn't move fast enough to justify a daily spend.
• Summaries are tier-gated. Only gold and silver countries get prose. The 92 bronze countries — the ones with the thinnest data — don't burn Sonnet tokens on a paragraph nobody would trust anyway.
• The one paid data source got cut. The platform originally used a paid FX feed (Open Exchange Rates). It was dropped entirely in a "free-data pivot," replaced by the ECB feed plus those seven central-bank scrapers. Free wasn't a constraint I was handed; it was a design goal I kept.

The result is a data product whose marginal cost per day is small enough that it doesn't need customers to survive — which, for something you're building solo, is the difference between a project and a liability.

Two things that went wrong

Teardowns that only show the clean parts aren't worth reading. Two war stories.

The model that faked it. Early on, I blended FinBERT — a general-purpose financial sentiment model — into the hawk/dove stance scoring. It looked reasonable until I actually inspected the per-country outputs and found they were near-constant: Brazil came out 0.60 basically every time, Turkey 0.00. It wasn't discriminating stance at all; it had found a comfortable average and parked there. The fix was to zero its weight, make the LLM the sole authority on stance, and leave a design note to only re-enable blending once a real stance model exists. The lesson isn't "FinBERT is bad" — it's that a model producing plausible numbers and a model producing correct ones look identical until you go read the actual values it emits.

The config that won a fight it shouldn't have. For weeks, production showed no live data, and everything looked fine locally. The front end deploys to a small PaaS, and the root cause was config precedence: the PaaS merges configuration as manifest-env first, then dashboard overrides on top — dashboard always wins. A stale value I'd typed into that dashboard months earlier was silently beating the correct value in my deploy manifest, and it pointed the app at an API hostname gated behind Cloudflare Access, so every request 403'd and the site fell back to empty. Worse, the auto-deploy webhook had never actually been wired, so nothing I pushed was going live to begin with. Two independent silent failures stacked on top of each other. The generalizable lesson: know your platform's config-override order before it teaches you at the worst possible time, and make "is this actually serving live data?" a thing you can see, not assume.

The weight-fit I built and threw away

There's a tempting move with a composite score: fit the seven dimension weights to historical returns and call it rigorous. I built exactly that — a backtest-driven information-coefficient weight fit — ran it, and then rejected it.

It zeroed out two of the seven dimensions entirely and performed worse out-of-sample over a 2025 dollar-bull sample. In other words, it overfit to a regime and threw away signal I had good reason to keep. So I kept the hand-set weights and logged the decision, explicitly, rather than quietly shipping the fancier version because it sounded more sophisticated.

That's the whole discipline in one story. The impressive-sounding thing was measurably worse, and the job was to notice that and write down why — not to perform rigor, but to actually have it.

Why this is really a consulting post

I can build a Next.js front end and a Dagster pipeline. So can a lot of people, and so can a lot of tools now. That was never the hard part, and it was never the point of this teardown.

The hard parts — the parts that decide whether a data product is trustworthy, legal, and affordable to run — are point-in-time correctness, source reliability, licensing diligence, and cost discipline. They don't demo well. They don't show up in a screenshot. They show up eighteen months later when someone bets on your numbers, or reruns your history, or reads your terms of use with a lawyer next to them.

Those four things are what I do for clients: taking messy public and vendor data and turning it into something a business can actually stand on. If you're trying to build a data product and you can feel that the risk isn't in the code, let's talk.

And if you want the longer version of how I think about building analytics you can trust on data you actually control, that's what my book is about — you can grab it free here.