What it actually costs to build a data pipeline (real numbers)

"How much does it cost to build a data pipeline?" is a fair question with a frustrating answer: it depends. But "it depends" is useless on its own. What it depends on is knowable, and once you see the drivers you can put a real range on your own situation. Here's the honest version.

Why there's no single number

Two projects both called "a data pipeline" can differ by 20x in cost, because these things drive it:

• How many sources, and how messy. One clean Postgres database is a different job from twelve APIs, three of which paginate badly, rate-limit you, and change their schema without warning.
• Batch or real-time. A nightly batch job is straightforward. Sub-minute freshness — streaming, change-data-capture, exactly-once handling — is a different discipline and cost.
• How much transformation and modelling. Moving raw data is cheap. Turning it into correct, tested, business-ready models — with the data-quality checks that keep it correct — is most of the work.
• Volume. Gigabytes fit on one machine and stay cheap. Genuine tens-of-terabytes-per-day changes the architecture and the bill.
• Serving and dashboards. Who consumes it, how many of them, how fast it needs to be.
• Compliance and integration. PII, audit requirements, and having to fit into systems you can't change all add real cost.

The three ways to pay for it

1. Build it in-house. You pay in engineer-time. That's a senior data engineer's fully-loaded cost (often $175k–$250k/year all-in) for the months it takes, plus the opportunity cost of what they're not doing. Cheap-looking if you already have the person idle; expensive if you're hiring for it. (More on that trade-off in consultant vs. full-time hire.)

2. Buy tools and glue them. The modern SaaS stack — a managed ingestion tool, a transformation layer, a cloud warehouse, a BI tool — gets you running fast. The catch is recurring cost that scales against you: per-row ingestion pricing, per-second warehouse credits, per-seat BI. It looks cheap at signup and grows quietly. This is where a lot of "our data costs are out of control" stories start.

3. Bring someone in to build it. A consultant or contractor delivers the pipeline as a project. You pay once for the build (plus whatever recurring infrastructure it runs on), and ideally you own it afterward.

Honest market ranges

Broad strokes, because the drivers above move these a lot:

• A simple pipeline — a handful of clean sources, batch, into one warehouse, with basic dashboards — is typically a low-to-mid five-figure project, or a few weeks of senior time.
• A real platform — many sources, proper modelling and testing, some real-time, ML or serving, governance — runs mid-five to six figures, or a few months.
• The recurring infrastructure to run it can be anywhere from near-zero (self-hosted, right-sized) to thousands a month (a full managed SaaS stack). This is the number people underestimate most.

The build is a one-time cost you can see coming. The recurring bill is the one that surprises you — which is why it's worth estimating your warehouse spend before you commit to an architecture, not after.

The cheapest build is often the most expensive to run

The instinct is to minimise the build cost. The mistake is ignoring what the choices you make during the build do to the monthly bill for years afterward. A pipeline built on per-row and per-second pricing can cost more in eighteen months of running than it did to build. A leaner architecture — right-sized infrastructure, engines that don't meter every query — can cost a little more to design well and a lot less to live with. I run a full production platform this way at near-zero marginal infrastructure cost; the teardown shows how.

How to get a real number for your situation

Sketch your drivers: sources, freshness, volume, how much modelling, who consumes it. That alone turns "it depends" into a range you can reason about. To turn it into an actual number — build cost and the recurring bill — the work is a proper scoping of your specific stack. That's exactly what a Data Platform Audit produces: a week, a written roadmap with effort and cost estimates, yours to keep. The scoping call below is free and there's no pitch.