Do people actually use DuckDB in production?

Yes, and far more than the blog posts suggest — because it usually hides. DuckDB rarely appears on an architecture diagram as "our warehouse"; it appears inside a batch job, a service, a CLI, or a notebook that runs in CI. It is an engine, not a platform, so real usage is largely invisible from outside. It is past 1.0, MIT-licensed, with a foundation and a company behind it rather than one maintainer with a day job.

What is DuckDB good for?

Four shapes, reliably. Transform-heavy batch jobs that read Parquet, join, aggregate and write Parquet back. Embedded analytics inside your own service, so query capability ships with the app instead of calling a warehouse. Local and CI data work that runs identically on a laptop and in a pipeline. And as the fast interactive layer over a Parquet lake in object storage. What it is not good for: dozens of people querying one shared instance, governed company-wide data with roles and audit, OLTP writes, or streaming freshness measured in seconds.

How do I use DuckDB in production?

Three deployment patterns work. In-process inside a service — your API imports DuckDB and queries it directly, with your API acting as the auth and concurrency layer. A containerised batch job that starts, reads Parquet from object storage, transforms, writes back and exits; this is the lowest-risk pattern and where most of my own usage lives. Or a Parquet lake with many read-only readers querying in parallel. The anti-pattern to avoid: several processes on a shared network filesystem all wanting write access to one DuckDB file. That is the setup that produces "DuckDB is unreliable" posts.

What are the best practices for using DuckDB in production?

Keep Parquet in object storage as the source of truth and treat the .duckdb file as a derived artifact you can rebuild — that turns durability from a bet into a routine operation. Let exactly one process hold write access. Put your own service in front so it owns auth, rate limiting and concurrency. And set DuckDB's memory limit below the container's, always: it runs inside your process, so a join that fans out further than expected takes down the whole service rather than one query. In-process is a feature and a blast radius at the same time.

Can DuckDB be used as a lightweight data warehouse for a small SaaS product?

Yes, inside a specific shape — worth knowing the shape before you commit rather than after. One writer holds write access and everything else reads. Users never connect to DuckDB; they hit your API, which queries it in-process. Parquet in object storage is the source of truth. The working set fits on one machine — gigabytes to low terabytes. Freshness is measured in minutes or hours, not seconds. Inside that shape it is genuinely excellent and dramatically less infrastructure than a warehouse. Outside it, the failures are predictable: a second team wanting its own access, row-level permissions per customer, or a working set that stops fitting on the box.

What's the best way to set up a production DuckDB database for faster querying, and can I run it on my own infrastructure?

Running it on your own infrastructure is the normal case, not the exception — there is no service to buy, because DuckDB runs inside your process. For speed, the wins are mostly in the data layout rather than the engine: store Parquet, partition it so queries can skip whole files, keep the hot working set inside RAM, and set an explicit memory limit so larger-than-memory work spills to a temp directory with real room on it instead of being discovered under load. You scale up, not out — one bigger machine, which goes further than most people expect but is a hard ceiling rather than a soft one.

I keep seeing DuckDB benchmarks. How do I actually use it to analyse data quickly — what infrastructure do I need, and can I just buy it as SaaS?

Start with no infrastructure at all: install the Python package or the CLI on your laptop and point it at Parquet or CSV files, including files sitting on S3. There is no server and no load step, and quick analysis is DuckDB's easiest win. The benchmarks are real, but they measure the engine on well-laid-out data — the speed you get depends mostly on your files: Parquet rather than CSV, partitioned so queries skip what they do not need. Infrastructure only enters when a second person or a scheduled job needs the same data; then keep the Parquet in object storage as the shared copy and run DuckDB wherever the work runs. If you want it as a service, MotherDuck is DuckDB hosted — same engine and SQL, someone else storing and sharing the data. Buy that for sharing and a managed home for the data, not for speed: for one person's analysis, the free local install is already the fast version.

June 27, 2026

DuckDB in production: what it's actually good at (and what it isn't)

By Arshad Ansari

DuckDB is having a moment, and like every tool having a moment, the hype runs ahead of the nuance. "Just use DuckDB" is now a reflex answer to questions it doesn't actually fit. So let me give you the honest version, from someone who ships it.

What DuckDB is

DuckDB is an in-process analytical (OLAP) database — think "SQLite for analytics." It runs a columnar, vectorized query engine inside your application process. No server, no network hop, no cluster. You point it at Parquet, CSV, or its own format and run real SQL — window functions, joins, the works — at speeds that embarrass a round-trip to a cloud warehouse for the same-size data.

Is DuckDB production-ready? Do people actually run it?

Yes — with a shape attached, and the shape is the entire answer.

"Production-ready" doesn't mean "safe to drop in anywhere." It means the engine is stable, the file format is stable, and the failure modes are known and boring. DuckDB clears that bar. It's past 1.0, it's MIT-licensed, and there's a foundation and a company behind it rather than one maintainer with a day job. In the systems I run it has never lost data or quietly returned a wrong answer. The bugs have all been in my SQL.

What people are usually asking is the less polite version: do serious teams run this, or am I the guinea pig? They run it — mostly in a shape you can't see from outside. DuckDB rarely appears on an architecture diagram as "our warehouse." It appears inside a batch job, a service, a CLI, a notebook that runs in CI. It's an engine, not a platform, so it hides inside things, which is why real usage is far more common than the blog posts about it.

The caveat worth internalising: production-readiness here is a property of the system you build around it, not of the database. A warehouse hands you concurrency, access control and a serving layer in the box. DuckDB hands you a query engine and expects you to bring the rest. Bring it, and it's production-ready. Expect it included, and you'll conclude the tool is immature when what's missing is your service layer.

Where it shines in production

Transform-heavy batch jobs. Reading Parquet, joining, aggregating, and writing Parquet back out. DuckDB will saturate your cores and finish before a warehouse has authenticated your session.
Embedded analytics in an app or service. Ship query capability inside your service instead of calling out to a warehouse. Lower latency, no per-query bill, fewer moving parts.
Local and CI data work. Notebooks, ad-hoc analysis, and test fixtures that run identically on a laptop and in CI — no shared environment to provision or pollute.
The "interactive" layer over a lake. Parquet in object storage as the source of truth, DuckDB as the fast query engine over it. You get warehouse-ish ergonomics without the warehouse.

Can DuckDB be the warehouse for a small SaaS?

Yes — up to a specific shape. Worth knowing the shape before you commit rather than after.

The shape that works:

One writer. A single process — your ETL job, your API — holds write access. Everything else reads.
It sits behind your own service. Your users never connect to DuckDB. They hit your API, which queries DuckDB in-process and returns JSON. Your API is the concurrency layer, the auth layer and the rate limiter.
Parquet in object storage is the source of truth. The .duckdb file is a materialisation you can rebuild from scratch, not the only copy of the company's data.
The working set fits on one machine — gigabytes to low terabytes, with the hot tables comfortably inside RAM and disk.
Freshness is measured in minutes or hours, not seconds.

Inside that shape it's genuinely excellent, and it's dramatically less infrastructure than the alternative — no warehouse to keep warm, no per-query bill, no credentials rotation, and a stack a single engineer can hold in their head. The local-first walkthrough is the concrete version of this pattern in code: Parquet on disk or in a bucket, SQL straight over it, nothing in between.

Outside that shape, the failures are predictable rather than mysterious. The day a second team wants its own access. The day someone asks for row-level permissions per customer. The day the working set stops fitting on the box. The day "how fresh is this?" needs to be answered in seconds. Those are the signals I wrote up in when local-first runs out of road. Short version: one or two of them means fix your setup, not your architecture. All of them means graduate, and graduate deliberately. When the workload that forces the move is constant sub-second dashboards, the decision that follows is ClickHouse vs Snowflake.

The limitations, in one paragraph

Four things bound it, and none of them are subtle: one process writes at a time (many can read, but there is no shared server fifty clients connect to); there is no governance layer — the file is the permission boundary, so whoever can read it reads everything; memory is the failure you'll actually hit, because DuckDB runs inside your process and an OOM kill takes the whole service with it; and you build the serving layer yourself — auth, concurrency, caching, rate limiting, metrics, none of which ships with the database.

That's the summary. The full treatment — the single-writer model in detail, what ACID does and doesn't cover here, how spill behaviour actually bites, where DuckDB is genuinely the wrong choice, and a pre-production checklist — is its own post: is DuckDB safe for production? the honest limitations. If you are trying to decide whether the edges disqualify it for your workload, read that one rather than this one.

The short version of the safety question, since it's the one I get asked most: the real bet is on Parquet and Arrow, not on DuckDB, because keeping Parquet in object storage as the source of truth makes the .duckdb file a derived artifact you rebuild rather than a thing you can lose. And the useful risk measure isn't a prediction about the project — it's the exit cost. If DuckDB vanished tomorrow you'd point another engine at the same files and fix some dialect quirks. Leaving a proprietary warehouse is a migration project with a budget line. Low switching cost is the safety.

Three deployment patterns that work

1. In-process inside a service. Your API imports DuckDB and queries it directly — no network hop, no query service to operate, microseconds instead of milliseconds of overhead. One process writes, the API reads, your service handles auth and concurrency. Watch memory: DuckDB's limit goes below the container's, and a runaway query taking out the process takes out the request handler with it.

2. Containerised batch job. Ephemeral container starts, reads Parquet from object storage, transforms, writes Parquet back, exits. Nothing to keep alive between runs, retries are free, and "scaling" means a bigger box for ten minutes. Point cron, GitHub Actions, Dagster or Airflow at it and you're done. This is the lowest-risk pattern of the three and it's where most of my DuckDB usage lives.

3. Parquet lake with DuckDB readers. Object storage holds the truth; many independent read-only readers — a batch job, a notebook, a service, even the browser via duckdb-wasm — query it in parallel. Reads fan out infinitely because there's no shared server to contend on. You manage consistency by writing new partitions and swapping a prefix or manifest, rather than mutating files in place.

The anti-pattern, for completeness: several processes on a shared network filesystem all wanting write access to one DuckDB file. Don't. That's the setup that produces "DuckDB is unreliable" posts.

At a glance: where DuckDB fits

Workload shape	DuckDB?
Batch transforms over files — read Parquet, join, aggregate, write back	Yes
Embedded analytics inside one service, behind your own API	Yes
Read-only query layer over a Parquet lake	Yes
Local and CI analysis, reproducible reports, test fixtures	Yes
The warehouse for a small SaaS — single writer, working set on one box	Yes, within the shape above
Dozens of people querying one shared instance directly	No — put a service in front, or use a warehouse
Governed company-wide source of truth: roles, audit, sharing	No
High-rate concurrent writes, OLTP, application primary store	No — that's Postgres
Constant sub-second dashboard load from many users at once	No — ClickHouse territory
Streaming ingest with seconds-level freshness	No
Hot working set larger than one machine	No

The pattern that works

The mistake is treating DuckDB as a replacement for your warehouse. The win is treating it as the right-sized compute engine for the large fraction of your workload that never needed a warehouse in the first place:

Truth lives in Parquet. DuckDB is the fast, cheap, serverless way to ask questions of it. Reach for the warehouse only for the workloads that genuinely earn it.

Done this way, DuckDB isn't a toy or a hype cycle — it's the least infrastructure that actually solves the problem. Which, in my experience, is almost always the right amount.

Going deeper

Three follow-ups that answer the questions this post tends to raise:

Is DuckDB safe for production? — the single-writer model, durability semantics, and where it's genuinely the wrong choice.
Running DuckDB on your own infrastructure — the memory, temp-directory and partitioning settings that decide whether a single node feels fast or slow.
Can DuckDB be your SaaS product's warehouse? — per-tenant partitioning for customer-facing analytics, and where the ceiling actually is.
DuckDB vs ClickHouse — a library in your process against a server for many clients, and the pattern that runs both.

The book: Local-First Analytics is the long version of all this — DuckDB, Parquet and Arrow in practice, with runnable code and real datasets. It's on Amazon, and if you'd like to read it before you buy it, you can request a review copy — I send those out by hand.

Using it to cache LLM responses is its own question, with its own single-writer trap: DuckDB as an LLM cache.

Trying to figure out where DuckDB fits in your stack without betting the company on it? Let's talk.

Common questions

Do people actually use DuckDB in production?: Yes, and far more than the blog posts suggest — because it usually hides. DuckDB rarely appears on an architecture diagram as "our warehouse"; it appears inside a batch job, a service, a CLI, or a notebook that runs in CI. It is an engine, not a platform, so real usage is largely invisible from outside. It is past 1.0, MIT-licensed, with a foundation and a company behind it rather than one maintainer with a day job.
What is DuckDB good for?: Four shapes, reliably. Transform-heavy batch jobs that read Parquet, join, aggregate and write Parquet back. Embedded analytics inside your own service, so query capability ships with the app instead of calling a warehouse. Local and CI data work that runs identically on a laptop and in a pipeline. And as the fast interactive layer over a Parquet lake in object storage. What it is not good for: dozens of people querying one shared instance, governed company-wide data with roles and audit, OLTP writes, or streaming freshness measured in seconds.
How do I use DuckDB in production?: Three deployment patterns work. In-process inside a service — your API imports DuckDB and queries it directly, with your API acting as the auth and concurrency layer. A containerised batch job that starts, reads Parquet from object storage, transforms, writes back and exits; this is the lowest-risk pattern and where most of my own usage lives. Or a Parquet lake with many read-only readers querying in parallel. The anti-pattern to avoid: several processes on a shared network filesystem all wanting write access to one DuckDB file. That is the setup that produces "DuckDB is unreliable" posts.
What are the best practices for using DuckDB in production?: Keep Parquet in object storage as the source of truth and treat the .duckdb file as a derived artifact you can rebuild — that turns durability from a bet into a routine operation. Let exactly one process hold write access. Put your own service in front so it owns auth, rate limiting and concurrency. And set DuckDB's memory limit below the container's, always: it runs inside your process, so a join that fans out further than expected takes down the whole service rather than one query. In-process is a feature and a blast radius at the same time.
Can DuckDB be used as a lightweight data warehouse for a small SaaS product?: Yes, inside a specific shape — worth knowing the shape before you commit rather than after. One writer holds write access and everything else reads. Users never connect to DuckDB; they hit your API, which queries it in-process. Parquet in object storage is the source of truth. The working set fits on one machine — gigabytes to low terabytes. Freshness is measured in minutes or hours, not seconds. Inside that shape it is genuinely excellent and dramatically less infrastructure than a warehouse. Outside it, the failures are predictable: a second team wanting its own access, row-level permissions per customer, or a working set that stops fitting on the box.
What's the best way to set up a production DuckDB database for faster querying, and can I run it on my own infrastructure?: Running it on your own infrastructure is the normal case, not the exception — there is no service to buy, because DuckDB runs inside your process. For speed, the wins are mostly in the data layout rather than the engine: store Parquet, partition it so queries can skip whole files, keep the hot working set inside RAM, and set an explicit memory limit so larger-than-memory work spills to a temp directory with real room on it instead of being discovered under load. You scale up, not out — one bigger machine, which goes further than most people expect but is a hard ceiling rather than a soft one.
I keep seeing DuckDB benchmarks. How do I actually use it to analyse data quickly — what infrastructure do I need, and can I just buy it as SaaS?: Start with no infrastructure at all: install the Python package or the CLI on your laptop and point it at Parquet or CSV files, including files sitting on S3. There is no server and no load step, and quick analysis is DuckDB's easiest win. The benchmarks are real, but they measure the engine on well-laid-out data — the speed you get depends mostly on your files: Parquet rather than CSV, partitioned so queries skip what they do not need. Infrastructure only enters when a second person or a scheduled job needs the same data; then keep the Parquet in object storage as the shared copy and run DuckDB wherever the work runs. If you want it as a service, MotherDuck is DuckDB hosted — same engine and SQL, someone else storing and sharing the data. Buy that for sharing and a managed home for the data, not for speed: for one person's analysis, the free local install is already the fast version.

Working on this for real? Run the 18-check data platform teardown against your own stack — free, no email. Or read what a fractional data engineer actually does.

Does DuckDB fit your system?

The 16-question production-fit checklist I run before putting DuckDB on a critical path — writers, working set, durability, memory, and who talks to it. Each question comes with what a bad answer sounds like.

One email with the whole checklist. Nothing follows it. Reply and it reaches me, not a queue.

Want the whole playbook?

If this was useful, the long version is my book. Local-First Analytics — 314 pages, runnable code for every chapter — is the full build: DuckDB, Parquet and Arrow, from install to production. On Amazon, or request a free review copy.

Get the book

Not ready to buy? Read chapter 1 free — the whole chapter, no email required.

Rather talk it through? Book a free 30-minute call. No slot that suits your time zone? Email [email protected].