Postgres and Iceberg: pg_lake, pg_duckdb and the Postgres lakehouse
By Arshad Ansari
"Postgres Iceberg" covers four different jobs. You might want to query Iceberg or Parquet files on S3 from Postgres. You might want Postgres to write Iceberg tables. You might want Postgres to be the catalog that other engines use. Or you might want your Postgres data to land in Iceberg for a warehouse to read. Each job has a different tool, and two of the tools people still recommend are no longer maintained.
This post sorts them out. I have not run pg_lake or Iceberg in production. What I did run, on my laptop: PyIceberg 0.12.0 with a plain Postgres 16 container as the Iceberg catalog, and the pg_duckdb Docker image (Postgres 18.1) reading the same table. The numbers below come from those runs. Everything else links to its source, checked on 1 October 2026.
Postgres and Iceberg: the options at a glance
| Option | Job | Licence | Latest release | Status |
|---|---|---|---|---|
| pg_lake | Write and read Iceberg, Postgres as catalog, COPY to/from S3 | Apache-2.0 | v3.5.3, 2026-09-24 | Active |
| pg_duckdb | Read Parquet, CSV, JSON, Iceberg, Delta; fast analytics | MIT | v1.1.1, 2025-12-18 | Active, PG 14 to 18 |
| PyIceberg / Iceberg JDBC SQL catalog | Plain Postgres as the catalog only | Apache-2.0 | PyIceberg 0.12.0 | Active |
| Snowflake Postgres | Managed Postgres with pg_lake | Commercial | GA 2026-02-24 | Active |
| Supabase Analytics Buckets + Supabase ETL | CDC from Postgres into Iceberg on S3 Tables | ETL is Apache-2.0 | — | Public alpha |
| Databricks Lakebase | Managed Postgres, sync to the lakehouse | Commercial | GA on AWS 2026-02-03 | Sync is beta, Delta only |
| pg_mooncake | "Real-time analytics on Postgres tables" | MIT | v0.1.3 | Dormant: team joined Databricks |
| ParadeDB pg_analytics | DuckDB-backed lake queries | PostgreSQL | — | Archived 2025-03-19 |
The short version: pg_lake if Postgres should own Iceberg tables, pg_duckdb if Postgres only needs to read the lake, and a managed product if you want someone else to run the plumbing.
pg_lake: Postgres that writes Iceberg
pg_lake began as Crunchy Data Warehouse. Snowflake bought Crunchy Data in June 2025 and open-sourced the code as pg_lake in November 2025. Per its README, it lets you:
- Create and change Iceberg tables from Postgres, inside ordinary transactions.
- Query Parquet, CSV, JSON and Iceberg files in object storage.
COPYquery results out to S3 as Parquet, CSV or JSON, and back in.
CREATE EXTENSION pg_lake CASCADE;
CREATE TABLE events (id bigint, ts timestamptz, kind text)
USING iceberg;
INSERT INTO events VALUES (1, now(), 'view');
COPY (SELECT * FROM events) TO 's3://bucket/exports/events.parquet';
The design has two parts. Postgres runs the pg_lake extensions. A separate process, pgduck_server, runs DuckDB and speaks the Postgres wire protocol locally. pg_lake hands scans and computation to it. You don't query it directly, but you do deploy and size it: it defaults to 80% of system memory, per the README.
From the Iceberg tables doc:
- Postgres is the catalog. Commits update pg_lake's catalog tables in the same transaction as the data change. An
iceberg_tablesview lets Spark, PyIceberg and iceberg-rust find the tables through their JDBC or SQL catalogs. - REST catalogs work too. A table can live in an external REST catalog, so other engines see it where they already look.
- Compaction is built in.
VACUUMmerges small files and removes expired ones, and autovacuum runs it every 10 minutes. - Some SQL is not supported on Iceberg tables:
MERGE,INSERT ... ON CONFLICT,SELECT ... FOR UPDATE. An update or delete locks the table, so only one runs at a time. - Other engines read, Postgres writes. External tools cannot yet write to tables pg_lake created. Tables attached from a REST catalog are read-only in pg_lake.
The managed form is Snowflake Postgres, generally available since 24 February 2026. Since 14 July 2026 a catalog integration lets Snowflake discover those tables and read them as read-only Iceberg tables.
I did not run pg_lake for this post. Its Docker setup builds the images locally rather than pulling a ready one, and I chose to test the parts I could run honestly in an afternoon.
Postgres as the Iceberg catalog, concretely
A catalog sounds like a big service. With a SQL catalog it is two tables. I pointed PyIceberg at a throwaway Postgres 16 container, created a table and appended 50 batches of 1,000 rows. Postgres then held:
List of relations
Schema | Name | Type
--------+------------------------------+------
public | iceberg_namespace_properties | table
public | iceberg_tables | table
And iceberg_tables held one row:
table_namespace | table_name | metadata_location | previous_metadata_location
-----------------+------------+------------------------------------+-----------------------------------
shop | events | .../metadata/00055-0ecd....json | .../metadata/00054-6dea....json
That is the whole catalog: a pointer to the current metadata.json and the one before it. The data and every other metadata file live in storage. Postgres holds a few hundred bytes per table.
A commit is a compare-and-swap. PyIceberg's SQL catalog runs an UPDATE ... WHERE metadata_location = <the one I started from>. If no row matches, someone else committed first, and it raises CommitFailedException. I tested it by opening the table twice and appending from both. The second append conflicted, PyIceberg retried 108 ms later on top of the new state, and both rows landed.
Fifty commits took 0.99 seconds against Postgres, about 20 ms each, and 0.84 seconds against SQLite. Locally the catalog is not the bottleneck. Writing the files is.
pg_lake's iceberg_tables view has the same shape, which is why PyIceberg and Spark can use it. If you already run Postgres and need a catalog for a few tables, a SQL or JDBC catalog on that Postgres is the cheapest catalog there is. You give up what REST catalogs add: access control, credential vending and a standard HTTP API for every engine.
pg_duckdb: read the Postgres data lake from inside Postgres
pg_duckdb embeds DuckDB in the Postgres process. I covered what it does for analytical queries in DuckDB vs Postgres. For the lake, it adds DuckDB's readers: read_parquet, read_csv, iceberg_scan and delta_scan against S3, GCS, Azure and R2.
I ran the pgduckdb/pgduckdb:18-v1.1.1 image with my test table's folder mounted read-only and queried it:
SELECT duckdb.install_extension('iceberg');
SELECT r['kind'] AS kind, count(*)
FROM iceberg_scan('/path/to/events/metadata/00055-....metadata.json') r
GROUP BY 1 ORDER BY 1;
kind | count
------+-------
buy | 17000
cart | 17000
view | 333
The first query took 1.08 seconds, including DuckDB loading the extension. The group-by after it took 4 ms. It returned the table's current 34,333 rows, after a delete and a column rename, which a plain Parquet glob over the same folder gets wrong.
One gotcha cost me a rerun. Iceberg metadata stores absolute file paths. I first mounted the folder at a different path inside the container, and the query failed with Cannot open file ".../snap-....avro". The manifest list was where the metadata said, just not where the container could see it. On S3 this rarely bites, since the URI is the same from everywhere. It does bite when you copy a table to another bucket or mount it differently.
pg_duckdb's extension docs describe its Iceberg support as functions to read Iceberg tables. For writing Iceberg from Postgres, pg_lake is the tool built for it.
pg_lake vs pg_duckdb
- Reading lake files from Postgres: both. pg_duckdb is one extension in the Postgres process. pg_lake adds a separate DuckDB server to deploy and watch.
- Writing Iceberg: pg_lake. Transactional, with Postgres as the catalog.
- Speeding up analytics on ordinary Postgres tables: pg_duckdb's main purpose.
- Keeping Iceberg tables healthy: pg_lake's autovacuum compacts and expires. With pg_duckdb, whoever writes the table owns that.
- Managed option: pg_lake via Snowflake Postgres.
If you only need to query files now and then, start with pg_duckdb. If Postgres should be where lake tables are born, use pg_lake.
Postgres CDC to Iceberg
Sometimes the goal is not to query the lake from Postgres but to get Postgres data into it. In October 2026 the options are:
- Supabase: Analytics Buckets store Iceberg tables on Amazon S3 Tables, with automatic compaction. Supabase ETL (Apache-2.0) streams changes from Postgres into them, and an Iceberg foreign data wrapper queries them back. It is a public alpha.
- Databricks Lakebase: its Lakehouse Sync is in beta and replicates to Delta tables in Unity Catalog, not Iceberg. Fine if Databricks is your warehouse. Not an Iceberg answer.
- Snowflake Postgres: not CDC. pg_lake writes Iceberg tables in Postgres, and the catalog integration makes them readable in Snowflake. Snowflake's own guide uses
pg_incrementalto export only new rows on each run. - Do it yourself: a scheduled job that reads changed rows by a watermark column and appends them with PyIceberg. It works, and it is what I'd prototype first. You then own small files, compaction and snapshot cleanup, and my laptop runs showed those grow fast with frequent small commits.
Dormant and archived: check before you build
Two names still come up in older posts.
pg_mooncake (MIT) describes itself as "real-time analytics on Postgres tables". Databricks acquired Mooncake Labs on 1 October 2025. The last release is v0.1.3 and the last commit I found is from 26 October 2025. Treat it as dormant.
ParadeDB pg_analytics was archived on 19 March 2025.
Neither is a base for new work.
Do you need a Postgres lakehouse at all?
Most teams asking "Postgres data lake" have a slow analytical query on Postgres. That is usually fixed with indexes, a replica or a summary table first. If not, it's a columnar engine next to Postgres, not a lake. DuckDB vs Postgres and ClickHouse vs Postgres cover those routes.
A Postgres lakehouse earns its place when the data has to be shared: a warehouse, Spark or another team reads the same tables, or history must outlive what you want to keep in Postgres. Then pg_lake, or a managed version of it, is the most direct route in 2026.
So: which one?
- Query the lake now and then: pg_duckdb.
- Postgres owns Iceberg tables others read: pg_lake, or Snowflake Postgres if you want it managed.
- A catalog for a few tables, on Postgres you already run: PyIceberg's SQL catalog or Iceberg's JDBC catalog.
- Stream Postgres changes into Iceberg: Supabase ETL if you are on Supabase and accept an alpha. Otherwise a batch job you own.
- pg_mooncake or pg_analytics: neither, for new work.
If you are still deciding whether you need a table format at all, start with Iceberg vs Parquet. For the catalog choice in more depth, see Iceberg catalogs explained.
Local-First Analytics covers the DuckDB and Parquet side of this: the files, the layout and when a table format starts to pay. Chapter 1 is free to read; the rest is on Amazon.
If you are deciding whether your Postgres needs a lake behind it, a Data Platform Audit is a week and a written roadmap you keep. If an index would do, the roadmap will say so.
Common questions
- Can Postgres write Iceberg tables?
- Yes, with the pg_lake extension. pg_lake (Apache-2.0, from Snowflake-Labs, v3.5.3 in September 2026) lets you run CREATE TABLE ... USING iceberg and then INSERT, UPDATE and DELETE inside normal Postgres transactions, with the data written as Parquet to S3-compatible storage. It does not support MERGE or INSERT ... ON CONFLICT on Iceberg tables, and external engines can read but not yet write the tables it creates. Snowflake Postgres ships pg_lake as a managed option. pg_duckdb, by contrast, documents its Iceberg support as reading, not writing.
- Can Postgres be an Iceberg catalog?
- Yes. An Iceberg catalog only has to store, per table, a pointer to the current metadata.json and swap it atomically on commit, and a Postgres table does that well. PyIceberg's SQL catalog and Iceberg's Java JDBC catalog both use a plain Postgres database this way: one row per table in an iceberg_tables table, updated with a compare-and-swap on the old pointer. pg_lake goes further: Postgres is the catalog for the Iceberg tables it creates, exposed through an iceberg_tables view that Spark, PyIceberg and iceberg-rust can read through their JDBC or SQL catalogs.
- How do I query Iceberg or Parquet files on S3 from Postgres?
- Install an extension that embeds DuckDB. pg_duckdb (MIT, v1.1.1, Postgres 14 to 18) adds read_parquet, read_csv, iceberg_scan and delta_scan, so you can query files on S3, GCS, Azure or R2 from a Postgres session and join them to ordinary tables. pg_lake also reads Parquet, CSV, JSON and Iceberg on object storage, and can COPY results back out. Both hand the scan to DuckDB, so the heavy work runs on a columnar engine rather than in Postgres's row executor.
- pg_lake vs pg_duckdb: which should I use?
- Use pg_duckdb if you want to read lake files and speed up analytical queries from an existing Postgres, with one extension and no extra process. Use pg_lake if Postgres should own Iceberg tables: create them, write to them transactionally, keep them compacted and act as their catalog. pg_lake is larger to run, with a separate pgduck_server process that hosts DuckDB, and more of a platform than an add-on. If you only need to query files now and then, start with pg_duckdb.
- How do I replicate Postgres to Iceberg with CDC?
- The managed routes in 2026: Supabase ETL (Apache-2.0) streams Postgres changes into Supabase Analytics Buckets, which are Iceberg tables on Amazon S3 Tables and still a public alpha. Databricks Lakebase's Lakehouse Sync is in beta and writes Delta tables in Unity Catalog, not Iceberg. Snowflake Postgres takes a different route: pg_lake writes Iceberg tables and a catalog integration exposes them to Snowflake as read-only Iceberg tables. Without a platform, a scheduled job that exports changed rows and appends them to an Iceberg table works, but you then own compaction.
Get new posts by email
Data engineering notes like this one — what breaks and what it costs, in production.
What breaks and what it costs — pipelines, warehouse bills, and the failures that only show up in production. A few a month, never padded to hit a schedule. No sequence, no pitch deck. Reply 'stop' once and you're off — it reaches me, not a queue.
Want the whole playbook?
If this was useful, the long version is my book. Local-First Analytics — 313 pages, runnable code for every chapter — is the full build: DuckDB, Parquet and Arrow, from install to production. On Amazon, and chapter 1 is free to read here.
Get the bookNot ready to buy? Read chapter 1 free — the whole chapter, no email required.
Rather talk it through? Book a free 30-minute call. No slot that suits your time zone? Email info@hikmahtech.in.