pg-cdc is an open-source PostgreSQL Change Data Capture engine that streams WAL changes directly to Apache Iceberg and Parquet on Amazon S3. Unlike traditional CDC pipelines, it runs as a single binary—no Kafka, Spark, or JVM required. The output is immediately queryable with Athena, DuckDB, Spark, Trino, and other Iceberg-compatible engines, making operational PostgreSQL data ready for analytics and AI workloads.

Hi Product Hunt! 👋 I’m excited to introduce pg-cdc, an open-source PostgreSQL Change Data Capture engine built to make operational data easy to use for analytics and AI. Instead of assembling a pipeline with Kafka, Spark, and multiple connectors, pg-cdc streams PostgreSQL WAL changes directly to Apache Iceberg and Parquet on Amazon S3 from a single binary. The output is immediately queryable with Athena, DuckDB, Spark, Trino, and other Iceberg-compatible engines. I’d love your feedback on: * The developer experience * Installation and setup * Performance and scalability * Features you’d like to see next If you’re building data platforms or AI applications on PostgreSQL, I’d really appreciate your thoughts. Thanks for checking out the project!

We ve found that the hardest part of building AI applications isn t the model it s getting governed, trustworthy operational data into AI systems. How are you handling change data capture today? Are you streaming directly from PostgreSQL, building custom pipelines, or relying on periodic ETL?

pg-cdc

PostgreSQL CDC to Iceberg & S3 in one binary

PostgreSQL CDC to Iceberg & S3 in one binary

Forum Threads

How are you handling PostgreSQL CDC for AI applications?

Forum Threads

How are you handling PostgreSQL CDC for AI applications?