Revolutionizing PostgreSQL 19: Inside the Rapid Development of pg_vexec

The debate over whether PostgreSQL can natively support high-performance, columnar, vectorized query execution without fracturing into an unmaintainable fork has finally reached a major turning point. Following last week’s port of Apache Cloudberry to PostgreSQL 19 as a set of modular extensions, database architect Igor Suhorukov has unveiled pg_vexec. This family of extensions brings true vectorized execution, Arrow IPC-backed inter-node communication, native Flight SQL capabilities, and advanced query optimizers—such as GPORCA—directly to an unmodified core of PostgreSQL 19.

Built entirely over a rigorous three-day development sprint in early October 2026 using parallel AI-driven coding agents and strict architectural controls, pg_vexec challenges the traditional paradigm that open-source relational databases must choose between full SQL/ACID semantics and the blistering analytical speeds of modern columnar engines like DuckDB or ClickHouse.

Main Facts: The Architecture of pg_vexec

Standard PostgreSQL architectures execute queries row by row. Each individual row travels upward through the execution node tree independently, passing every single column value through the Function Manager (fmgr). Even in massively parallel processing (MPP) configurations like Greenplum or Apache Cloudberry, cluster segments simply deploy more processors to run the identical, inefficient row-by-row loop.

pg_vexec fundamentally changes this by introducing a vector-oriented execution model.

• Batch Processing: Data is processed in columnar buffers of up to 1,024 rows. This geometry ensures that active fragments fit neatly within L1 and L2 CPU caches during filtering, projection, and aggregation phases.
• Non-Invasive Integration: Crucially, pg_vexec requires zero modifications to the core PostgreSQL 19 codebase. It hooks cleanly into existing PostgreSQL extension points (such as path hooks, CustomScanMethods, and planner hooks). If the extension is unloaded or disabled, vanilla PostgreSQL behavior remains entirely uncompromised.
• Dual-Format Flexibility: The engine supports both a native "postgres format" (matching standard PostgreSQL internal structures, data types, and null bitmaps) and a native "Apache Arrow format" (utilizing zero-copy memory layouts, utf8_view, and standard Arrow epochs).
• GPORCA Integration: Through a lightweight stub layer (-Dorca_single_node), the advanced GPORCA query planner can now be leveraged on vanilla PostgreSQL instances without requiring the bulky gp_core infrastructure, producing highly optimized vector plans.

Chronology: The Three-Day Sprint to Vectorization

The timeline of pg_vexec demonstrates a masterclass in rapid, disciplined systems architecture. Rather than relying on months of manual boilerplate coding, Suhorukov utilized parallel git worktrees managed by Claude Code, acting as the system architect while automated agents handled implementation iterations under strict operational constraints.

• Late August – Early October 2026: Initial architectural explorations focus on embedding Apache DataFusion. However, deep semantic incompatibilities—such as DataFusion’s static handling of now(), collation differences, and expensive memory copy overheads between processes—lead the project to pivot toward a native PostgreSQL extension architecture.
• October 2, 2026: Comprehensive research phase begins, yielding six technical reports and a 4,700-line specification plan detailing boundary conditions, hooks, and data flows.
• October 5, 2026 (Phases VB, V0, V1): Establishes the ClickBench baseline metrics. Groundwork is laid for the query oracle, cost models, dual batch formats, and the source storage contract. VecScan, VecResult, the expression compiler, and PAX/gp_ao table readers are successfully implemented.
• October 6, 2026 (Phases V2 through V6, VK): A flurry of parallel worktrees yields VecAgg (aggregates derived directly from PAX statistics), VecHashJoin, heap page readers, parallel paths, VecSort, VecRepartition, and VecWindowHashAgg. Crucially, GPORCA is successfully decoupled and run on vanilla PostgreSQL 19 (Phase V6), and kernel packs for pgvector and PostGIS are integrated.
• October 7, 2026 (Phases VI, V7): Implementation of batch inserts (VecInsert, VecIngest) and Arrow IPC frames traversing cluster Motions. The high-performance shared-memory transport (shm) for single-host multi-segment configurations is finalized.

Supporting Data: Performance Benchmarks

Performance evaluations run across standard analytical benchmarks on a 16-core Ryzen 9 9955HX3D with 60 GB of memory reveal dramatic efficiency gains across multiple operational vectors.

ClickBench Analytics (10 Million Rows)

When running ClickBench queries, pg_vexec significantly outpaces standard planners by avoiding row-overhead penalties:

• PostgreSQL Planner (Vanilla): 625 ms (heap) / 402 ms (with PK)
• PostgreSQL Planner + pg_vexec (Auto Mode): 506 ms (heap) / 311 ms (with PK) — an 19% to 24% performance improvement.
• GPORCA + pg_vexec (Auto Mode): Dropped execution times from 1,647 ms down to 1,349 ms. Peak individual queries (such as Q15, Q16, and Q29) demonstrated speedups ranging from 8.8x to 9.5x.

Flight SQL vs. Binary COPY (TPC-H Q1 at SF1)

Delivering analytics straight to data science workflows via Apache Arrow over Flight SQL eliminated serialization bottlenecks:

• Standard adbc_driver_postgresql (vexec off): 0.743 seconds
• adbc_driver_flightsql via pg_vexec (vexec auto): 0.389 seconds
• Transferring the first 1,000,000 lineitem rows via standard binary COPY took 0.917 seconds, whereas Flight SQL served the exact same dataset column-native in 0.194 seconds—a 4.7x speedup.

Ingest Performance (200,000 Rows)

Massive bulk-loading operations targeting columnar tables (porc_vec) showcased unprecedented throughput:

• COPY CSV from a server file: 955,491 rows/sec
• Binary COPY through ADBC: 547,494 rows/sec
• Flight SQL adbc_ingest through VecIngest and VecInsert: 1,580,626 rows/sec into porc_vec (1.65x faster than CSV COPY and 2.9x faster than binary COPY).

Official Responses and Technical Design Philosophy

The foundational philosophy guiding pg_vexec is strict modularity combined with zero semantic regression. Throughout development, the strict requirement was maintained: the extension must plug in solely through existing PostgreSQL hooks, and without the extension loaded, the server must behave identically to vanilla PostgreSQL.

To prove this resilience, PostgreSQL’s complete suite of 239 regression tests was executed against instances running pg_vexec in preloaded, disabled, and explain modes. All regression tests passed cleanly against unchanged baseline files. Furthermore, if a vectorized query plan accidentally reaches a cluster segment where the extension libraries have been stripped, the system fails gracefully with a clean, descriptive error (ExtensibleNodeMethods "VecScan" was not registered) while keeping the wider database cluster fully operational.

Implications for the PostgreSQL and AI/ML Ecosystems

The successful deployment of pg_vexec carries profound implications for enterprise database architectures and the burgeoning AI/ML landscape:

1. The Death of the "Fork Dilemma": Historically, achieving columnar vectorization meant adopting massive, unmergeable forks of PostgreSQL (such as openScale or proprietary variants). pg_vexec proves that modular extensions combined with advanced optimizers can deliver identical or superior performance while staying closely aligned with upstream PostgreSQL releases.
2. Native AI and BI Integration: By embedding Apache Arrow natively into memory structures, cluster interconnects (shm), and client-facing endpoints via Flight SQL (pg_vexec_flight), data pipelines no longer suffer the heavy CPU tax of row-to-column serialization. Machine learning frameworks (like PyTorch and Polars) and BI tools can ingest database memory layouts directly with zero-copy efficiency.
3. Seamless Extensibility: Through specialized "kernel packs" (pg_vexec_pgvector and pg_vexec_postgis), high-dimensional vector similarity searches and complex geospatial computations can now execute across vectorized data packs without sacrificing correctness or breaking third-party extension compatibility.

Ultimately, pg_vexec demonstrates that the future of high-performance analytical databases does not require abandoning PostgreSQL in favor of closed silos. Through intelligent extension engineering, modular architecture, and modern compilation techniques, PostgreSQL 19 is fully equipped to compete at the absolute bleeding edge of analytical data processing.

The complete source code and extension modules are publicly available via the pg_vexec GitHub Repository.