Amazon S3 Tables Embraces Apache Iceberg V3: A New Era for Petabyte-Scale Cloud Analytics

SEATTLE — In a major development for cloud-native data architectures, Amazon Web Services (AWS) announced full, native support for the Apache Iceberg V3 specification across Amazon S3 Tables. Rolling out globally starting today, this upgrade allows data engineers and enterprise analytics teams to immediately provision brand-new V3 tables or seamlessly upgrade existing V2 structures in place. By bridging the gap between high-performance object storage and advanced open-table features—such as deletion vectors, row lineage, and a robust suite of native data types—AWS aims to eliminate the performance bottlenecks and costly storage workarounds that have historically challenged petabyte-scale data lakes.

The announcement positions AWS at the forefront of the open-source analytics movement, delivering tightly integrated support for Iceberg V3 through a unified stack that spans Amazon S3, Amazon EMR, AWS Glue, and Amazon Redshift.

Main Facts: What Amazon S3 Tables V3 Support Delivers

Apache Iceberg has rapidly evolved into the de facto open standard for managing massive analytical datasets in cloud object storage. While the V2 specification unlocked essential capabilities like schema evolution, hidden partitioning, and time-travel queries using open Parquet files, it also exposed notable operational frictions as data lakes scaled into the tens of petabytes.

The integration of Iceberg V3 into Amazon S3 Tables directly addresses these pain points by introducing core architectural advancements:

• Deletion Vectors: Replacing traditional, resource-heavy positional delete files with a compact binary format, deletion vectors drastically streamline write-heavy and compliance-driven operations (such as GDPR or CCPA record deletions) without degrading downstream query performance.
• Row Lineage: Automatically assigning internal identifiers (_row_id and _last_updated_sequence_number) to every record, allowing downstream data pipelines to execute highly efficient incremental queries rather than full-table scans.
• Advanced Data Types: Native support for modern analytical needs via variant (for columnar semi-structured data), nanosecond-precision timestamps, geometry and geography (for geospatial analytics), and an unknown type.
• Fully Managed Infrastructure: Amazon S3 Tables maintains its core value proposition by handling automatic table compaction, background maintenance, replication, and Intelligent-Tiering for V3 tables at no additional cost beyond standard S3 pricing.

Chronology: The Evolution to Iceberg V3

The journey toward native V3 support reflects the rapid maturation of open table formats and enterprise demands for real-time, flexible data ingestion:

• The V2 Era and Its Limits: Over the past several years, organizations embraced Apache Iceberg V2 for its reliability and independence from proprietary vendor lock-in. However, teams processing semi-structured JSON payloads or executing frequent row-level updates quickly encountered performance friction. Compliance mandates requiring the deletion of thousands of user records from multi-billion-row tables generated thousands of positional delete files, forcing database engines to run expensive, frequent compaction jobs to maintain query speeds.
• The Rise of the V3 Specification: The Apache Software Foundation introduced the Iceberg V3 specification to solve these structural challenges, moving beyond basic table management to optimize row-level mutations, native type safety, and governance tracking.
• AWS Integration and Rollout: Recognizing the enterprise urgency for these features, AWS engineering teams developed comprehensive, backwards-compatible support across the S3 ecosystem. Today’s general availability marks the culmination of this effort, enabling production-grade V3 deployments across all AWS regions that support S3 Tables.

Supporting Data and Technical Architecture

To understand the tangible impact of the V3 upgrade, it is helpful to examine how data teams previously handled semi-structured data and row modifications versus how they operate under the new specification.

Overcoming the JSON Parsing Bottleneck

In legacy V2 workflows, semi-structured events originating from web and mobile applications were frequently ingested as raw JSON strings. Every downstream query required compute-heavy runtime parsing (e.g., via PARSE_JSON), inflating CPU usage and I/O costs.

With the introduction of the Iceberg V3 variant data type in Amazon S3 Tables, data is shredded into hidden columns and statistically profiled during the write phase. When an analyst queries the table using services like Amazon EMR Spark, the engine leverages these built-in statistics to prune irrelevant files instantly:

SELECT
  event_id,
  user_id,
  variant_get(payload, '$.action', 'string') AS action,
  variant_get(payload, '$.amount', 'double') AS amount
FROM my_catalog.namespace.clickstream
WHERE variant_get(payload, '$.action', 'string') = 'purchase'
  AND variant_get(payload, '$.amount', 'double') > 50.00

Streamlining Deletions with Merge-on-Read

Compliance requests often demand the immediate removal of specific user partitions or identifiers. In V2 tables, performing a delete on a massive dataset generated fragmented positional delete files. By configuring Amazon S3 Tables to use merge-on-read mode with deletion vectors:

ALTER TABLE my_catalog.namespace.clickstream
SET TBLPROPERTIES (
  'write.delete.mode' = 'merge-on-read',
  'write.update.mode' = 'merge-on-read',
  'write.merge.mode' = 'merge-on-read'
);

DELETE FROM my_catalog.namespace.clickstream
WHERE user_id = 'user-42';

Instead of rewriting multi-gigabyte data files or littering the storage bucket with tiny delete files, the engine writes a single, lightweight deletion vector. Background maintenance managed natively by S3 Tables absorbs and optimizes these structures during routine cycles.

Building Efficient Incremental Pipelines via Row Lineage

Traditional change data capture (CDC) or incremental processing required complex watermarking or full table scans to identify modified rows. Iceberg V3 automates row tracking, allowing data engineers to build lightning-fast incremental pipelines:

SELECT *, _row_id, _last_updated_sequence_number
FROM my_catalog.namespace.clickstream
WHERE _last_updated_sequence_number > 42;

By persisting a simple checkpoint of the last processed sequence number, downstream jobs consume only the newly mutated data, drastically reducing pipeline execution times and cloud compute expenditures.

Official Perspectives and Ecosystem Implications

According to Daniel Abib, the AWS engineering team behind the launch, the integration is designed to minimize migration friction while unlocking immediate performance gains. AWS provides comprehensive backwards compatibility, allowing existing V2 readers to continue querying upgraded tables until engineering organizations are ready to fully transition their downstream tooling to V3 standards.

Seamless Migration Path

Upgrading an existing table requires a simple, atomic metadata operation without necessitating a physical rewrite of the underlying data files:

ALTER TABLE my_catalog.namespace.existing_table
SET TBLPROPERTIES ('format-version' = '3');

Note: Because the Apache Iceberg specification treats V3 upgrades as a one-way operation (downgrading from V3 to V2 is not supported), AWS recommends verifying that all third-party and internal engines accessing the catalog fully support the V3 specification prior to execution.

Interoperability Across the Modern Data Stack

AWS continues to champion open architectures by ensuring that S3 Tables and the AWS Glue Data Catalog fully support the Iceberg REST Catalog (IRC) API. This guarantees broad interoperability, allowing enterprises to ingest data via Amazon EMR, govern metadata with AWS Glue 6.0, and execute high-performance business intelligence queries via Amazon Redshift, all while leveraging open-format Parquet storage on Amazon S3.

Broader Industry Implications

The arrival of native Apache Iceberg V3 support on Amazon S3 Tables signals a maturation point for modern data architecture. As organizations grapple with escalating data volumes, the traditional trade-off between flexible, schema-less ingestion and high-performance querying is rapidly disappearing.

By pushing advanced optimizations—such as columnar variant shredding and binary deletion vectors—directly into the storage layer, AWS is empowering data teams to reduce infrastructure overhead, lower cloud storage expenditure, and simplify data governance compliance. For enterprises heavily invested in open data lakes, the upgrade offers a compelling blueprint for scaling analytics into the next decade without sacrificing architectural freedom or performance.