rewrite_table_path fails with NotFoundException on expired position delete files referenced by DELETED manifest entries

[vgkowski]

Apache Iceberg version

1.11.0 (latest release)

Query engine

Spark

Please describe the bug 🐞

rewrite_table_path permanently fails with NotFoundException / NoSuchKey on tables where a position delete file (or deletion vector) was dropped by compaction and later physically removed by expire_snapshots. The table itself is perfectly healthy — no reader ever touches the removed file — but it can no longer be replicated, either with a full rewrite or with an incremental rewrite whose delta spans the expiry.

Root cause

RewriteTablePathUtil.writeDeleteFileEntry applies a filter when building the copy plan: only entries that are live and added within the requested delta are copied. Entries that are DELETED (history markers) or outside the delta are still written into the rewritten manifest, but correctly excluded from the copy plan.

However, the physical-rewrite set does not apply the same filter — every position delete entry is queued for rewriting unconditionally:

if (entry.isLive() && snapshotIds.contains(entry.snapshotId())) {
  result.copyPlan().add(...);
}
result.toRewrite().add(file.copy());   // unconditional

Position delete files must be physically rewritten (they embed absolute data file paths), so RewriteTablePathSparkAction opens every file in toRewrite. When the entry is a DELETED marker whose underlying file was removed by expire_snapshots, the open fails and the whole procedure aborts — even though the file was never going to be copied.

Because the DELETED entry is baked into the current snapshot's delete manifest, the failure is not transient: every subsequent rewrite_table_path run fails the same way.

Reproduction (format version 3)

1. Create a table, append data files.
2. Commit a row delta with a deletion vector (newRowDelta().addDeletes(dv)).
3. Run rewrite_data_files — compaction merges the deletes and drops the DV, leaving a DELETED entry in the current snapshot's delete manifest.
4. Run expire_snapshots expiring everything older than the compaction snapshot — the DV file is now physically removed from storage, while the DELETED entry referencing it survives.
5. Run rewrite_table_path (full rewrite) — fails with NotFoundException from PuffinReader while opening the removed DV.

Note the DELETED entry actually passes the snapshotIds.contains(entry.snapshotId()) check (it is stamped with the compaction snapshot's id), so it is only the isLive() half of the copy-plan guard that excludes it from copying — while the rewrite set ignores both.

Incremental variant

1. Full rewrite_table_path to a target (run 1).
2. On the source: compaction drops a position delete file, then expire_snapshots removes it from storage.
3. Incremental rewrite_table_path with a startVersion/endVersion spanning the expiry — the delta manifests still carry the entry (as DELETED, or as EXISTING outside the delta), it is queued for physical rewriting, and the run fails.

This breaks the common setup of periodic incremental replication combined with routine table maintenance on the source.

Non-fatal variant (format version 2)

In v2, compaction leaves the dropped position delete file on storage, so nothing fails — but every dead entry's file is opened, fully rewritten, and written to staging, then never copied (it is not in the copy plan). Wasted I/O and staging storage proportional to the table's history of dropped delete files.

Expected behavior

The physical-rewrite set should be aligned with the copy plan: an entry that is excluded from the copy plan is kept in the rewritten manifest exactly as the source has it and must not be opened. This is already how the data-file and equality-delete branches behave — only the position-delete branch is inconsistent.

The main branch is not affected: positionDeletesToRewrite() collects files to rewrite from live manifest entries. This affects the released line, where the set is built inside writeDeleteFileEntry.

Willingness to contribute

• I can contribute a fix for this bug independently