Spark: Remove duplicated TestTables and use core test helper

[ArnabKarmakar1108]

Feature Request / Improvement

Problem

Spark v3.5, v4.0, and v4.1 each have a duplicate TestTables helper (spark/*/spark/src/test/java/org/apache/iceberg/spark/source/TestTables.java:42) with a TODO to use the core version instead.

Core already has a richer org.apache.iceberg.TestTables in core/src/test/java/org/apache/iceberg/TestTables.java. The Spark copies are outdated and risk drifting from core test behavior.

Affected Spark tests: TestBaseReader, TestSparkReadProjection, TestSparkMetadataColumns, TestSparkCatalog.

Proposed behavior

1. Delete the three Spark TestTables copies.
2. Switch affected tests to org.apache.iceberg.TestTables (already available via iceberg-core test artifacts).
3. Update call sites for API differences:
   • create(...) requires a formatVersion argument
   • load(name) → load(File temp, String name)
   • replaceMetadata(...) is not in core — add it there or refactor TestSparkReadProjection to commit via TableOperations

Scope

• Test-only refactor across spark/v3.5, spark/v4.0, spark/v4.1
• ~4 test classes per Spark version
• May need small additions to core TestTables (e.g. replaceMetadata, public readMetadata)
• TestSparkCatalog needs extra care: core keys tables by (location, name), not name alone

Notes

• No production code changes or new dependencies.
• Can be done one Spark version at a time or all three together.

Query engine

Spark

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time