Column name mapping support

[Tishj]

This PR implements #147

[Tmonster]

Awesome thanks!
Couple of questions,

Is there no way to get the tests working with our current test infra?

Can we also check renaming of columns to different (previously valid names?) and that the contents of the tables is consistent? Something like the following tests should be good

check column drop also drops values

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 drop column b;
alter table t1 add column b;
# in test
select count(*) from t1 where b is null

check column rename to other (previously valid) column name also preserves correct values.

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 rename column a to c;
# in test
select c from t1;
# verify selecting a is an error
select a from t1;

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 drop column b;
alter table t1 rename column a to b;
# in test
# verify a is the range column
# verify selecting a results in error
select a from t1;

[Tmonster]

nit: could you put these files in data/persistent/column_mapping?

[kevinjqliu]

heres where you can find some tests related to name-mapping
https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/mapping/TestMappingUpdates.java
https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/TestSchemaAndMappingUpdate.java

[Tishj]

Awesome thanks! Couple of questions,

Is there no way to get the tests working with our current test infra?

Can we also check renaming of columns to different (previously valid names?) and that the contents of the tables is consistent? Something like the following tests should be good

check column drop also drops values

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 drop column b;
alter table t1 add column b;
# in test
select count(*) from t1 where b is null

check column rename to other (previously valid) column name also preserves correct values.

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 rename column a to c;
# in test
select c from t1;
# verify selecting a is an error
select a from t1;

create table t1 as select range a, random()*1000 b from range(10000);
alter table t1 drop column b;
alter table t1 rename column a to b;
# in test
# verify a is the range column
# verify selecting a results in error
select a from t1;

Thanks for the suggestion, I've crafted a test for one of these, but I don't know how relevant these scenarios are for this feature.

All this PR does is fill in field-ids for parquet files that don't have them.
The drop+re-add scenario creates a new schema, where the new b field will have a new field-id (3).
The name-mapping will assign field-id 2 to the parquet file's b column, so it doesn't serve as data for the new b field, because the field-id doesn't match.

Same goes for the rename from a -> c, this happens at the global level (metadata.json), the field-id doesn't change.

And lastly the drop b, rename a -> b is the same as above, only the name in the global schema changes, the name-mapping is applied at the local level, and field-ids are unchanged.