Partition stats task tracker

[ajantha-bhat]

Mailing list discussion archive: https://lists.apache.org/thread/5pzyvncvz6td1vmf02o02ncknbvbzrs7
Design Doc: https://docs.google.com/document/d/1vaufuD47kMijz97LxM67X8OX-W2Wq7nmlz3jRo8J5Qk/edit?usp=sharing

• Define the partition stats schema and requirements then add them to the Iceberg spec. #8451
• Define how to track partition stats files and add it to the Iceberg spec. #8452
• Implement stats file tracking via TableMetadata. #8457
• Core: Remove partition statistics files during purge table #9409
• Ensure that partition stats files are considered for GC procedures #9336
• Core: Add a util to compute partition stats #11146
• Data: Add partition stats writer and reader #11216
• Core: Support incremental compute for partition stats #12629
• Spark-3.5: Add spark action to compute partition stats #12450
• Spark-3.5: Add procedure to compute partition stats #13480
• Core: Support DV for partition stats #13425

Additional improvement from the community

• Store min/max stats per column per partition #11083

Good to have future enhancements

• Implement Synchronous partition stats writing during write operation (controlled by table property). #8458

[ajantha-bhat]

Closing this task as completed today. V4 and future work can still be tracked in this even after closing it.