TableMetadataBuilder

[rwwwx]

Draft

[c-thiel]

Nicely written PartitionBinder - thanks @rwwwx!
Just a few minor comments from my side and I believe one check is missing.

[c-thiel]

I only got half-through today. I'll check the rest tomorrow.
It would be great if you could check the pipeline errors.
Once they are solved, I'll add some integration tests for spark & pyiceberg to actually use all of these fancy new features :)

[c-thiel]

@rwwwx a few things to do:

• For current-schema-id, default-spec-id & last-partition-id updates the datatypes also aren't right (search for i64::from). Lets PR fixes over to iceberg-rust as well (i.e. https://github.com/rwwwx/iceberg-rest-server/blob/f869f34ce4730bcad5eef91eeb63b2ac97aae238/crates/iceberg-ext/src/catalog/rest/table.rs#L192)
• I fixed some problems with reuse_or_create_...id functions, but I really don't like the way how I implemented partial equality (without id). https://github.com/rwwwx/iceberg-rest-server/blob/f869f34ce4730bcad5eef91eeb63b2ac97aae238/crates/iceberg-ext/src/spec/table_metadata.rs#L249 . Maybe we can add a .eq_without_id method to the structs in the main crate?
• We desperately need unit tests
• I changed the add_schema / sort_order / spec_id functions together with their reuse or create id functions. Please review the changes and try to improve them. A few bugs in the previous implementation: If an explicit id was specified (lets say 2), it could still happen that the old reuse function assigned a new one. It should only assign a new one if -1 was specified. Also, comparisons where made which included the .id fields, so that the .eq comparison would never hit. We need tests please.
• Calculated field partitioning is not working: https://github.com/rwwwx/iceberg-rest-server/blob/f869f34ce4730bcad5eef91eeb63b2ac97aae238/tests/python/tests/test_spark.py#L175C1-L184C27