[feature request] provide data type conversion between avro and iceberg data types

[kevinjqliu]

Feature Request / Improvement

see #2173 (comment) for more context

The thread address timestamp-millis conversion specifically from avro to iceberg. the complication is that iceberg spec does not support timestamp-millis natively, it supports timestamp-micros. Some conversion logic is needed

AvroSchemaConversion().avro_to_iceberg is currently only used to convert avro data types to those supported by iceberg spec

Also related to #2226

[matthias-Q]

This has to also be considered:

iceberg-python/pyiceberg/utils/schema_conversion.py

Line 168 in 4cac691

# For the Iceberg spec it is required to set the default value to null

In pure AVRO this strict requirement on the order of union types does not exist.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'