Create table properties does not support boolean value

[tlegrave]

Apache Iceberg version

main (development)

Please describe the bug 🐞

Hello there,

When creating a table, I can pass properties like bloom filters like so:

catalog.create_table(
 "default.example",
 schema=Schema(
 NestedField(1, "created_at", TimestampType(), required=True)
 ),
 properties={"write.parquet.bloom-filter-enabled.column.material_id": "true"},
)

This gives me the correct configuration for the table:

CREATE TABLE iceberg.default.example (
 created_at timestamp(6) NOT NULL
)
WITH (
 format = 'PARQUET',
 format_version = 2,
 location = 's3://test-bucket/default/example',
 parquet_bloom_filter_columns = ARRAY['material_id']
)

However, if I use a boolean instead of a string in the properties (e.g. {"write.parquet.bloom-filter-enabled.column.material_id": True}, it silently ignores the filter.

CREATE TABLE iceberg.default.example (
 created_at timestamp(6) NOT NULL
)
WITH (
 format = 'PARQUET',
 format_version = 2,
 location = 's3://test-bucket/default/example'
)

[sungwy]

@tlegrave thank you for raising this. From what I can see, this PR introduced the ability to handle cast non-string inputs in the TableProperties dict when setting the Table Properties: #504

I'm having a difficult time reproducing issue because I'm seeing that the property is being set as: metadata.properties.get("write.parquet.bloom-filter-enabled.column.material_id") == "True", which is expected.

On a tangent, I see that this property is still unsupported in PyIceberg for writes - just wanted to make note of that before you dived in deeper into trying to enable this flag for the table

[tlegrave]

Thank you for your reply @syun64.

On a tangent, I see that this property is still unsupported in PyIceberg for writes - just wanted to make note of that before you dived in deeper into trying to enable this flag for the table

Yes, I'm aware of this limitation, and for now we only use pyiceberg to create tables. Write operations are done with spark to handle this bloom filter feature and to bypass some other known limitations of pyiceberg.

Regarding the issue, I made some further tests based on what you mentionned.

This code also ignores the bloom filter (although passing directly "True" as string). Would it be a case sensitive issue?

catalog.create_table(
 "default.example",
 schema=Schema(
 NestedField(1, "created_at", TimestampType(), required=True)
 ),
 properties={"write.parquet.bloom-filter-enabled.column.material_id": "True"},
)

For info, I'm extracting the schema after creation of the table doing a SHOW create table catalog.default.example on Trino.

Thanks

[kevinjqliu]

This code also ignores the bloom filter (although passing directly "True" as string). Would it be a case sensitive issue?

Likely, can you set it to lowercase and see?
In the Trino repo, all the references are lowercased

https://github.com/search?q=repo%3Atrinodb%2Ftrino%20write.parquet.bloom-filter-enabled&type=code

[kevinjqliu]

Ah, if you pass in True (boolean value), this function will turn it into "True"
https://github.com/kevinjqliu/iceberg-python/blob/428b894aacb107547ba41433b4b6f37e1ad1914a/pyiceberg/types.py#L65-L70

which is kind of a footgun

>>> str(True)
'True'

[kevinjqliu]

Created #919 to track the fix

[kevinjqliu]

Resolved in #924