Apache Iceberg version
main @ 9d7b2c5
Query engine
None — the bug is in iceberg-mr and is engine-agnostic.
Please describe the bug
IcebergSplit.getLocations() (mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.java line 70-82) returns the computed block locations on the first call and ["*"] on every call after that, even though nothing about the split changed.
The transient locations field and the locations == null check say the value is computed once and cached. The else branch defeats that: it runs whenever locations is already set and overwrites it with ANYWHERE. Hadoop's InputSplit.getLocations() contract expects repeated calls on an unchanged split to return the same value.
The else branch came from #1582, which kept a deserialized split on a worker node (where conf is null) from hitting an NPE. Before that, #1192 had if (locations == null) alone.
Steps to reproduce
Build splits with iceberg.mr.locality enabled (InputFormatConfig.ConfigBuilder.preferLocality()) and call getLocations() twice on the same split.
- Expected: both calls return the computed hosts, e.g.
["localhost"].
- Actual: the first returns
["localhost"], the second returns ["*"].
TestIcebergInputFormats#testLocality reproduces this without Docker.
Additional context
N/A
Apache Iceberg version
main @ 9d7b2c5
Query engine
None — the bug is in
iceberg-mrand is engine-agnostic.Please describe the bug
IcebergSplit.getLocations()(mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.javaline 70-82) returns the computed block locations on the first call and["*"]on every call after that, even though nothing about the split changed.The
transient locationsfield and thelocations == nullcheck say the value is computed once and cached. Theelsebranch defeats that: it runs wheneverlocationsis already set and overwrites it withANYWHERE. Hadoop'sInputSplit.getLocations()contract expects repeated calls on an unchanged split to return the same value.The
elsebranch came from #1582, which kept a deserialized split on a worker node (whereconfis null) from hitting an NPE. Before that, #1192 hadif (locations == null)alone.Steps to reproduce
Build splits with
iceberg.mr.localityenabled (InputFormatConfig.ConfigBuilder.preferLocality()) and callgetLocations()twice on the same split.["localhost"].["localhost"], the second returns["*"].TestIcebergInputFormats#testLocalityreproduces this without Docker.Additional context
N/A