GH-3451. Add a JMH benchmark for variants

[steveloughran]

Rationale for this change

There's no benchmark for variant IO and so there's no knowledge of any problems which exist now, or any way to detect regressions.

What changes are included in this PR?

• adds parquet-variant to parquet-benchmark dependencies
• new JMH benchmark VariantBuilderBenchmark to measure builder cost.
• VariantProjectionBenchmark: selective read of fields from shredded/unshredded files
• Improves VariantConverter of a binary to a string through a new package-private method in VariantBuilder. Benchmark VariantConverterBenchmark shows a 20% reduction in time to convert a marshalled string to a java string within a builder. This codepath is used when reconstructing partially shredded variants.

Are these changes tested?

Apart from a minor change in VariantConverter/VariantBuilder, this is all benchmark

See https://steveloughran.github.io/benchmarking-variants/ for the writeup and the interactive benchmark results of Iceberg and Parquet benchmarks.

Are there any user-facing changes?

No

Closes#3451

[steveloughran]

Still thinking of what else can be done here...suggestions welcome.

Probably a real write to the localfs and read back in

[steveloughran]

I'll add a "deep" option too, for consistency with the iceberg pr

[xiaoxuandev]

only resets the local copy, counter keeps growing？

[steveloughran]

good point. will fix.

[steveloughran]

There's now a new benchmark which writes a file using the same simple schema as I'm doing in iceberg apache/iceberg#15629 , and tries to do a projection on it.

 SELECT id, category, variant_get('nested.varcategory') FROM table

Review by the copilot

---

Setup: 1M rows, 4-field nested variant (idstr, varid, varcategory, col4), querying varcategory only. SingleShotTime, 15 iterations, @fork(0).

Raw Results

┌───────────────────────────┬──────────┬───────────────┬─────────┬────────┐
│ Benchmark │ shredded │ Score (ms/op) │ Error │ µs/row │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readAllRecords │ false │ 728.514 │ ±11.253 │ 0.729 │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readProjectedFileSchema │ false │ 760.287 │ ±3.314 │ 0.760 │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readProjectedLeanSchema │ false │ 1405.264 │ ±8.399 │ 1.405 │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readAllRecords │ true │ 1315.615 │ ±14.598 │ 1.316 │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readProjectedFileSchema │ true │ 1297.870 │ ±19.621 │ 1.298 │
├───────────────────────────┼──────────┼───────────────┼─────────┼────────┤
│ readProjectedLeanSchema │ true │ 725.618 │ ±10.574 │ 0.726 │
└───────────────────────────┴──────────┴───────────────┴─────────┴────────┘

Speedup/Penalty vs readAllRecords Baseline

┌───────────────────────────┬──────────────────┬──────────────────┐
│ Benchmark │ shredded=false │ shredded=true │
├───────────────────────────┼──────────────────┼──────────────────┤
│ readProjectedFileSchema │ −4% (overhead) │ +1% (noise) │
├───────────────────────────┼──────────────────┼──────────────────┤
│ readProjectedLeanSchema │ −93% penalty │ +45% speedup │
└───────────────────────────┴──────────────────┴──────────────────┘

• Lean schema projection is the only technique that skips columns. Projecting the full file schema (readProjectedFileSchema) gives zero benefit in either case — Parquet still reads all column chunks.
• Lean schema + shredded = 45% faster than reading all columns. Skipping idstr, varid, and col4 typed columns saves ~590ms per 1M rows.
• Lean schema + unshredded = 93% slower. The lean schema requests typed_value.varcategory which does not exist in the unshredded file. Parquet handles the missing columns at every row, which is more expensive than
  reading the single binary blob directly.
• Schema detection in ReadSupport.init() is essential. Applying containsField("typed_value") to choose between lean and full schema prevents the unshredded penalty while preserving the shredded speedup.

Recommendation

Always detect file layout in ReadSupport.init() and apply the lean projection only when the file was written with a shredded schema. For unshredded files, use the full file schema or no projection.

If you have a query with a pushdown predicate that wants to look inside a variant, creating a MessageType schema referring to the shredded values is counterproductive unless you know that the variant is shedded.

That can be determined by looking at the schema and use `.containsField("typed_value") to see if the target variant has any nested values.

@OverridepublicReadContextinit(InitContextcontext) {
MessageTypefileSchema = context.getFileSchema();
GroupTypenested = fileSchema.getType("nested").asGroupType();
if (nested.containsField("typed_value")) {
returnnewReadContext(VARCATEGORY_PROJECTION);
}
// Unshredded file: projection designed for typed columns provides no benefit and// causes schema mismatch overhead — fall back to the full file schema.returnnewReadContext(fileSchema);
}

[steveloughran]

build failures are all because java11 javadoc is extra-fussy than the versions either side of it.

[steveloughran]

Flame graph profiles so you can compare what is using CPU time when working with variants

profiles.zip

Variant rebuild is expensive, I've got a minor (and will keep package private) tweak to VariantBuilder to allow VariantStringConverter to add a Binary as a string to the variant without converting it to a string en-route, which would save a lot of needless string creation.

Variant.getFieldAtIndex() is pretty expensive too.

[steveloughran]

Slides. Reading the PR will make the notion of file vs lean schema clearer.

2026-04-01-variant reads considered suboptimal

[Fokko]

This looks great to me, thanks @steveloughran 🙌

[steveloughran]

thanks @Fokko! Can you do the merge?

[nssalian]

LGTM.

[steveloughran]

just doing some feature creep here

• make col4 a long string (and name it as such)
• add a uuid

[steveloughran]

latest numbers

before adding uuid read (but with that long column)

Benchmark (tableType) Mode Cnt Score Error Units
VariantProjectionBenchmark.readAllRecords Unshredded ss 10 1645.855 ± 27.618 ms/op
VariantProjectionBenchmark.readAllRecords Shredded ss 10 2381.192 ± 41.940 ms/op
VariantProjectionBenchmark.readProjectedFileSchema Unshredded ss 10 932.050 ± 37.143 ms/op
VariantProjectionBenchmark.readProjectedFileSchema Shredded ss 10 1596.800 ± 50.421 ms/op
VariantProjectionBenchmark.readProjectedLeanSchema Unshredded ss 10 1750.998 ± 10.982 ms/op
VariantProjectionBenchmark.readProjectedLeanSchema Shredded ss 10 724.603 ± 18.377 ms/op

after adding UUID variant field.

Benchmark (tableType) Mode Cnt Score Error Units
VariantProjectionBenchmark.readAllRecords Unshredded ss 10 1913.765 ± 18.896 ms/op
VariantProjectionBenchmark.readAllRecords Shredded ss 10 2679.631 ± 150.978 ms/op
VariantProjectionBenchmark.readProjectedFileSchema Unshredded ss 10 910.009 ± 33.074 ms/op
VariantProjectionBenchmark.readProjectedFileSchema Shredded ss 10 1616.585 ± 57.818 ms/op
VariantProjectionBenchmark.readProjectedLeanSchema Unshredded ss 10 1777.288 ± 14.049 ms/op
VariantProjectionBenchmark.readProjectedLeanSchema Shredded ss 10 723.679 ± 8.473 ms/op

points to note

• full record read is now ~ 15% slower on both unshredded and shredded files just by adding a UUID. Surprisingly Expensive.
• Reading all records on a shredded file is still ~40% slower than on an unshredded one.
• same odd behaviours on a projected schema

[steveloughran]

No further changes planned...now it's time for people to look at those numbers and think of improvements. Looking forward!

[nssalian]

@steveloughran Let's keep the benchmark code here and move the changes to #3481 (I've already pulled some of the changes there). It's great to verify here, but since the original intent was to add JMH benchmarks, let's scope this to that and get the improvements in another focused PR.

[steveloughran]

@nssalian will do.

[Fokko]

Thanks @steveloughran for working on this, and thanks @nssalian and @xiaoxuandev for the review 🚀

[steveloughran]

@Fokko thanks for merging this!