Set default metadata prefetch size to 512k

[alamb]

Which issue does this PR close?

• Closes #.

Rationale for this change

Inspired by looking at some profiling results I think the metadata prefetch hint will reduce the number of required IOs by 1/2 for most parquet queries at minimal additional overhead. Let's give this a try.

I will write this idea up in more detail if it turns out to make a difference

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[alamb]

🤖 ./gh_compare_branch.shBenchmark Script Running
Linux aal-dev 6.8.0-1016-gcp #18-Ubuntu SMP Fri Oct 4 22:16:29 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/prefetch (099499f) to 5ab5a03diff
Benchmarks: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

[alamb]

🤖: Benchmark completed

Details

Comparing HEAD and alamb_prefetch
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ alamb_prefetch ┃ Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0 │ 1944.05ms │ 1940.66ms │ no change │
│ QQuery 1 │ 748.05ms │ 750.69ms │ no change │
│ QQuery 2 │ 1471.52ms │ 1445.58ms │ no change │
│ QQuery 3 │ 703.03ms │ 691.35ms │ no change │
│ QQuery 4 │ 1536.24ms │ 1485.23ms │ no change │
│ QQuery 5 │ 16937.70ms │ 16804.20ms │ no change │
│ QQuery 6 │ 2016.33ms │ 2030.05ms │ no change │
└──────────────┴────────────┴────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 25356.92ms │
│ Total Time (alamb_prefetch) │ 25147.75ms │
│ Average Time (HEAD) │ 3622.42ms │
│ Average Time (alamb_prefetch) │ 3592.54ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 0 │
│ Queries with No Change │ 7 │
└───────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ alamb_prefetch ┃ Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.88ms │ 1.89ms │ no change │
│ QQuery 1 │ 36.69ms │ 40.84ms │ 1.11x slower │
│ QQuery 2 │ 92.32ms │ 89.68ms │ no change │
│ QQuery 3 │ 98.54ms │ 97.22ms │ no change │
│ QQuery 4 │ 782.75ms │ 760.18ms │ no change │
│ QQuery 5 │ 833.26ms │ 881.13ms │ 1.06x slower │
│ QQuery 6 │ 1.91ms │ 1.90ms │ no change │
│ QQuery 7 │ 43.39ms │ 43.62ms │ no change │
│ QQuery 8 │ 925.29ms │ 926.41ms │ no change │
│ QQuery 9 │ 1205.14ms │ 1245.17ms │ no change │
│ QQuery 10 │ 262.10ms │ 265.58ms │ no change │
│ QQuery 11 │ 306.40ms │ 302.72ms │ no change │
│ QQuery 12 │ 923.85ms │ 936.69ms │ no change │
│ QQuery 13 │ 1375.17ms │ 1275.04ms │ +1.08x faster │
│ QQuery 14 │ 862.83ms │ 872.23ms │ no change │
│ QQuery 15 │ 1062.95ms │ 1051.33ms │ no change │
│ QQuery 16 │ 1746.84ms │ 1786.07ms │ no change │
│ QQuery 17 │ 1614.93ms │ 1623.43ms │ no change │
│ QQuery 18 │ 3142.17ms │ 3279.39ms │ no change │
│ QQuery 19 │ 98.79ms │ 95.73ms │ no change │
│ QQuery 20 │ 1171.47ms │ 1147.32ms │ no change │
│ QQuery 21 │ 1337.25ms │ 1325.19ms │ no change │
│ QQuery 22 │ 2328.64ms │ 2380.56ms │ no change │
│ QQuery 23 │ 8359.96ms │ 8405.68ms │ no change │
│ QQuery 24 │ 460.85ms │ 471.21ms │ no change │
│ QQuery 25 │ 381.20ms │ 400.42ms │ 1.05x slower │
│ QQuery 26 │ 534.33ms │ 536.65ms │ no change │
│ QQuery 27 │ 1695.23ms │ 1667.68ms │ no change │
│ QQuery 28 │ 12759.54ms │ 12754.58ms │ no change │
│ QQuery 29 │ 526.58ms │ 536.29ms │ no change │
│ QQuery 30 │ 824.73ms │ 829.15ms │ no change │
│ QQuery 31 │ 862.08ms │ 884.26ms │ no change │
│ QQuery 32 │ 2741.33ms │ 2756.61ms │ no change │
│ QQuery 33 │ 3341.94ms │ 3425.23ms │ no change │
│ QQuery 34 │ 3528.85ms │ 3444.66ms │ no change │
│ QQuery 35 │ 1340.19ms │ 1314.81ms │ no change │
│ QQuery 36 │ 133.41ms │ 133.65ms │ no change │
│ QQuery 37 │ 58.96ms │ 59.00ms │ no change │
│ QQuery 38 │ 127.73ms │ 128.35ms │ no change │
│ QQuery 39 │ 200.21ms │ 203.42ms │ no change │
│ QQuery 40 │ 49.07ms │ 51.30ms │ no change │
│ QQuery 41 │ 46.07ms │ 45.58ms │ no change │
│ QQuery 42 │ 40.51ms │ 40.22ms │ no change │
└──────────────┴────────────┴────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 58267.33ms │
│ Total Time (alamb_prefetch) │ 58518.06ms │
│ Average Time (HEAD) │ 1355.05ms │
│ Average Time (alamb_prefetch) │ 1360.89ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 39 │
└───────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ alamb_prefetch ┃ Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 127.84ms │ 124.31ms │ no change │
│ QQuery 2 │ 24.74ms │ 24.25ms │ no change │
│ QQuery 3 │ 35.56ms │ 36.24ms │ no change │
│ QQuery 4 │ 21.25ms │ 20.97ms │ no change │
│ QQuery 5 │ 57.34ms │ 58.09ms │ no change │
│ QQuery 6 │ 8.24ms │ 8.05ms │ no change │
│ QQuery 7 │ 102.61ms │ 104.99ms │ no change │
│ QQuery 8 │ 26.98ms │ 26.29ms │ no change │
│ QQuery 9 │ 63.37ms │ 62.92ms │ no change │
│ QQuery 10 │ 60.83ms │ 58.13ms │ no change │
│ QQuery 11 │ 12.89ms │ 13.04ms │ no change │
│ QQuery 12 │ 37.63ms │ 39.22ms │ no change │
│ QQuery 13 │ 28.82ms │ 30.62ms │ 1.06x slower │
│ QQuery 14 │ 9.92ms │ 9.96ms │ no change │
│ QQuery 15 │ 24.36ms │ 25.93ms │ 1.06x slower │
│ QQuery 16 │ 23.69ms │ 23.80ms │ no change │
│ QQuery 17 │ 96.84ms │ 95.51ms │ no change │
│ QQuery 18 │ 247.24ms │ 246.83ms │ no change │
│ QQuery 19 │ 28.74ms │ 29.20ms │ no change │
│ QQuery 20 │ 39.39ms │ 38.88ms │ no change │
│ QQuery 21 │ 172.44ms │ 175.64ms │ no change │
│ QQuery 22 │ 16.64ms │ 20.19ms │ 1.21x slower │
└──────────────┴──────────┴────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD) │ 1267.37ms │
│ Total Time (alamb_prefetch) │ 1273.06ms │
│ Average Time (HEAD) │ 57.61ms │
│ Average Time (alamb_prefetch) │ 57.87ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 3 │
│ Queries with No Change │ 19 │
└───────────────────────────────┴───────────┘

[alamb]

TLDR is that this didn't seem to have much/any effect on performance so closing