Optimize planning / stop cloning Strings / Fields so much (2-3% faster planning time) by alamb · Pull Request #18415 · apache/datafusion

alamb · 2025-10-31T19:05:49Z

Which issue does this PR close?

ClosesOptimize planning speed / stop cloning Strings / Fields so much #18413

Rationale for this change

Avoid a bunch of clones / String copies during planning

What changes are included in this PR?

Change several methods on DFSchema to return &FieldRef rather than &Field which permits Arc::clone rather than a deep Field clone

Are these changes tested?

yes by CI

I also ran benchmarks that show a small but consistent speedup in the planning benchmarks

Are there any user-facing changes?

Yes, there are several API changes in DFSchema that now return FieldRef rather than Field which allows using Arc::clone rather than clone. I have updated the upgrading guide too

alamb · 2025-10-31T19:37:42Z

 metadata,
 ..
 }) => {
- let field = expr.to_field(schema).map(|(_, f)| f.as_ref().clone())?;


these as_ref().clone() type methods are doing a deep clone on the Fields each time even when the name is not changing

alamb · 2025-10-31T19:42:23Z

🤖 ./gh_compare_branch_bench.shBenchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/stop_cloning_fields (074e485) to e65dafediff
BENCH_NAME=sql_planner
BENCH_COMMAND=cargo bench --bench sql_planner
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_stop_cloning_fields
Results will be posted here when complete

alamb · 2025-10-31T19:38:54Z

 /// Returns an immutable reference of a specific `Field` instance selected using an
 /// offset within the internal `fields` vector
- pub fn field(&self, i: usize) -> &Field {
+ pub fn field(&self, i: usize) -> &FieldRef {


the changes in this file are the public API changes -- they return a reference to the Arc rather than what is in the Arc meaning the callsite can clone the arcs when needed

alamb · 2025-10-31T20:47:47Z

🤖: Benchmark completed

Details

group alamb_stop_cloning_fields main
----- ------------------------- ----
logical_aggregate_with_join 1.00 623.2±2.35µs ? ?/sec 1.01 631.0±5.28µs ? ?/sec
logical_select_all_from_1000 1.00 10.6±0.10ms ? ?/sec 1.04 11.0±0.04ms ? ?/sec
logical_select_one_from_700 1.00 415.6±4.41µs ? ?/sec 1.00 415.7±5.60µs ? ?/sec
logical_trivial_join_high_numbered_columns 1.01 373.5±5.33µs ? ?/sec 1.00 369.4±2.36µs ? ?/sec
logical_trivial_join_low_numbered_columns 1.01 359.0±3.75µs ? ?/sec 1.00 355.1±1.50µs ? ?/sec
physical_intersection 1.00 833.7±7.62µs ? ?/sec 1.00 834.4±3.02µs ? ?/sec
physical_join_consider_sort 1.00 1399.4±5.55µs ? ?/sec 1.00 1394.1±9.37µs ? ?/sec
physical_join_distinct 1.00 348.1±1.43µs ? ?/sec 1.00 348.6±4.03µs ? ?/sec
physical_many_self_joins 1.00 9.5±0.05ms ? ?/sec 1.00 9.6±0.06ms ? ?/sec
physical_plan_clickbench_all 1.01 196.8±4.05ms ? ?/sec 1.00 195.6±5.82ms ? ?/sec
physical_plan_clickbench_q1 1.00 2.4±0.03ms ? ?/sec 1.06 2.5±0.08ms ? ?/sec
physical_plan_clickbench_q10 1.00 3.3±0.06ms ? ?/sec 1.07 3.5±0.11ms ? ?/sec
physical_plan_clickbench_q11 1.00 3.5±0.05ms ? ?/sec 1.10 3.9±0.14ms ? ?/sec
physical_plan_clickbench_q12 1.00 3.6±0.08ms ? ?/sec 1.11 4.0±0.14ms ? ?/sec
physical_plan_clickbench_q13 1.00 3.2±0.04ms ? ?/sec 1.09 3.6±0.11ms ? ?/sec
physical_plan_clickbench_q14 1.00 3.5±0.04ms ? ?/sec 1.08 3.8±0.12ms ? ?/sec
physical_plan_clickbench_q15 1.00 3.4±0.05ms ? ?/sec 1.06 3.6±0.10ms ? ?/sec
physical_plan_clickbench_q16 1.00 3.2±0.04ms ? ?/sec 1.05 3.3±0.07ms ? ?/sec
physical_plan_clickbench_q17 1.00 3.3±0.06ms ? ?/sec 1.05 3.4±0.09ms ? ?/sec
physical_plan_clickbench_q18 1.00 2.8±0.03ms ? ?/sec 1.06 3.0±0.10ms ? ?/sec
physical_plan_clickbench_q19 1.00 3.7±0.06ms ? ?/sec 1.09 4.0±0.11ms ? ?/sec
physical_plan_clickbench_q2 1.00 2.9±0.04ms ? ?/sec 1.06 3.0±0.12ms ? ?/sec
physical_plan_clickbench_q20 1.00 2.5±0.03ms ? ?/sec 1.09 2.8±0.09ms ? ?/sec
physical_plan_clickbench_q21 1.00 2.9±0.03ms ? ?/sec 1.08 3.1±0.12ms ? ?/sec
physical_plan_clickbench_q22 1.00 3.8±0.11ms ? ?/sec 1.01 3.8±0.14ms ? ?/sec
physical_plan_clickbench_q23 1.00 4.0±0.13ms ? ?/sec 1.02 4.1±0.14ms ? ?/sec
physical_plan_clickbench_q24 1.00 4.5±0.12ms ? ?/sec 1.02 4.6±0.15ms ? ?/sec
physical_plan_clickbench_q25 1.00 3.3±0.10ms ? ?/sec 1.01 3.3±0.12ms ? ?/sec
physical_plan_clickbench_q26 1.02 3.1±0.10ms ? ?/sec 1.00 3.0±0.11ms ? ?/sec
physical_plan_clickbench_q27 1.00 3.3±0.15ms ? ?/sec 1.02 3.3±0.13ms ? ?/sec
physical_plan_clickbench_q28 1.00 4.1±0.15ms ? ?/sec 1.01 4.2±0.19ms ? ?/sec
physical_plan_clickbench_q29 1.00 4.5±0.14ms ? ?/sec 1.01 4.5±0.18ms ? ?/sec
physical_plan_clickbench_q3 1.00 2.8±0.03ms ? ?/sec 1.07 3.0±0.10ms ? ?/sec
physical_plan_clickbench_q30 1.00 13.1±0.21ms ? ?/sec 1.02 13.4±0.16ms ? ?/sec
physical_plan_clickbench_q31 1.00 4.0±0.12ms ? ?/sec 1.04 4.2±0.15ms ? ?/sec
physical_plan_clickbench_q32 1.00 4.0±0.11ms ? ?/sec 1.01 4.0±0.15ms ? ?/sec
physical_plan_clickbench_q33 1.00 3.5±0.12ms ? ?/sec 1.01 3.5±0.16ms ? ?/sec
physical_plan_clickbench_q34 1.00 3.2±0.12ms ? ?/sec 1.00 3.2±0.13ms ? ?/sec
physical_plan_clickbench_q35 1.00 3.3±0.09ms ? ?/sec 1.00 3.3±0.12ms ? ?/sec
physical_plan_clickbench_q36 1.00 3.9±0.11ms ? ?/sec 1.03 4.0±0.16ms ? ?/sec
physical_plan_clickbench_q37 1.00 4.2±0.13ms ? ?/sec 1.02 4.3±0.18ms ? ?/sec
physical_plan_clickbench_q38 1.00 4.2±0.14ms ? ?/sec 1.02 4.2±0.21ms ? ?/sec
physical_plan_clickbench_q39 1.00 4.1±0.13ms ? ?/sec 1.02 4.1±0.16ms ? ?/sec
physical_plan_clickbench_q4 1.00 2.5±0.04ms ? ?/sec 1.05 2.6±0.07ms ? ?/sec
physical_plan_clickbench_q40 1.00 4.8±0.14ms ? ?/sec 1.02 4.9±0.19ms ? ?/sec
physical_plan_clickbench_q41 1.00 4.2±0.11ms ? ?/sec 1.03 4.3±0.18ms ? ?/sec
physical_plan_clickbench_q42 1.00 4.1±0.12ms ? ?/sec 1.03 4.2±0.15ms ? ?/sec
physical_plan_clickbench_q43 1.00 4.5±0.10ms ? ?/sec 1.03 4.7±0.19ms ? ?/sec
physical_plan_clickbench_q44 1.00 2.7±0.06ms ? ?/sec 1.02 2.8±0.11ms ? ?/sec
physical_plan_clickbench_q45 1.00 2.7±0.05ms ? ?/sec 1.03 2.8±0.10ms ? ?/sec
physical_plan_clickbench_q46 1.00 3.2±0.08ms ? ?/sec 1.02 3.2±0.09ms ? ?/sec
physical_plan_clickbench_q47 1.00 3.8±0.10ms ? ?/sec 1.01 3.8±0.17ms ? ?/sec
physical_plan_clickbench_q48 1.00 4.6±0.12ms ? ?/sec 1.04 4.8±0.20ms ? ?/sec
physical_plan_clickbench_q49 1.00 4.9±0.13ms ? ?/sec 1.01 5.0±0.23ms ? ?/sec
physical_plan_clickbench_q5 1.00 2.7±0.04ms ? ?/sec 1.06 2.9±0.09ms ? ?/sec
physical_plan_clickbench_q50 1.00 4.6±0.14ms ? ?/sec 1.00 4.6±0.21ms ? ?/sec
physical_plan_clickbench_q51 1.00 3.3±0.09ms ? ?/sec 1.00 3.3±0.15ms ? ?/sec
physical_plan_clickbench_q6 1.00 2.7±0.04ms ? ?/sec 1.06 2.9±0.10ms ? ?/sec
physical_plan_clickbench_q7 1.00 2.4±0.03ms ? ?/sec 1.07 2.6±0.10ms ? ?/sec
physical_plan_clickbench_q8 1.00 3.3±0.06ms ? ?/sec 1.09 3.6±0.14ms ? ?/sec
physical_plan_clickbench_q9 1.00 3.2±0.05ms ? ?/sec 1.08 3.4±0.12ms ? ?/sec
physical_plan_tpcds_all 1.00 1038.4±5.10ms ? ?/sec 1.02 1055.2±15.34ms ? ?/sec
physical_plan_tpch_all 1.00 64.6±0.53ms ? ?/sec 1.01 64.9±0.55ms ? ?/sec
physical_plan_tpch_q1 1.00 1993.5±11.78µs ? ?/sec 1.03 2.1±0.02ms ? ?/sec
physical_plan_tpch_q10 1.00 3.9±0.03ms ? ?/sec 1.02 4.0±0.03ms ? ?/sec
physical_plan_tpch_q11 1.00 3.6±0.02ms ? ?/sec 1.01 3.6±0.02ms ? ?/sec
physical_plan_tpch_q12 1.00 1815.6±11.28µs ? ?/sec 1.00 1822.6±5.01µs ? ?/sec
physical_plan_tpch_q13 1.00 1451.8±20.14µs ? ?/sec 1.02 1476.8±9.65µs ? ?/sec
physical_plan_tpch_q14 1.00 1946.3±10.46µs ? ?/sec 1.02 1980.6±20.35µs ? ?/sec
physical_plan_tpch_q16 1.00 2.5±0.01ms ? ?/sec 1.01 2.5±0.02ms ? ?/sec
physical_plan_tpch_q17 1.00 2.6±0.01ms ? ?/sec 1.02 2.6±0.01ms ? ?/sec
physical_plan_tpch_q18 1.00 2.6±0.01ms ? ?/sec 1.03 2.7±0.07ms ? ?/sec
physical_plan_tpch_q19 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.02ms ? ?/sec
physical_plan_tpch_q2 1.00 6.1±0.09ms ? ?/sec 1.00 6.1±0.06ms ? ?/sec
physical_plan_tpch_q20 1.00 3.2±0.02ms ? ?/sec 1.01 3.2±0.02ms ? ?/sec
physical_plan_tpch_q21 1.00 4.2±0.03ms ? ?/sec 1.01 4.3±0.03ms ? ?/sec
physical_plan_tpch_q22 1.00 2.8±0.02ms ? ?/sec 1.01 2.8±0.01ms ? ?/sec
physical_plan_tpch_q3 1.01 2.8±0.05ms ? ?/sec 1.00 2.7±0.02ms ? ?/sec
physical_plan_tpch_q4 1.00 1505.7±13.90µs ? ?/sec 1.01 1517.6±9.68µs ? ?/sec
physical_plan_tpch_q5 1.02 3.3±0.04ms ? ?/sec 1.00 3.3±0.02ms ? ?/sec
physical_plan_tpch_q6 1.00 859.8±4.47µs ? ?/sec 1.01 866.9±4.01µs ? ?/sec
physical_plan_tpch_q7 1.00 4.2±0.02ms ? ?/sec 1.01 4.3±0.03ms ? ?/sec
physical_plan_tpch_q8 1.00 5.5±0.05ms ? ?/sec 1.01 5.6±0.10ms ? ?/sec
physical_plan_tpch_q9 1.00 4.0±0.02ms ? ?/sec 1.01 4.0±0.02ms ? ?/sec
physical_select_aggregates_from_200 1.00 16.9±0.10ms ? ?/sec 1.01 17.0±0.10ms ? ?/sec
physical_select_all_from_1000 1.00 23.6±0.49ms ? ?/sec 1.02 24.2±0.09ms ? ?/sec
physical_select_one_from_700 1.00 1074.4±4.18µs ? ?/sec 1.00 1076.7±5.93µs ? ?/sec
physical_sorted_union_order_by_10 1.00 12.7±0.14ms ? ?/sec 1.04 13.2±0.16ms ? ?/sec
physical_sorted_union_order_by_50 1.00 374.3±2.45ms ? ?/sec 1.02 382.5±4.03ms ? ?/sec
physical_theta_join_consider_sort 1.00 1765.6±8.84µs ? ?/sec 1.00 1760.4±14.43µs ? ?/sec
physical_unnest_to_join 1.00 1817.6±9.83µs ? ?/sec 1.02 1858.4±16.44µs ? ?/sec
physical_window_function_partition_by_12_on_values 1.00 1088.8±6.97µs ? ?/sec 1.00 1092.7±4.53µs ? ?/sec
physical_window_function_partition_by_30_on_values 1.00 2.2±0.01ms ? ?/sec 1.00 2.2±0.01ms ? ?/sec
physical_window_function_partition_by_4_on_values 1.00 664.3±5.65µs ? ?/sec 1.01 669.0±5.94µs ? ?/sec
physical_window_function_partition_by_7_on_values 1.00 810.6±3.49µs ? ?/sec 1.01 815.0±12.95µs ? ?/sec
physical_window_function_partition_by_8_on_values 1.00 867.6±6.07µs ? ?/sec 1.01 878.6±7.25µs ? ?/sec
with_param_values_many_columns 1.00 129.0±0.85µs ? ?/sec 1.00 128.8±0.73µs ? ?/sec

alamb · 2025-10-31T23:28:27Z

🤖 ./gh_compare_branch_bench.shBenchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/stop_cloning_fields (074e485) to e65dafediff
BENCH_NAME=sql_planner
BENCH_COMMAND=cargo bench --bench sql_planner
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_stop_cloning_fields
Results will be posted here when complete

alamb · 2025-11-01T00:34:13Z

🤖: Benchmark completed

Details

group alamb_stop_cloning_fields main
----- ------------------------- ----
logical_aggregate_with_join 1.00 627.6±3.84µs ? ?/sec 1.00 627.6±8.16µs ? ?/sec
logical_select_all_from_1000 1.00 10.8±0.06ms ? ?/sec 1.04 11.1±0.06ms ? ?/sec
logical_select_one_from_700 1.01 418.2±3.54µs ? ?/sec 1.00 415.9±2.80µs ? ?/sec
logical_trivial_join_high_numbered_columns 1.00 372.0±2.24µs ? ?/sec 1.00 371.7±5.18µs ? ?/sec
logical_trivial_join_low_numbered_columns 1.00 357.0±3.65µs ? ?/sec 1.00 355.9±1.08µs ? ?/sec
physical_intersection 1.00 833.0±3.86µs ? ?/sec 1.01 838.5±5.75µs ? ?/sec
physical_join_consider_sort 1.01 1403.1±15.26µs ? ?/sec 1.00 1390.8±11.71µs ? ?/sec
physical_join_distinct 1.00 350.3±2.22µs ? ?/sec 1.00 349.4±5.90µs ? ?/sec
physical_many_self_joins 1.01 9.8±0.06ms ? ?/sec 1.00 9.7±0.07ms ? ?/sec
physical_plan_clickbench_all 1.07 195.4±2.81ms ? ?/sec 1.00 183.4±1.76ms ? ?/sec
physical_plan_clickbench_q1 1.10 2.6±0.09ms ? ?/sec 1.00 2.4±0.03ms ? ?/sec
physical_plan_clickbench_q10 1.00 3.3±0.04ms ? ?/sec 1.08 3.6±0.13ms ? ?/sec
physical_plan_clickbench_q11 1.00 3.6±0.07ms ? ?/sec 1.17 4.2±0.23ms ? ?/sec
physical_plan_clickbench_q12 1.00 3.7±0.06ms ? ?/sec 1.10 4.1±0.17ms ? ?/sec
physical_plan_clickbench_q13 1.00 3.3±0.06ms ? ?/sec 1.10 3.6±0.14ms ? ?/sec
physical_plan_clickbench_q14 1.00 3.5±0.06ms ? ?/sec 1.12 4.0±0.14ms ? ?/sec
physical_plan_clickbench_q15 1.00 3.4±0.06ms ? ?/sec 1.11 3.8±0.21ms ? ?/sec
physical_plan_clickbench_q16 1.00 3.2±0.05ms ? ?/sec 1.05 3.4±0.11ms ? ?/sec
physical_plan_clickbench_q17 1.00 3.4±0.06ms ? ?/sec 1.09 3.7±0.18ms ? ?/sec
physical_plan_clickbench_q18 1.00 2.9±0.04ms ? ?/sec 1.03 3.0±0.09ms ? ?/sec
physical_plan_clickbench_q19 1.00 3.8±0.07ms ? ?/sec 1.04 3.9±0.24ms ? ?/sec
physical_plan_clickbench_q2 1.11 3.1±0.11ms ? ?/sec 1.00 2.8±0.02ms ? ?/sec
physical_plan_clickbench_q20 1.02 2.6±0.04ms ? ?/sec 1.00 2.5±0.07ms ? ?/sec
physical_plan_clickbench_q21 1.09 3.1±0.31ms ? ?/sec 1.00 2.9±0.04ms ? ?/sec
physical_plan_clickbench_q22 1.10 3.9±0.14ms ? ?/sec 1.00 3.5±0.18ms ? ?/sec
physical_plan_clickbench_q23 1.02 4.2±0.13ms ? ?/sec 1.00 4.1±0.30ms ? ?/sec
physical_plan_clickbench_q24 1.03 4.6±0.14ms ? ?/sec 1.00 4.5±0.31ms ? ?/sec
physical_plan_clickbench_q25 1.13 3.4±0.14ms ? ?/sec 1.00 3.0±0.03ms ? ?/sec
physical_plan_clickbench_q26 1.11 3.1±0.11ms ? ?/sec 1.00 2.8±0.04ms ? ?/sec
physical_plan_clickbench_q27 1.03 3.4±0.12ms ? ?/sec 1.00 3.3±0.14ms ? ?/sec
physical_plan_clickbench_q28 1.12 4.2±0.14ms ? ?/sec 1.00 3.8±0.05ms ? ?/sec
physical_plan_clickbench_q29 1.11 4.6±0.15ms ? ?/sec 1.00 4.1±0.07ms ? ?/sec
physical_plan_clickbench_q3 1.12 3.0±0.11ms ? ?/sec 1.00 2.7±0.02ms ? ?/sec
physical_plan_clickbench_q30 1.01 13.2±0.14ms ? ?/sec 1.00 13.0±0.35ms ? ?/sec
physical_plan_clickbench_q31 1.10 4.2±0.14ms ? ?/sec 1.00 3.8±0.16ms ? ?/sec
physical_plan_clickbench_q32 1.10 4.2±0.15ms ? ?/sec 1.00 3.8±0.13ms ? ?/sec
physical_plan_clickbench_q33 1.00 3.6±0.13ms ? ?/sec 1.07 3.9±0.17ms ? ?/sec
physical_plan_clickbench_q34 1.00 3.2±0.13ms ? ?/sec 1.08 3.4±0.12ms ? ?/sec
physical_plan_clickbench_q35 1.00 3.2±0.11ms ? ?/sec 1.12 3.6±0.13ms ? ?/sec
physical_plan_clickbench_q36 1.00 3.8±0.06ms ? ?/sec 1.16 4.4±0.15ms ? ?/sec
physical_plan_clickbench_q37 1.00 4.1±0.08ms ? ?/sec 1.14 4.7±0.15ms ? ?/sec
physical_plan_clickbench_q38 1.00 4.1±0.08ms ? ?/sec 1.17 4.7±0.11ms ? ?/sec
physical_plan_clickbench_q39 1.00 3.9±0.07ms ? ?/sec 1.06 4.1±0.17ms ? ?/sec
physical_plan_clickbench_q4 1.04 2.6±0.05ms ? ?/sec 1.00 2.5±0.02ms ? ?/sec
physical_plan_clickbench_q40 1.00 4.6±0.07ms ? ?/sec 1.07 4.9±0.18ms ? ?/sec
physical_plan_clickbench_q41 1.00 4.1±0.07ms ? ?/sec 1.05 4.3±0.18ms ? ?/sec
physical_plan_clickbench_q42 1.00 4.0±0.06ms ? ?/sec 1.05 4.3±0.19ms ? ?/sec
physical_plan_clickbench_q43 1.00 4.5±0.08ms ? ?/sec 1.05 4.7±0.17ms ? ?/sec
physical_plan_clickbench_q44 1.00 2.7±0.05ms ? ?/sec 1.03 2.8±0.09ms ? ?/sec
physical_plan_clickbench_q45 1.01 2.7±0.04ms ? ?/sec 1.00 2.7±0.10ms ? ?/sec
physical_plan_clickbench_q46 1.08 3.3±0.16ms ? ?/sec 1.00 3.1±0.03ms ? ?/sec
physical_plan_clickbench_q47 1.12 4.0±0.13ms ? ?/sec 1.00 3.6±0.04ms ? ?/sec
physical_plan_clickbench_q48 1.11 4.9±0.16ms ? ?/sec 1.00 4.4±0.08ms ? ?/sec
physical_plan_clickbench_q49 1.12 5.2±0.15ms ? ?/sec 1.00 4.7±0.07ms ? ?/sec
physical_plan_clickbench_q5 1.01 2.8±0.07ms ? ?/sec 1.00 2.8±0.19ms ? ?/sec
physical_plan_clickbench_q50 1.12 4.8±0.14ms ? ?/sec 1.00 4.2±0.06ms ? ?/sec
physical_plan_clickbench_q51 1.05 3.3±0.12ms ? ?/sec 1.00 3.2±0.08ms ? ?/sec
physical_plan_clickbench_q6 1.00 2.8±0.11ms ? ?/sec 1.11 3.2±0.40ms ? ?/sec
physical_plan_clickbench_q7 1.00 2.5±0.07ms ? ?/sec 1.04 2.6±0.10ms ? ?/sec
physical_plan_clickbench_q8 1.00 3.5±0.09ms ? ?/sec 1.12 3.9±0.20ms ? ?/sec
physical_plan_clickbench_q9 1.00 3.3±0.13ms ? ?/sec 1.13 3.7±0.20ms ? ?/sec
physical_plan_tpcds_all 1.01 1067.4±9.05ms ? ?/sec 1.00 1052.4±14.96ms ? ?/sec
physical_plan_tpch_all 1.00 64.6±0.47ms ? ?/sec 1.03 66.7±0.81ms ? ?/sec
physical_plan_tpch_q1 1.01 2.1±0.04ms ? ?/sec 1.00 2.1±0.02ms ? ?/sec
physical_plan_tpch_q10 1.03 4.1±0.07ms ? ?/sec 1.00 4.0±0.03ms ? ?/sec
physical_plan_tpch_q11 1.04 3.7±0.06ms ? ?/sec 1.00 3.6±0.02ms ? ?/sec
physical_plan_tpch_q12 1.03 1862.9±44.70µs ? ?/sec 1.00 1816.4±8.55µs ? ?/sec
physical_plan_tpch_q13 1.00 1466.6±11.01µs ? ?/sec 1.00 1462.5±11.12µs ? ?/sec
physical_plan_tpch_q14 1.00 1954.2±15.86µs ? ?/sec 1.02 1987.0±18.98µs ? ?/sec
physical_plan_tpch_q16 1.00 2.5±0.01ms ? ?/sec 1.02 2.5±0.02ms ? ?/sec
physical_plan_tpch_q17 1.00 2.6±0.02ms ? ?/sec 1.03 2.7±0.03ms ? ?/sec
physical_plan_tpch_q18 1.00 2.6±0.02ms ? ?/sec 1.02 2.7±0.03ms ? ?/sec
physical_plan_tpch_q19 1.00 3.2±0.02ms ? ?/sec 1.02 3.3±0.04ms ? ?/sec
physical_plan_tpch_q2 1.05 6.4±0.09ms ? ?/sec 1.00 6.0±0.10ms ? ?/sec
physical_plan_tpch_q20 1.00 3.2±0.02ms ? ?/sec 1.03 3.3±0.05ms ? ?/sec
physical_plan_tpch_q21 1.00 4.2±0.03ms ? ?/sec 1.04 4.4±0.06ms ? ?/sec
physical_plan_tpch_q22 1.00 2.8±0.03ms ? ?/sec 1.04 2.9±0.04ms ? ?/sec
physical_plan_tpch_q3 1.05 2.8±0.06ms ? ?/sec 1.00 2.7±0.03ms ? ?/sec
physical_plan_tpch_q4 1.01 1522.5±22.73µs ? ?/sec 1.00 1508.0±5.57µs ? ?/sec
physical_plan_tpch_q5 1.06 3.5±0.07ms ? ?/sec 1.00 3.3±0.01ms ? ?/sec
physical_plan_tpch_q6 1.00 866.6±10.60µs ? ?/sec 1.00 863.8±9.50µs ? ?/sec
physical_plan_tpch_q7 1.05 4.4±0.07ms ? ?/sec 1.00 4.2±0.02ms ? ?/sec
physical_plan_tpch_q8 1.06 5.8±0.08ms ? ?/sec 1.00 5.5±0.05ms ? ?/sec
physical_plan_tpch_q9 1.04 4.2±0.06ms ? ?/sec 1.00 4.0±0.02ms ? ?/sec
physical_select_aggregates_from_200 1.01 17.3±0.09ms ? ?/sec 1.00 17.1±0.55ms ? ?/sec
physical_select_all_from_1000 1.00 23.8±0.12ms ? ?/sec 1.02 24.2±0.13ms ? ?/sec
physical_select_one_from_700 1.01 1088.8±10.38µs ? ?/sec 1.00 1080.7±9.20µs ? ?/sec
physical_sorted_union_order_by_10 1.00 13.2±0.13ms ? ?/sec 1.02 13.4±0.15ms ? ?/sec
physical_sorted_union_order_by_50 1.00 385.8±6.22ms ? ?/sec 1.03 397.6±3.80ms ? ?/sec
physical_theta_join_consider_sort 1.00 1769.3±9.87µs ? ?/sec 1.00 1765.2±13.13µs ? ?/sec
physical_unnest_to_join 1.00 1839.2±16.14µs ? ?/sec 1.02 1873.8±9.34µs ? ?/sec
physical_window_function_partition_by_12_on_values 1.00 1089.6±10.18µs ? ?/sec 1.00 1093.8±7.14µs ? ?/sec
physical_window_function_partition_by_30_on_values 1.01 2.3±0.03ms ? ?/sec 1.00 2.3±0.01ms ? ?/sec
physical_window_function_partition_by_4_on_values 1.00 666.7±7.30µs ? ?/sec 1.00 665.5±3.62µs ? ?/sec
physical_window_function_partition_by_7_on_values 1.00 817.0±7.47µs ? ?/sec 1.00 818.3±4.13µs ? ?/sec
physical_window_function_partition_by_8_on_values 1.00 879.2±6.38µs ? ?/sec 1.00 877.6±5.22µs ? ?/sec
with_param_values_many_columns 1.00 127.7±0.76µs ? ?/sec 1.00 127.9±0.76µs ? ?/sec

alamb · 2025-11-11T21:15:35Z

 let filter_df_fields = filter_df_fields
 .into_iter()
 .map(|(qualifier, field)| {
- (qualifier.cloned(), Arc::new(field.clone()))


this was always a deep clone that is now remved

alamb · 2025-11-11T23:52:27Z

🤖 ./gh_compare_branch_bench.shBenchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/stop_cloning_fields (9dcfdfc) to 03d1974diff
BENCH_NAME=sql_planner
BENCH_COMMAND=cargo bench --bench sql_planner
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_stop_cloning_fields
Results will be posted here when complete

alamb · 2025-11-12T00:57:42Z

🤖: Benchmark completed

Details

group alamb_stop_cloning_fields main
----- ------------------------- ----
logical_aggregate_with_join 1.00 631.0±15.72µs ? ?/sec 1.01 636.6±2.18µs ? ?/sec
logical_select_all_from_1000 1.00 10.6±0.03ms ? ?/sec 1.05 11.1±0.11ms ? ?/sec
logical_select_one_from_700 1.00 420.7±6.30µs ? ?/sec 1.00 420.6±1.48µs ? ?/sec
logical_trivial_join_high_numbered_columns 1.00 376.3±2.09µs ? ?/sec 1.00 377.4±1.67µs ? ?/sec
logical_trivial_join_low_numbered_columns 1.00 362.3±1.67µs ? ?/sec 1.00 362.2±2.11µs ? ?/sec
physical_intersection 1.00 834.0±4.70µs ? ?/sec 1.01 841.5±9.93µs ? ?/sec
physical_join_consider_sort 1.00 1397.9±5.68µs ? ?/sec 1.00 1401.3±6.42µs ? ?/sec
physical_join_distinct 1.00 354.4±1.72µs ? ?/sec 1.00 353.2±1.24µs ? ?/sec
physical_many_self_joins 1.00 9.7±0.04ms ? ?/sec 1.00 9.7±0.05ms ? ?/sec
physical_plan_clickbench_all 1.00 180.7±2.21ms ? ?/sec 1.00 180.7±1.40ms ? ?/sec
physical_plan_clickbench_q1 1.01 2.4±0.03ms ? ?/sec 1.00 2.3±0.02ms ? ?/sec
physical_plan_clickbench_q10 1.00 3.2±0.03ms ? ?/sec 1.01 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q11 1.00 3.4±0.03ms ? ?/sec 1.01 3.4±0.04ms ? ?/sec
physical_plan_clickbench_q12 1.00 3.5±0.04ms ? ?/sec 1.01 3.6±0.04ms ? ?/sec
physical_plan_clickbench_q13 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.04ms ? ?/sec
physical_plan_clickbench_q14 1.00 3.4±0.04ms ? ?/sec 1.00 3.4±0.05ms ? ?/sec
physical_plan_clickbench_q15 1.00 3.3±0.03ms ? ?/sec 1.00 3.3±0.03ms ? ?/sec
physical_plan_clickbench_q16 1.00 3.1±0.03ms ? ?/sec 1.00 3.1±0.04ms ? ?/sec
physical_plan_clickbench_q17 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q18 1.01 2.8±0.24ms ? ?/sec 1.00 2.8±0.03ms ? ?/sec
physical_plan_clickbench_q19 1.00 3.6±0.03ms ? ?/sec 1.01 3.6±0.06ms ? ?/sec
physical_plan_clickbench_q2 1.00 2.8±0.03ms ? ?/sec 1.02 2.8±0.13ms ? ?/sec
physical_plan_clickbench_q20 1.00 2.5±0.02ms ? ?/sec 1.00 2.5±0.03ms ? ?/sec
physical_plan_clickbench_q21 1.00 2.8±0.03ms ? ?/sec 1.00 2.8±0.02ms ? ?/sec
physical_plan_clickbench_q22 1.00 3.4±0.03ms ? ?/sec 1.00 3.4±0.03ms ? ?/sec
physical_plan_clickbench_q23 1.00 3.7±0.04ms ? ?/sec 1.00 3.7±0.03ms ? ?/sec
physical_plan_clickbench_q24 1.00 4.1±0.05ms ? ?/sec 1.03 4.2±0.06ms ? ?/sec
physical_plan_clickbench_q25 1.00 3.0±0.03ms ? ?/sec 1.00 3.0±0.02ms ? ?/sec
physical_plan_clickbench_q26 1.00 2.8±0.02ms ? ?/sec 1.00 2.8±0.04ms ? ?/sec
physical_plan_clickbench_q27 1.00 3.0±0.02ms ? ?/sec 1.00 3.0±0.03ms ? ?/sec
physical_plan_clickbench_q28 1.00 3.7±0.06ms ? ?/sec 1.00 3.7±0.05ms ? ?/sec
physical_plan_clickbench_q29 1.00 4.0±0.05ms ? ?/sec 1.00 4.0±0.04ms ? ?/sec
physical_plan_clickbench_q3 1.00 2.7±0.03ms ? ?/sec 1.00 2.7±0.03ms ? ?/sec
physical_plan_clickbench_q30 1.00 12.3±0.14ms ? ?/sec 1.04 12.8±0.16ms ? ?/sec
physical_plan_clickbench_q31 1.00 3.7±0.05ms ? ?/sec 1.00 3.7±0.04ms ? ?/sec
physical_plan_clickbench_q32 1.00 3.7±0.06ms ? ?/sec 1.00 3.7±0.04ms ? ?/sec
physical_plan_clickbench_q33 1.00 3.2±0.03ms ? ?/sec 1.01 3.2±0.04ms ? ?/sec
physical_plan_clickbench_q34 1.00 2.9±0.03ms ? ?/sec 1.00 2.9±0.04ms ? ?/sec
physical_plan_clickbench_q35 1.00 3.0±0.04ms ? ?/sec 1.01 3.0±0.03ms ? ?/sec
physical_plan_clickbench_q36 1.00 3.7±0.06ms ? ?/sec 1.01 3.7±0.05ms ? ?/sec
physical_plan_clickbench_q37 1.04 4.0±0.08ms ? ?/sec 1.00 3.9±0.06ms ? ?/sec
physical_plan_clickbench_q38 1.03 4.0±0.07ms ? ?/sec 1.00 3.8±0.05ms ? ?/sec
physical_plan_clickbench_q39 1.03 3.8±0.06ms ? ?/sec 1.00 3.7±0.11ms ? ?/sec
physical_plan_clickbench_q4 1.00 2.5±0.02ms ? ?/sec 1.00 2.5±0.03ms ? ?/sec
physical_plan_clickbench_q40 1.04 4.7±0.08ms ? ?/sec 1.00 4.5±0.06ms ? ?/sec
physical_plan_clickbench_q41 1.03 4.0±0.06ms ? ?/sec 1.00 3.9±0.04ms ? ?/sec
physical_plan_clickbench_q42 1.04 4.0±0.06ms ? ?/sec 1.00 3.8±0.04ms ? ?/sec
physical_plan_clickbench_q43 1.04 4.4±0.08ms ? ?/sec 1.00 4.3±0.05ms ? ?/sec
physical_plan_clickbench_q44 1.02 2.6±0.03ms ? ?/sec 1.00 2.6±0.03ms ? ?/sec
physical_plan_clickbench_q45 1.01 2.6±0.03ms ? ?/sec 1.00 2.6±0.03ms ? ?/sec
physical_plan_clickbench_q46 1.02 3.1±0.04ms ? ?/sec 1.00 3.0±0.04ms ? ?/sec
physical_plan_clickbench_q47 1.00 3.6±0.05ms ? ?/sec 1.00 3.6±0.03ms ? ?/sec
physical_plan_clickbench_q48 1.00 4.3±0.06ms ? ?/sec 1.00 4.3±0.04ms ? ?/sec
physical_plan_clickbench_q49 1.00 4.6±0.07ms ? ?/sec 1.00 4.6±0.07ms ? ?/sec
physical_plan_clickbench_q5 1.00 2.7±0.02ms ? ?/sec 1.00 2.7±0.03ms ? ?/sec
physical_plan_clickbench_q50 1.00 4.2±0.04ms ? ?/sec 1.01 4.2±0.07ms ? ?/sec
physical_plan_clickbench_q51 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q6 1.01 2.7±0.04ms ? ?/sec 1.00 2.7±0.02ms ? ?/sec
physical_plan_clickbench_q7 1.00 2.4±0.01ms ? ?/sec 1.00 2.4±0.02ms ? ?/sec
physical_plan_clickbench_q8 1.00 3.2±0.02ms ? ?/sec 1.02 3.3±0.04ms ? ?/sec
physical_plan_clickbench_q9 1.00 3.1±0.03ms ? ?/sec 1.01 3.1±0.04ms ? ?/sec
physical_plan_tpcds_all 1.00 1016.1±3.64ms ? ?/sec 1.03 1043.3±8.69ms ? ?/sec
physical_plan_tpch_all 1.00 63.1±0.22ms ? ?/sec 1.03 64.8±0.42ms ? ?/sec
physical_plan_tpch_q1 1.00 1986.8±10.16µs ? ?/sec 1.03 2.0±0.01ms ? ?/sec
physical_plan_tpch_q10 1.00 3.9±0.01ms ? ?/sec 1.03 4.0±0.02ms ? ?/sec
physical_plan_tpch_q11 1.00 3.5±0.03ms ? ?/sec 1.01 3.5±0.02ms ? ?/sec
physical_plan_tpch_q12 1.00 1792.3±20.52µs ? ?/sec 1.01 1801.9±10.83µs ? ?/sec
physical_plan_tpch_q13 1.00 1437.4±12.85µs ? ?/sec 1.01 1453.5±21.30µs ? ?/sec
physical_plan_tpch_q14 1.00 1941.1±6.73µs ? ?/sec 1.01 1962.8±10.82µs ? ?/sec
physical_plan_tpch_q16 1.00 2.4±0.01ms ? ?/sec 1.01 2.5±0.01ms ? ?/sec
physical_plan_tpch_q17 1.00 2.6±0.01ms ? ?/sec 1.02 2.6±0.01ms ? ?/sec
physical_plan_tpch_q18 1.00 2.6±0.01ms ? ?/sec 1.02 2.6±0.01ms ? ?/sec
physical_plan_tpch_q19 1.00 3.2±0.02ms ? ?/sec 1.01 3.2±0.02ms ? ?/sec
physical_plan_tpch_q2 1.00 5.8±0.02ms ? ?/sec 1.02 5.9±0.09ms ? ?/sec
physical_plan_tpch_q20 1.00 3.1±0.02ms ? ?/sec 1.01 3.2±0.03ms ? ?/sec
physical_plan_tpch_q21 1.00 4.1±0.02ms ? ?/sec 1.02 4.2±0.02ms ? ?/sec
physical_plan_tpch_q22 1.00 2.7±0.01ms ? ?/sec 1.03 2.8±0.02ms ? ?/sec
physical_plan_tpch_q3 1.00 2.6±0.01ms ? ?/sec 1.01 2.7±0.01ms ? ?/sec
physical_plan_tpch_q4 1.00 1458.0±5.12µs ? ?/sec 1.02 1486.5±9.09µs ? ?/sec
physical_plan_tpch_q5 1.00 3.2±0.01ms ? ?/sec 1.01 3.2±0.02ms ? ?/sec
physical_plan_tpch_q6 1.00 850.8±4.88µs ? ?/sec 1.01 859.6±5.38µs ? ?/sec
physical_plan_tpch_q7 1.00 4.1±0.02ms ? ?/sec 1.01 4.2±0.02ms ? ?/sec
physical_plan_tpch_q8 1.00 5.4±0.04ms ? ?/sec 1.01 5.5±0.04ms ? ?/sec
physical_plan_tpch_q9 1.00 4.0±0.02ms ? ?/sec 1.01 4.0±0.02ms ? ?/sec
physical_select_aggregates_from_200 1.00 16.7±0.08ms ? ?/sec 1.02 17.0±0.11ms ? ?/sec
physical_select_all_from_1000 1.00 23.1±0.09ms ? ?/sec 1.04 24.0±0.10ms ? ?/sec
physical_select_one_from_700 1.01 1092.0±9.06µs ? ?/sec 1.00 1086.2±6.81µs ? ?/sec
physical_sorted_union_order_by_10_int64 1.00 5.9±0.02ms ? ?/sec 1.02 6.0±0.03ms ? ?/sec
physical_sorted_union_order_by_10_uint64 1.00 12.6±0.11ms ? ?/sec 1.03 13.0±0.11ms ? ?/sec
physical_sorted_union_order_by_50_int64 1.00 160.8±1.80ms ? ?/sec 1.02 163.8±1.38ms ? ?/sec
physical_sorted_union_order_by_50_uint64 1.00 368.3±2.45ms ? ?/sec 1.03 380.1±2.26ms ? ?/sec
physical_theta_join_consider_sort 1.00 1765.3±11.23µs ? ?/sec 1.00 1763.4±8.53µs ? ?/sec
physical_unnest_to_join 1.00 1804.1±7.14µs ? ?/sec 1.03 1851.5±7.45µs ? ?/sec
physical_window_function_partition_by_12_on_values 1.00 1078.9±6.24µs ? ?/sec 1.01 1085.7±5.99µs ? ?/sec
physical_window_function_partition_by_30_on_values 1.00 2.2±0.01ms ? ?/sec 1.01 2.2±0.01ms ? ?/sec
physical_window_function_partition_by_4_on_values 1.00 655.2±3.38µs ? ?/sec 1.01 663.2±3.64µs ? ?/sec
physical_window_function_partition_by_7_on_values 1.00 800.5±4.32µs ? ?/sec 1.01 809.6±5.31µs ? ?/sec
physical_window_function_partition_by_8_on_values 1.00 862.4±3.24µs ? ?/sec 1.01 872.3±5.12µs ? ?/sec
with_param_values_many_columns 1.00 127.7±1.50µs ? ?/sec 1.00 127.4±0.53µs ? ?/sec

alamb · 2025-11-12T00:57:44Z

🤖 ./gh_compare_branch_bench.shBenchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/stop_cloning_fields (9dcfdfc) to 03d1974diff
BENCH_NAME=sql_planner
BENCH_COMMAND=cargo bench --bench sql_planner
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_stop_cloning_fields
Results will be posted here when complete

alamb · 2025-11-12T02:03:04Z

🤖: Benchmark completed

Details

group alamb_stop_cloning_fields main
----- ------------------------- ----
logical_aggregate_with_join 1.00 627.0±4.35µs ? ?/sec 1.01 630.8±2.60µs ? ?/sec
logical_select_all_from_1000 1.00 10.6±0.06ms ? ?/sec 1.05 11.1±0.11ms ? ?/sec
logical_select_one_from_700 1.00 417.8±1.53µs ? ?/sec 1.01 420.7±5.61µs ? ?/sec
logical_trivial_join_high_numbered_columns 1.00 375.3±1.27µs ? ?/sec 1.00 376.8±1.82µs ? ?/sec
logical_trivial_join_low_numbered_columns 1.00 360.9±1.48µs ? ?/sec 1.01 363.8±2.45µs ? ?/sec
physical_intersection 1.01 846.0±4.87µs ? ?/sec 1.00 839.7±5.72µs ? ?/sec
physical_join_consider_sort 1.00 1388.7±7.14µs ? ?/sec 1.01 1398.7±12.78µs ? ?/sec
physical_join_distinct 1.00 353.2±5.85µs ? ?/sec 1.01 355.6±1.94µs ? ?/sec
physical_many_self_joins 1.00 9.7±0.03ms ? ?/sec 1.00 9.7±0.11ms ? ?/sec
physical_plan_clickbench_all 1.00 181.4±1.66ms ? ?/sec 1.00 181.2±1.86ms ? ?/sec
physical_plan_clickbench_q1 1.00 2.3±0.02ms ? ?/sec 1.01 2.4±0.02ms ? ?/sec
physical_plan_clickbench_q10 1.00 3.2±0.02ms ? ?/sec 1.00 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q11 1.00 3.4±0.02ms ? ?/sec 1.00 3.4±0.05ms ? ?/sec
physical_plan_clickbench_q12 1.00 3.5±0.03ms ? ?/sec 1.01 3.6±0.05ms ? ?/sec
physical_plan_clickbench_q13 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.05ms ? ?/sec
physical_plan_clickbench_q14 1.01 3.4±0.03ms ? ?/sec 1.00 3.4±0.03ms ? ?/sec
physical_plan_clickbench_q15 1.01 3.3±0.04ms ? ?/sec 1.00 3.3±0.03ms ? ?/sec
physical_plan_clickbench_q16 1.01 3.1±0.05ms ? ?/sec 1.00 3.1±0.03ms ? ?/sec
physical_plan_clickbench_q17 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q18 1.01 2.8±0.02ms ? ?/sec 1.00 2.8±0.03ms ? ?/sec
physical_plan_clickbench_q19 1.00 3.6±0.03ms ? ?/sec 1.00 3.6±0.04ms ? ?/sec
physical_plan_clickbench_q2 1.00 2.8±0.02ms ? ?/sec 1.00 2.8±0.03ms ? ?/sec
physical_plan_clickbench_q20 1.00 2.5±0.02ms ? ?/sec 1.00 2.5±0.04ms ? ?/sec
physical_plan_clickbench_q21 1.00 2.8±0.03ms ? ?/sec 1.00 2.8±0.03ms ? ?/sec
physical_plan_clickbench_q22 1.00 3.4±0.07ms ? ?/sec 1.00 3.4±0.04ms ? ?/sec
physical_plan_clickbench_q23 1.00 3.7±0.03ms ? ?/sec 1.00 3.7±0.03ms ? ?/sec
physical_plan_clickbench_q24 1.00 4.1±0.03ms ? ?/sec 1.02 4.2±0.05ms ? ?/sec
physical_plan_clickbench_q25 1.01 3.0±0.02ms ? ?/sec 1.00 3.0±0.02ms ? ?/sec
physical_plan_clickbench_q26 1.01 2.8±0.03ms ? ?/sec 1.00 2.8±0.02ms ? ?/sec
physical_plan_clickbench_q27 1.02 3.0±0.03ms ? ?/sec 1.00 3.0±0.03ms ? ?/sec
physical_plan_clickbench_q28 1.00 3.7±0.03ms ? ?/sec 1.00 3.7±0.04ms ? ?/sec
physical_plan_clickbench_q29 1.00 4.1±0.06ms ? ?/sec 1.01 4.1±0.19ms ? ?/sec
physical_plan_clickbench_q3 1.00 2.7±0.03ms ? ?/sec 1.01 2.7±0.03ms ? ?/sec
physical_plan_clickbench_q30 1.00 12.4±0.12ms ? ?/sec 1.03 12.7±0.15ms ? ?/sec
physical_plan_clickbench_q31 1.00 3.7±0.03ms ? ?/sec 1.00 3.7±0.04ms ? ?/sec
physical_plan_clickbench_q32 1.00 3.7±0.03ms ? ?/sec 1.00 3.7±0.03ms ? ?/sec
physical_plan_clickbench_q33 1.00 3.2±0.04ms ? ?/sec 1.01 3.2±0.05ms ? ?/sec
physical_plan_clickbench_q34 1.00 2.9±0.02ms ? ?/sec 1.01 2.9±0.04ms ? ?/sec
physical_plan_clickbench_q35 1.00 3.0±0.03ms ? ?/sec 1.00 3.0±0.02ms ? ?/sec
physical_plan_clickbench_q36 1.00 3.7±0.03ms ? ?/sec 1.01 3.7±0.03ms ? ?/sec
physical_plan_clickbench_q37 1.00 3.9±0.03ms ? ?/sec 1.01 3.9±0.05ms ? ?/sec
physical_plan_clickbench_q38 1.01 3.9±0.03ms ? ?/sec 1.00 3.8±0.04ms ? ?/sec
physical_plan_clickbench_q39 1.00 3.7±0.03ms ? ?/sec 1.00 3.7±0.04ms ? ?/sec
physical_plan_clickbench_q4 1.01 2.5±0.02ms ? ?/sec 1.00 2.4±0.02ms ? ?/sec
physical_plan_clickbench_q40 1.00 4.6±0.08ms ? ?/sec 1.00 4.6±0.07ms ? ?/sec
physical_plan_clickbench_q41 1.00 3.9±0.03ms ? ?/sec 1.02 3.9±0.04ms ? ?/sec
physical_plan_clickbench_q42 1.00 3.8±0.04ms ? ?/sec 1.00 3.9±0.04ms ? ?/sec
physical_plan_clickbench_q43 1.00 4.3±0.05ms ? ?/sec 1.00 4.3±0.05ms ? ?/sec
physical_plan_clickbench_q44 1.00 2.6±0.02ms ? ?/sec 1.00 2.6±0.02ms ? ?/sec
physical_plan_clickbench_q45 1.00 2.6±0.02ms ? ?/sec 1.00 2.6±0.02ms ? ?/sec
physical_plan_clickbench_q46 1.00 3.0±0.06ms ? ?/sec 1.00 3.0±0.02ms ? ?/sec
physical_plan_clickbench_q47 1.00 3.5±0.03ms ? ?/sec 1.01 3.6±0.04ms ? ?/sec
physical_plan_clickbench_q48 1.00 4.3±0.04ms ? ?/sec 1.01 4.3±0.04ms ? ?/sec
physical_plan_clickbench_q49 1.00 4.6±0.07ms ? ?/sec 1.01 4.6±0.05ms ? ?/sec
physical_plan_clickbench_q5 1.00 2.7±0.02ms ? ?/sec 1.00 2.7±0.03ms ? ?/sec
physical_plan_clickbench_q50 1.00 4.2±0.05ms ? ?/sec 1.01 4.2±0.05ms ? ?/sec
physical_plan_clickbench_q51 1.00 3.2±0.03ms ? ?/sec 1.00 3.2±0.02ms ? ?/sec
physical_plan_clickbench_q6 1.00 2.7±0.02ms ? ?/sec 1.00 2.7±0.03ms ? ?/sec
physical_plan_clickbench_q7 1.00 2.4±0.02ms ? ?/sec 1.00 2.4±0.03ms ? ?/sec
physical_plan_clickbench_q8 1.01 3.3±0.05ms ? ?/sec 1.00 3.2±0.03ms ? ?/sec
physical_plan_clickbench_q9 1.00 3.1±0.03ms ? ?/sec 1.00 3.1±0.03ms ? ?/sec
physical_plan_tpcds_all 1.00 1015.8±3.91ms ? ?/sec 1.02 1031.8±3.59ms ? ?/sec
physical_plan_tpch_all 1.00 63.6±0.48ms ? ?/sec 1.01 64.0±0.36ms ? ?/sec
physical_plan_tpch_q1 1.00 1977.2±5.48µs ? ?/sec 1.03 2.0±0.01ms ? ?/sec
physical_plan_tpch_q10 1.00 3.9±0.03ms ? ?/sec 1.01 4.0±0.02ms ? ?/sec
physical_plan_tpch_q11 1.00 3.5±0.02ms ? ?/sec 1.01 3.5±0.03ms ? ?/sec
physical_plan_tpch_q12 1.00 1791.0±8.51µs ? ?/sec 1.01 1805.6±7.71µs ? ?/sec
physical_plan_tpch_q13 1.00 1439.9±7.42µs ? ?/sec 1.01 1447.2±6.15µs ? ?/sec
physical_plan_tpch_q14 1.00 1930.1±15.24µs ? ?/sec 1.02 1959.4±11.80µs ? ?/sec
physical_plan_tpch_q16 1.00 2.4±0.01ms ? ?/sec 1.01 2.5±0.01ms ? ?/sec
physical_plan_tpch_q17 1.00 2.5±0.01ms ? ?/sec 1.02 2.6±0.01ms ? ?/sec
physical_plan_tpch_q18 1.00 2.6±0.02ms ? ?/sec 1.02 2.7±0.01ms ? ?/sec
physical_plan_tpch_q19 1.00 3.2±0.01ms ? ?/sec 1.01 3.2±0.02ms ? ?/sec
physical_plan_tpch_q2 1.00 5.8±0.03ms ? ?/sec 1.02 5.9±0.03ms ? ?/sec
physical_plan_tpch_q20 1.00 3.1±0.07ms ? ?/sec 1.01 3.2±0.01ms ? ?/sec
physical_plan_tpch_q21 1.00 4.1±0.03ms ? ?/sec 1.02 4.2±0.02ms ? ?/sec
physical_plan_tpch_q22 1.00 2.7±0.01ms ? ?/sec 1.02 2.8±0.02ms ? ?/sec
physical_plan_tpch_q3 1.00 2.6±0.01ms ? ?/sec 1.02 2.7±0.01ms ? ?/sec
physical_plan_tpch_q4 1.00 1457.7±6.69µs ? ?/sec 1.02 1489.2±13.68µs ? ?/sec
physical_plan_tpch_q5 1.00 3.2±0.02ms ? ?/sec 1.01 3.2±0.01ms ? ?/sec
physical_plan_tpch_q6 1.00 860.7±7.34µs ? ?/sec 1.00 862.3±3.97µs ? ?/sec
physical_plan_tpch_q7 1.01 4.2±0.03ms ? ?/sec 1.00 4.2±0.02ms ? ?/sec
physical_plan_tpch_q8 1.00 5.5±0.06ms ? ?/sec 1.00 5.5±0.03ms ? ?/sec
physical_plan_tpch_q9 1.00 4.0±0.09ms ? ?/sec 1.00 4.0±0.01ms ? ?/sec
physical_select_aggregates_from_200 1.00 16.8±0.10ms ? ?/sec 1.02 17.1±0.10ms ? ?/sec
physical_select_all_from_1000 1.00 23.1±0.08ms ? ?/sec 1.04 24.1±0.13ms ? ?/sec
physical_select_one_from_700 1.00 1085.0±6.64µs ? ?/sec 1.00 1086.7±6.23µs ? ?/sec
physical_sorted_union_order_by_10_int64 1.00 5.9±0.02ms ? ?/sec 1.03 6.0±0.03ms ? ?/sec
physical_sorted_union_order_by_10_uint64 1.00 12.5±0.08ms ? ?/sec 1.03 12.9±0.07ms ? ?/sec
physical_sorted_union_order_by_50_int64 1.00 160.3±1.01ms ? ?/sec 1.02 163.5±1.18ms ? ?/sec
physical_sorted_union_order_by_50_uint64 1.00 367.0±2.38ms ? ?/sec 1.03 378.3±2.47ms ? ?/sec
physical_theta_join_consider_sort 1.00 1751.7±7.49µs ? ?/sec 1.01 1764.4±8.06µs ? ?/sec
physical_unnest_to_join 1.00 1803.4±7.06µs ? ?/sec 1.03 1863.4±8.97µs ? ?/sec
physical_window_function_partition_by_12_on_values 1.00 1075.3±13.64µs ? ?/sec 1.00 1080.5±5.68µs ? ?/sec
physical_window_function_partition_by_30_on_values 1.00 2.2±0.01ms ? ?/sec 1.01 2.2±0.01ms ? ?/sec
physical_window_function_partition_by_4_on_values 1.00 649.8±3.57µs ? ?/sec 1.01 653.5±5.74µs ? ?/sec
physical_window_function_partition_by_7_on_values 1.00 807.4±5.12µs ? ?/sec 1.00 808.4±6.38µs ? ?/sec
physical_window_function_partition_by_8_on_values 1.00 862.0±5.67µs ? ?/sec 1.01 867.7±5.33µs ? ?/sec
with_param_values_many_columns 1.00 127.1±0.78µs ? ?/sec 1.01 128.1±2.70µs ? ?/sec

alamb · 2025-11-12T13:30:17Z

 }

 impl FieldExt for Field {
+ fn renamed(self, new_name: &str) -> Self {


I added several new methods to avoid cloning when it is unnecesary

alamb · 2025-11-12T13:32:18Z

this is called as part of Expr::nullable and Expr::nullable so it is a very hot path on planning

…elds

alamb · 2025-11-25T20:08:11Z

@Jefffrey do you happen to have time to review this PR?

…elds

Jefffrey

Makes sense to me 👍

Jefffrey · 2025-11-25T23:58:05Z

@@ -353,13 +353,13 @@ impl DFSchema {

 /// Returns an immutable reference of a specific `Field` instance selected using an


Perhaps need to update some of these method docstrings since they assume returning a Field

Jefffrey · 2025-11-26T00:16:22Z

+ /// let renamed_field = int_field.renamed("your_int");
+ /// assert_eq!(renamed_field.name(), "your_int");
+ /// ```
+ fn renamed(self, new_name: &str) -> Self;


I wonder for these new methods if there is existing documentation elsewhere we'd need to upgrade to make these new methods more visible? i.e. "use these new methods if possible", though not sure how relevant might be considering these mainly replace the old as_ref from Arc -> clone -> modify field -> wrap in new Arc loop 🤔

it is a good idea -- I will review the existing docs and try to make it more obvious / clear how to use the new methdods

I tried to update the documentation with some more details on how to discover the new traits

…elds

Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>

…on into alamb/stop_cloning_fields

xudong963

Thanks for the effort @alamb, our team is also interested in the planner performance!

alamb · 2025-12-01T12:21:10Z

Thanks for the effort @alamb, our team is also interested in the planner performance!

Thanks @xudong963 -- the good news is I think there is some more significant improvement we could make (by not copying strings so much). I'll try and explore it when I have more time

alamb · 2025-12-01T14:06:09Z

run benchmarks sql_planner

alamb · 2025-12-01T14:06:12Z

🤖 Hi @alamb, thanks for the request (#18415 (comment)).

scrape_comments.py only supports whitelisted benchmarks.

Standard: clickbench_1, clickbench_extended, clickbench_partitioned, clickbench_pushdown, tpch, tpch10, tpch_mem, tpch_mem10
Criterion: sql_planner

Please choose one or more of these with run benchmark <name> or run benchmark <name1> <name2>...

alamb · 2025-12-01T14:06:23Z

run benchmark sql_planner

alamb · 2025-12-01T14:06:33Z

show benchmark queue

alamb · 2025-12-01T14:06:35Z

🤖 Hi @alamb, you asked to view the benchmark queue (#18415 (comment)).

Job	User	Benchmarks	Comment
`18415_3596731381.sh`	alamb	sql_planner	#18415 (comment)
`18572_3596717224.sh`	Dandandan	tpch_mem clickbench_partitioned	#18572 (comment)
`18572_3596718568.sh`	Dandandan	tpch	#18572 (comment)
`19018_3596651489.sh`	rluvaton	default	#19018 (comment)

alamb · 2025-12-01T15:36:51Z

🤖 ./gh_compare_branch_bench.shBenchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/stop_cloning_fields (fbbc042) to a060739diff
BENCH_NAME=sql_planner
BENCH_COMMAND=cargo bench --bench sql_planner
BENCH_FILTER=
BENCH_BRANCH_NAME=alamb_stop_cloning_fields
Results will be posted here when complete

alamb · 2025-12-01T16:46:22Z

🤖: Benchmark completed

Details

group alamb_stop_cloning_fields main
----- ------------------------- ----
logical_aggregate_with_join 1.00 649.0±16.96µs ? ?/sec 1.00 651.7±15.19µs ? ?/sec
logical_select_all_from_1000 1.00 10.9±0.39ms ? ?/sec 1.05 11.5±0.13ms ? ?/sec
logical_select_one_from_700 1.00 423.4±6.78µs ? ?/sec 1.00 425.2±9.59µs ? ?/sec
logical_trivial_join_high_numbered_columns 1.00 379.7±5.15µs ? ?/sec 1.01 381.6±5.29µs ? ?/sec
logical_trivial_join_low_numbered_columns 1.00 365.1±10.13µs ? ?/sec 1.02 370.7±15.12µs ? ?/sec
physical_intersection 1.00 847.2±10.54µs ? ?/sec 1.01 851.8±10.78µs ? ?/sec
physical_join_consider_sort 1.00 1402.8±33.95µs ? ?/sec 1.01 1423.6±31.17µs ? ?/sec
physical_join_distinct 1.00 355.5±4.09µs ? ?/sec 1.01 358.5±11.08µs ? ?/sec
physical_many_self_joins 1.00 9.9±0.19ms ? ?/sec 1.00 9.8±0.25ms ? ?/sec
physical_plan_clickbench_all 1.00 186.1±2.69ms ? ?/sec 1.01 188.5±3.32ms ? ?/sec
physical_plan_clickbench_q1 1.00 2.4±0.04ms ? ?/sec 1.01 2.4±0.05ms ? ?/sec
physical_plan_clickbench_q10 1.00 3.3±0.05ms ? ?/sec 1.01 3.3±0.06ms ? ?/sec
physical_plan_clickbench_q11 1.00 3.5±0.07ms ? ?/sec 1.03 3.6±0.17ms ? ?/sec
physical_plan_clickbench_q12 1.00 3.6±0.07ms ? ?/sec 1.00 3.6±0.07ms ? ?/sec
physical_plan_clickbench_q13 1.00 3.2±0.04ms ? ?/sec 1.01 3.2±0.06ms ? ?/sec
physical_plan_clickbench_q14 1.00 3.4±0.05ms ? ?/sec 1.01 3.5±0.05ms ? ?/sec
physical_plan_clickbench_q15 1.00 3.3±0.05ms ? ?/sec 1.02 3.4±0.08ms ? ?/sec
physical_plan_clickbench_q16 1.00 3.2±0.06ms ? ?/sec 1.01 3.2±0.07ms ? ?/sec
physical_plan_clickbench_q17 1.00 3.3±0.05ms ? ?/sec 1.01 3.3±0.10ms ? ?/sec
physical_plan_clickbench_q18 1.00 2.8±0.15ms ? ?/sec 1.00 2.8±0.08ms ? ?/sec
physical_plan_clickbench_q19 1.00 3.6±0.06ms ? ?/sec 1.01 3.7±0.05ms ? ?/sec
physical_plan_clickbench_q2 1.00 2.8±0.05ms ? ?/sec 1.02 2.8±0.06ms ? ?/sec
physical_plan_clickbench_q20 1.00 2.5±0.04ms ? ?/sec 1.02 2.6±0.05ms ? ?/sec
physical_plan_clickbench_q21 1.00 2.8±0.09ms ? ?/sec 1.01 2.9±0.08ms ? ?/sec
physical_plan_clickbench_q22 1.00 3.4±0.05ms ? ?/sec 1.02 3.5±0.07ms ? ?/sec
physical_plan_clickbench_q23 1.00 3.7±0.08ms ? ?/sec 1.00 3.7±0.09ms ? ?/sec
physical_plan_clickbench_q24 1.00 4.6±0.17ms ? ?/sec 1.03 4.7±0.14ms ? ?/sec
physical_plan_clickbench_q25 1.00 3.0±0.05ms ? ?/sec 1.01 3.0±0.05ms ? ?/sec
physical_plan_clickbench_q26 1.00 2.8±0.10ms ? ?/sec 1.00 2.8±0.08ms ? ?/sec
physical_plan_clickbench_q27 1.00 3.0±0.09ms ? ?/sec 1.02 3.0±0.07ms ? ?/sec
physical_plan_clickbench_q28 1.00 3.7±0.10ms ? ?/sec 1.02 3.8±0.09ms ? ?/sec
physical_plan_clickbench_q29 1.00 4.1±0.07ms ? ?/sec 1.01 4.1±0.11ms ? ?/sec
physical_plan_clickbench_q3 1.00 2.8±0.06ms ? ?/sec 1.02 2.8±0.06ms ? ?/sec
physical_plan_clickbench_q30 1.00 15.0±0.26ms ? ?/sec 1.02 15.2±0.26ms ? ?/sec
physical_plan_clickbench_q31 1.00 3.8±0.11ms ? ?/sec 1.01 3.8±0.06ms ? ?/sec
physical_plan_clickbench_q32 1.00 3.7±0.07ms ? ?/sec 1.01 3.8±0.06ms ? ?/sec
physical_plan_clickbench_q33 1.00 3.3±0.09ms ? ?/sec 1.02 3.4±0.11ms ? ?/sec
physical_plan_clickbench_q34 1.00 3.0±0.08ms ? ?/sec 1.02 3.0±0.06ms ? ?/sec
physical_plan_clickbench_q35 1.00 3.0±0.07ms ? ?/sec 1.01 3.1±0.07ms ? ?/sec
physical_plan_clickbench_q36 1.01 3.9±0.14ms ? ?/sec 1.00 3.8±0.10ms ? ?/sec
physical_plan_clickbench_q37 1.00 3.9±0.11ms ? ?/sec 1.01 4.0±0.09ms ? ?/sec
physical_plan_clickbench_q38 1.00 3.9±0.07ms ? ?/sec 1.00 3.9±0.11ms ? ?/sec
physical_plan_clickbench_q39 1.00 3.8±0.05ms ? ?/sec 1.01 3.8±0.06ms ? ?/sec
physical_plan_clickbench_q4 1.00 2.5±0.06ms ? ?/sec 1.00 2.5±0.06ms ? ?/sec
physical_plan_clickbench_q40 1.00 4.6±0.13ms ? ?/sec 1.01 4.6±0.07ms ? ?/sec
physical_plan_clickbench_q41 1.00 4.0±0.14ms ? ?/sec 1.02 4.0±0.07ms ? ?/sec
physical_plan_clickbench_q42 1.00 3.9±0.06ms ? ?/sec 1.01 4.0±0.06ms ? ?/sec
physical_plan_clickbench_q43 1.00 4.3±0.07ms ? ?/sec 1.02 4.4±0.07ms ? ?/sec
physical_plan_clickbench_q44 1.00 2.6±0.03ms ? ?/sec 1.01 2.7±0.05ms ? ?/sec
physical_plan_clickbench_q45 1.00 2.7±0.04ms ? ?/sec 1.01 2.7±0.08ms ? ?/sec
physical_plan_clickbench_q46 1.00 3.1±0.08ms ? ?/sec 1.01 3.1±0.08ms ? ?/sec
physical_plan_clickbench_q47 1.00 3.7±0.09ms ? ?/sec 1.01 3.7±0.06ms ? ?/sec
physical_plan_clickbench_q48 1.00 4.3±0.07ms ? ?/sec 1.02 4.4±0.11ms ? ?/sec
physical_plan_clickbench_q49 1.00 4.6±0.08ms ? ?/sec 1.01 4.6±0.16ms ? ?/sec
physical_plan_clickbench_q5 1.00 2.7±0.08ms ? ?/sec 1.00 2.8±0.08ms ? ?/sec
physical_plan_clickbench_q50 1.00 4.2±0.07ms ? ?/sec 1.01 4.3±0.11ms ? ?/sec
physical_plan_clickbench_q51 1.00 3.3±0.08ms ? ?/sec 1.00 3.3±0.08ms ? ?/sec
physical_plan_clickbench_q6 1.00 2.7±0.07ms ? ?/sec 1.01 2.7±0.04ms ? ?/sec
physical_plan_clickbench_q7 1.00 2.5±0.07ms ? ?/sec 1.01 2.5±0.04ms ? ?/sec
physical_plan_clickbench_q8 1.00 3.3±0.04ms ? ?/sec 1.01 3.3±0.05ms ? ?/sec
physical_plan_clickbench_q9 1.00 3.2±0.05ms ? ?/sec 1.00 3.2±0.10ms ? ?/sec
physical_plan_tpcds_all 1.00 1057.5±7.03ms ? ?/sec 1.01 1070.9±6.97ms ? ?/sec
physical_plan_tpch_all 1.00 64.0±1.00ms ? ?/sec 1.02 65.3±0.72ms ? ?/sec
physical_plan_tpch_q1 1.00 2.1±0.03ms ? ?/sec 1.02 2.2±0.03ms ? ?/sec
physical_plan_tpch_q10 1.00 3.9±0.08ms ? ?/sec 1.03 4.0±0.12ms ? ?/sec
physical_plan_tpch_q11 1.00 3.6±0.08ms ? ?/sec 1.02 3.6±0.10ms ? ?/sec
physical_plan_tpch_q12 1.00 1835.2±15.01µs ? ?/sec 1.02 1864.5±24.20µs ? ?/sec
physical_plan_tpch_q13 1.00 1436.4±10.93µs ? ?/sec 1.02 1460.8±15.75µs ? ?/sec
physical_plan_tpch_q14 1.00 2.1±0.04ms ? ?/sec 1.00 2.1±0.02ms ? ?/sec
physical_plan_tpch_q16 1.00 2.5±0.08ms ? ?/sec 1.00 2.4±0.04ms ? ?/sec
physical_plan_tpch_q17 1.00 2.6±0.05ms ? ?/sec 1.01 2.7±0.02ms ? ?/sec
physical_plan_tpch_q18 1.00 2.7±0.04ms ? ?/sec 1.01 2.7±0.04ms ? ?/sec
physical_plan_tpch_q19 1.00 3.2±0.04ms ? ?/sec 1.00 3.2±0.04ms ? ?/sec
physical_plan_tpch_q2 1.00 5.8±0.18ms ? ?/sec 1.02 5.9±0.06ms ? ?/sec
physical_plan_tpch_q20 1.00 3.2±0.09ms ? ?/sec 1.01 3.2±0.07ms ? ?/sec
physical_plan_tpch_q21 1.00 4.1±0.08ms ? ?/sec 1.01 4.2±0.05ms ? ?/sec
physical_plan_tpch_q22 1.00 2.8±0.04ms ? ?/sec 1.01 2.9±0.05ms ? ?/sec
physical_plan_tpch_q3 1.00 2.7±0.05ms ? ?/sec 1.01 2.7±0.03ms ? ?/sec
physical_plan_tpch_q4 1.00 1472.4±20.65µs ? ?/sec 1.01 1486.8±11.12µs ? ?/sec
physical_plan_tpch_q5 1.00 3.3±0.05ms ? ?/sec 1.01 3.3±0.10ms ? ?/sec
physical_plan_tpch_q6 1.00 889.9±11.20µs ? ?/sec 1.00 893.3±12.31µs ? ?/sec
physical_plan_tpch_q7 1.00 4.2±0.07ms ? ?/sec 1.01 4.2±0.03ms ? ?/sec
physical_plan_tpch_q8 1.00 5.5±0.06ms ? ?/sec 1.02 5.6±0.07ms ? ?/sec
physical_plan_tpch_q9 1.00 4.0±0.09ms ? ?/sec 1.02 4.1±0.03ms ? ?/sec
physical_select_aggregates_from_200 1.00 17.0±0.22ms ? ?/sec 1.01 17.2±0.42ms ? ?/sec
physical_select_all_from_1000 1.00 24.1±0.43ms ? ?/sec 1.04 25.0±0.33ms ? ?/sec
physical_select_one_from_700 1.00 1098.4±26.63µs ? ?/sec 1.00 1103.7±15.74µs ? ?/sec
physical_sorted_union_order_by_10_int64 1.00 5.9±0.12ms ? ?/sec 1.02 6.0±0.14ms ? ?/sec
physical_sorted_union_order_by_10_uint64 1.00 15.0±0.35ms ? ?/sec 1.04 15.6±0.32ms ? ?/sec
physical_sorted_union_order_by_50_int64 1.00 160.7±2.66ms ? ?/sec 1.01 162.6±1.85ms ? ?/sec
physical_sorted_union_order_by_50_uint64 1.00 848.2±8.83ms ? ?/sec 1.12 952.1±9.41ms ? ?/sec
physical_theta_join_consider_sort 1.00 1766.8±21.17µs ? ?/sec 1.01 1776.8±25.12µs ? ?/sec
physical_unnest_to_join 1.00 1827.9±26.88µs ? ?/sec 1.03 1879.2±35.43µs ? ?/sec
physical_window_function_partition_by_12_on_values 1.00 1097.7±12.92µs ? ?/sec 1.01 1114.0±11.41µs ? ?/sec
physical_window_function_partition_by_30_on_values 1.00 2.2±0.02ms ? ?/sec 1.02 2.3±0.03ms ? ?/sec
physical_window_function_partition_by_4_on_values 1.00 678.4±5.95µs ? ?/sec 1.00 678.9±13.10µs ? ?/sec
physical_window_function_partition_by_7_on_values 1.01 834.7±18.34µs ? ?/sec 1.00 826.0±5.46µs ? ?/sec
physical_window_function_partition_by_8_on_values 1.00 893.3±11.14µs ? ?/sec 1.00 893.4±26.73µs ? ?/sec
with_param_values_many_columns 1.00 598.3±12.38µs ? ?/sec 1.12 667.5±11.50µs ? ?/sec

alamb · 2025-12-01T17:21:13Z

Thanks again @xudong963 and @Jefffrey

alamb changed the title ~~Alamb/stop cloning fields~~Optimize planning / stop cloning Strings / Fields so muchOct 31, 2025

github-actionsBot added sql SQL Planner logical-expr Logical plan and expressions core Core DataFusion crate substrait Changes to the substrait crate common Related to common crate labels Oct 31, 2025

alamb commented Oct 31, 2025
View reviewed changes

alamb mentioned this pull request Oct 31, 2025
Alamb/even less copy #18416
Closed

alamb commented Oct 31, 2025
View reviewed changes

alamb force-pushed the alamb/stop_cloning_fields branch from 074e485 to 879d1bfCompare November 11, 2025 21:19

github-actionsBot added the documentation Improvements or additions to documentation label Nov 11, 2025

Optimize planning / stop cloning Strings / Fields so much
9dcfdfc

alamb force-pushed the alamb/stop_cloning_fields branch from 879d1bf to 9dcfdfcCompare November 11, 2025 22:09

alamb commented Nov 11, 2025
View reviewed changes

alamb added the performance Make DataFusion faster label Nov 12, 2025

alamb changed the title ~~Optimize planning / stop cloning Strings / Fields so much~~Optimize planning / stop cloning Strings / Fields so much (2-3% faster planning time)Nov 12, 2025

alamb marked this pull request as ready for review November 12, 2025 13:29

alamb added the api change Changes the API exposed to users of the crate label Nov 12, 2025

alamb commented Nov 12, 2025
View reviewed changes

alamb mentioned this pull request Nov 20, 2025
Cache the result of Expr::data_type / Expr::nullable / Expr::to_field to speed up planning #18845
Open

Merge remote-tracking branch 'apache/main' into alamb/stop_cloning_fi…
1a1f546
…elds

alamb requested a review from Jefffrey November 25, 2025 20:07

Merge remote-tracking branch 'apache/main' into alamb/stop_cloning_fi…
91a5764
…elds

Jefffrey approved these changes Nov 26, 2025
View reviewed changes

alamband others added 7 commits November 26, 2025 13:39

Merge remote-tracking branch 'apache/main' into alamb/stop_cloning_fi…
448cf68
…elds

Add references to DataTypeExt and FieldExt in docs
21a7378

Update documentation
0958125

Update docs/source/library-user-guide/upgrading.md
2a49626
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>

more docs
0cb8e41

Merge branch 'alamb/stop_cloning_fields' of github.com:alamb/datafusi…
412edb8
…on into alamb/stop_cloning_fields

Merge branch 'main' into alamb/stop_cloning_fields
5e1925b

xudong963 approved these changes Dec 1, 2025
View reviewed changes

Merge branch 'main' into alamb/stop_cloning_fields
fbbc042

alamb mentioned this pull request Dec 1, 2025
TEST PR #19000
Closed

alamb mentioned this pull request Dec 1, 2025
TEST: enable pushdown_filters and reorder_filters by default #18873
Closed

alamb added this pull request to the merge queue Dec 1, 2025

Merged via the queue into apache:main with commit 477053dDec 1, 2025
32 checks passed

alamb-ghbot mentioned this pull request Dec 4, 2025
Some panic!s could more semantically be unimplemented! apache/arrow-rs#8933
Merged

		@@ -353,13 +353,13 @@ impl DFSchema {

		/// Returns an immutable reference of a specific `Field` instance selected using an

Uh oh!

Conversation

alamb commented Oct 31, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alambOct 31, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Oct 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Oct 31, 2025

Uh oh!

alamb commented Oct 31, 2025

Uh oh!

alamb commented Nov 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Nov 11, 2025

Uh oh!

alamb commented Nov 12, 2025

Uh oh!

alamb commented Nov 12, 2025

Uh oh!

alamb commented Nov 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Nov 25, 2025

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xudong963 left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

alamb commented Dec 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

alamb commented Oct 31, 2025•
edited
Loading

alambOct 31, 2025•
edited
Loading