[C++][Dataset] Ignored failures in dataset-scanner-benchmark

[pitrou]

Describe the bug, including details regarding any error messages, version, and platform.

When running arrow-dataset-scanner-benchmark locally I get the following failures. They are printed but just ignored afterwards:

[...]
ScanOnlyBench/num_batches:1000/batch_size:10/scan_alg:0/real_time 24661324 ns 2970995 ns 6 bytes_per_second=1.93354M/s items_per_second=40.5493k/s
ScanOnlyBench/num_batches:1000/batch_size:10/scan_alg:1/real_time 15358689 ns 135295 ns 8 bytes_per_second=3.10467M/s items_per_second=65.1097k/s
/home/antoine/arrow/dev/cpp/src/arrow/dataset/scanner_benchmark.cc:165: Failure
Expected equality of these values:
 collected->num_columns()
 Which is: 6
 2

[etc.]

Component(s)

Benchmarking, C++

[pitrou]

cc @westonpace

[Cerdore]

Can I try to fix this bug?
Looks like there's a problem with the scan parameters.

[kou]

Sure!

[Cerdore]

The extra 4 columns returned in the benchmark are:

const FieldVector kAugmentedFields{
 field("__fragment_index", int32()),
 field("__batch_index", int32()),
 field("__last_in_fragment", boolean()),
 field("__filename", utf8()),
};

In the failure case, 6 columns are returned instead of the expected 2 columns. This is because kScanFactory is used as scan options, which triggers the execution of the MakeScanNode function. In this function, additional columns shown above are added.

I also examined the TEST_F(TestReordering, ScanBatches) test case, which also invokes the MakeScanNode function. In this test, extra columns are also added. And the two case use the similar scan option.

Related PR:
If a projected_schema is not supplied ..
Implement ability to retrieve fragment filename