GH-48068: [C++][FlightRPC] Linux ODBC: Configure Dremio instance to allow remote testing by alinaliBQ · Pull Request #49695 · apache/arrow

alinaliBQ · 2026-04-08T21:40:33Z

Rationale for this change

The goal is to run remote tests against the Dremio instance on Ubuntu-latest runner, as Dremio docker instances are only available on Linux and they are not supported on macOS/Windows GitHub Actions.

What changes are included in this PR?

Added dremio to compose.yaml.
Added logic for setting up dremio account and creating a table in cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh.
Add ODBC remote test connection variable ARROW_FLIGHT_SQL_ODBC_CONN.
set up Dremio remote instance in CI

Note: This PR prepares the Dremio instance for remote testing, which is activated using ARROW_FLIGHT_SQL_ODBC_CONN={..., HOST=dremio.

Are these changes tested?

Tested in CI at local repository.

Are there any user-facing changes?

N/A

GitHub Issue: [C++] [FlightRPC][SQL][ODBC] Set Up Linux ODBC Driver Tests Against Dremio Docker #48068

github-actions · 2026-04-08T21:41:28Z

⚠️ GitHub issue #48068has been automatically assigned in GitHub to PR creator.

alinaliBQ · 2026-04-09T16:24:11Z

@lidavidm @kou This PR is ready for review, please have a look

alinaliBQ

Addressed code review comments

alinaliBQ · 2026-04-22T17:10:19Z

@kou @lidavidm All comments are addressed, please have another look.
cc @justing-bq

justing-bq · 2026-05-21T20:50:37Z

@lidavidm @kou
Comments addressed. Please review.

Copilot

Pull request overview

This PR prepares the C++ Flight SQL ODBC CI environment to run remote tests against a Linux-only Dremio Docker instance by adding a Dremio service to the repo’s Docker Compose setup and automating initial Dremio bootstrap (user + test table) from CI.

Changes:

Added a Dremio bootstrap script to create an admin user, obtain an auth token, and create $scratch.ODBCTest.
Extended compose.yaml with a dremio service and injected an ODBC remote connection string into the ubuntu-cpp-odbc service.
Updated cpp_extra.yml to start Dremio and run the bootstrap script during the ODBC Linux job.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File	Description
`cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh`	New bootstrap script to provision Dremio for remote ODBC tests.
`compose.yaml`	Adds Dremio service + wires ODBC connection env var and dependency into the ODBC Docker job.
`.pre-commit-config.yaml`	Adds the new script to ShellCheck’s allowlist.
`.github/workflows/cpp_extra.yml`	Starts Dremio and runs the bootstrap script in the ODBC Linux workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

justing-bq · 2026-05-22T22:09:21Z

@kou @lidavidm
Comments addressed. Please review.

kou

Could you update the PR description?

Is this Dremio instance used in Linux tests? How to confirm it?

kou · 2026-04-30T11:33:53Z

+ environment:
+ - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-8 -Dsaffron.default.nationalcharset=UTF-8 -Dsaffron.default.collation.name=UTF-8$$en_US
+ healthcheck:
+ test: curl --fail http://localhost:9047 || exit 1


Why do we need || exit 1 here?

cc @justing-bq

I don't believe it is necessary so I removed it.

alinaliBQ · 2026-06-15T17:30:01Z

Could you update the PR description?
Is this Dremio instance used in Linux tests? How to confirm it?

I have updated the PR description to match the files updated.

The Dremio instance is used in the ODBC Linux tests through the environment variable ARROW_FLIGHT_SQL_ODBC_CONN. To confirm it, the ODBC test log should show FlightSQLODBCRemoteTestBase::* and FlightSQLOdbcV2RemoteTestBase::* tests.

These test suites exclusively connect to the remote instance indicated in ARROW_FLIGHT_SQL_ODBC_CONN, which in our case is the Dremio instance.

Authored by Justin Co-authored-by: justing-bq <justin.gossett@improving.com> ^ Conflicts: ^ .github/workflows/cpp_extra.yml

kou

+1

The Dremio instance is used in the ODBC Linux tests through the environment variable ARROW_FLIGHT_SQL_ODBC_CONN. To confirm it, the ODBC test log should show FlightSQLODBCRemoteTestBase::* and FlightSQLOdbcV2RemoteTestBase::* tests.
These test suites exclusively connect to the remote instance indicated in ARROW_FLIGHT_SQL_ODBC_CONN, which in our case is the Dremio instance.

Thanks. We can't confirm it in CI because our CI log doesn't show executed test names, right?

https://github.com/apache/arrow/actions/runs/27574204620/job/81518244076?pr=49695#step:7:10585

+ ctest --label-regex unittest --output-on-failure --parallel 4 --repeat until-pass:3 --timeout 300
Test project /build/cpp
Start 1: arrow-array-test
Start 2: arrow-buffer-test
Start 3: arrow-extension-type-test
Start 4: arrow-misc-test
1/58 Test #3: arrow-extension-type-test ............. Passed 0.09 sec
Start 5: arrow-public-api-test
2/58 Test #2: arrow-buffer-test ..................... Passed 0.11 sec
Start 6: arrow-scalar-test
3/58 Test #4: arrow-misc-test ....................... Passed 0.12 sec
Start 7: arrow-type-test
4/58 Test #5: arrow-public-api-test ................. Passed 0.09 sec
Start 8: arrow-table-test
5/58 Test #7: arrow-type-test ....................... Passed 0.11 sec
Start 9: arrow-tensor-test
31/58 Test #34: arrow-compute-kernel-utility-test ..... Passed 0.08 sec
Start 35: arrow-canonical-extensions-test
32/58 Test #35: arrow-canonical-extensions-test ....... Passed 0.09 sec
Start 36: arrow-io-buffered-test
33/58 Test #33: arrow-compute-aggregate-test .......... Passed 0.60 sec
Start 37: arrow-io-compressed-test
34/58 Test #36: arrow-io-buffered-test ................ Passed 0.10 sec
Start 38: arrow-io-file-test
35/58 Test #31: arrow-compute-vector-selection-test ... Passed 1.00 sec
Start 39: arrow-io-memory-test
36/58 Test #38: arrow-io-file-test .................... Passed 0.16 sec
Start 40: arrow-utility-test
37/58 Test #40: arrow-utility-test .................... Passed 0.82 sec
Start 41: arrow-async-utility-test
38/58 Test #37: arrow-io-compressed-test .............. Passed 1.27 sec
Start 42: arrow-bit-utility-test
39/58 Test #39: arrow-io-memory-test .................. Passed 1.40 sec
Start 43: arrow-crc32-test
40/58 Test #43: arrow-crc32-test ...................... Passed 0.14 sec
Start 44: arrow-threading-utility-test
41/58 Test #42: arrow-bit-utility-test ................ Passed 1.85 sec
Start 45: arrow-json-integration-test
42/58 Test #45: arrow-json-integration-test ........... Passed 0.28 sec
Start 46: arrow-filesystem-test
43/58 Test #46: arrow-filesystem-test ................. Passed 1.27 sec
Start 47: arrow-flight-internals-test
44/58 Test #47: arrow-flight-internals-test ........... Passed 0.07 sec
Start 48: arrow-flight-test
45/58 Test #1: arrow-array-test ...................... Passed 9.82 sec
Start 49: arrow-flight-sql-test
46/58 Test #49: arrow-flight-sql-test ................. Passed 0.17 sec
Start 50: arrow-odbc-spi-impl-test
47/58 Test #50: arrow-odbc-spi-impl-test .............. Passed 0.08 sec
Start 51: arrow-flight-sql-odbc-test
48/58 Test #48: arrow-flight-test ..................... Passed 1.26 sec
Start 52: arrow-feather-test
49/58 Test #44: arrow-threading-utility-test .......... Passed 4.60 sec
Start 53: arrow-ipc-endianness-test
50/58 Test #53: arrow-ipc-endianness-test ............. Passed 0.15 sec
Start 54: arrow-ipc-message-internal-test
51/58 Test #54: arrow-ipc-message-internal-test ....... Passed 0.11 sec
Start 55: arrow-ipc-read-write-test
52/58 Test #52: arrow-feather-test .................... Passed 0.47 sec
Start 56: arrow-ipc-tensor-test
53/58 Test #56: arrow-ipc-tensor-test ................. Passed 0.10 sec
Start 57: arrow-json-test
54/58 Test #57: arrow-json-test ....................... Passed 0.35 sec
Start 58: arrow-telemetry-test
55/58 Test #41: arrow-async-utility-test .............. Passed 6.00 sec
56/58 Test #58: arrow-telemetry-test .................. Passed 0.11 sec
57/58 Test #55: arrow-ipc-read-write-test ............. Passed 2.16 sec
58/58 Test #51: arrow-flight-sql-odbc-test ............ Passed 9.48 sec
100% tests passed, 0 tests failed out of 58

conbench-apache-arrow · 2026-06-16T07:57:20Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit cb625c5.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

alinaliBQ · 2026-06-16T17:28:47Z

Yes, that's right. Since enabling tests against the Dremio instance doubles the number of test cases being run, we can perhaps check by the amount of time it takes to run the arrow-flight-sql-odbc-test, which now takes longer as the ODBC driver establishes connection to the Dremio instance.

Before Dremio instance is enabled:
https://github.com/apache/arrow/actions/runs/27562488300/job/81477934843#step:6:15521

 Start 51: arrow-flight-sql-odbc-test
48/58 Test #51: arrow-flight-sql-odbc-test ............ Passed 0.29 sec

After Dremio instance is enabled shows ~9 sec.

github-actionsBot added Component: C++ Component: FlightRPC CI: Extra: C++ Run extra C++ CI awaiting review Awaiting review labels Apr 8, 2026

alinaliBQ marked this pull request as ready for review April 9, 2026 16:24

alinaliBQ requested review from assignUser, jonkeane, kou, lidavidm and raulcd as code owners April 9, 2026 16:24

lidavidm reviewed Apr 9, 2026
View reviewed changes

Comment threadcpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml

github-actionsBot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Apr 9, 2026

kou reviewed Apr 10, 2026
View reviewed changes

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 10, 2026

alinaliBQ force-pushed the gh-48068-dremio-docker-test branch 3 times, most recently from 7b6d340 to 4a892cfCompare April 10, 2026 21:20

alinaliBQ commented Apr 10, 2026
View reviewed changes

alinaliBQ requested review from kou and lidavidm April 17, 2026 17:08

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 4a892cf to 3cf2376Compare April 22, 2026 17:18

kou reviewed Apr 27, 2026
View reviewed changes

Comment threadcompose.yaml Outdated
Comment threadcompose.yaml Outdated
Comment threadcompose.yaml Outdated
Comment threadcpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh

github-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Apr 27, 2026

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 3cf2376 to 3d18ebdCompare April 27, 2026 23:44

justing-bq force-pushed the gh-48068-dremio-docker-test branch 2 times, most recently from 9ca9402 to fa7ab48Compare May 21, 2026 16:34

kou requested a review from Copilot May 22, 2026 02:56

Copilotstarted reviewing on behalf of kou May 22, 2026 02:56 View session

CopilotAI reviewed May 22, 2026
View reviewed changes

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 70abfae to 35c324fCompare May 22, 2026 19:39

alinaliBQ mentioned this pull request May 25, 2026
GH-49482: [C++][FlightRPC][ODBC] Fix inconsistent SQLGetInfo values in global connection #50021
Merged

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 35c324f to ad2866fCompare May 25, 2026 21:51

justing-bq force-pushed the gh-48068-dremio-docker-test branch 2 times, most recently from 91b8fa6 to 01d5339Compare June 11, 2026 17:31

alinaliBQ requested a review from kou June 11, 2026 17:32

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 01d5339 to 1e29550Compare June 12, 2026 16:18

kou reviewed Jun 14, 2026
View reviewed changes

github-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jun 14, 2026

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jun 15, 2026

alinaliBQand others added 2 commits June 15, 2026 13:26

Configure Dremio instance to allow remote testing
f9d0e36
Authored by Justin Co-authored-by: justing-bq <justin.gossett@improving.com> ^ Conflicts: ^ .github/workflows/cpp_extra.yml

Address review feedback
cc3b4fc

justing-bq force-pushed the gh-48068-dremio-docker-test branch from e6605f1 to cc3b4fcCompare June 15, 2026 20:26

kou approved these changes Jun 16, 2026
View reviewed changes

kou merged commit cb625c5 into apache:mainJun 16, 2026
62 of 64 checks passed

kou removed the awaiting change review Awaiting change review label Jun 16, 2026

kou mentioned this pull request Jun 16, 2026
[C++] [FlightRPC][SQL][ODBC] Set Up Linux ODBC Driver Tests Against Dremio Docker #48068
Closed

github-actionsBot added the awaiting merge Awaiting merge label Jun 16, 2026

Uh oh!

Conversation

alinaliBQ commented Apr 8, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented Apr 8, 2026

Uh oh!

alinaliBQ commented Apr 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alinaliBQ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alinaliBQ commented Apr 22, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

justing-bq commented May 21, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

justing-bq commented May 22, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

kouApr 30, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQJun 15, 2026

Choose a reason for hiding this comment

Uh oh!

justing-bqJun 15, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQ commented Jun 15, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

conbench-apache-arrowBot commented Jun 16, 2026

Uh oh!

alinaliBQ commented Jun 16, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

alinaliBQ commented Apr 8, 2026•
edited
Loading

alinaliBQ commented Apr 22, 2026•
edited
Loading

alinaliBQ commented Jun 16, 2026•
edited
Loading