Uh oh!
There was an error while loading. Please reload this page.
acc: always run acceptance tests locally - #6196
Merged
Merged
Conversation
Every acceptance test already runs against the testserver, so the Local test.toml knob only existed to opt out. Drop it: local runs include all tests by default, and DATABRICKS_TEST_SKIPLOCAL still skips them on cloud PR/integration runs.
radakam
marked this pull request as ready for review
August 7, 2026 08:42
These only restated the root default after Local was removed, so delete them.
eng-dev-ecosystem-bot
commented
Aug 7, 2026
Collaborator
Integration test reportCommit: 9a6b40a
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 8 slowest tests (at least 2 minutes):
|
denik
commented
Aug 7, 2026
Contributor
This does not need to be in PR description, it's already seen in the diff. |
denik
reviewed
Aug 7, 2026
| const ( | ||
| SkipLocalEnvVar = "DATABRICKS_TEST_SKIPLOCAL" | ||
| // SkipLocalAll skips every test with Local = true. | ||
| // SkipLocalAll skips every acceptance test. |
Contributor
There was a problem hiding this comment.
We should probably delete this?
~/work/cli % git grep -B 5 DATABRICKS_TEST_SKIPLOCAL=true
Taskfile.yml- test-update-aws:
Taskfile.yml- desc: Update acceptance test output (integration, requires deco access)
Taskfile.yml- sources: *ACC_SOURCES_UPDATE
Taskfile.yml- generates: *ACC_GENERATES_UPDATE
Taskfile.yml- cmds:
Taskfile.yml: - "deco env run -i -n aws-prod-ucws -- env DATABRICKS_TEST_SKIPLOCAL=true go test ./acceptance -run ^TestAccept$ -update -timeout=1h -v"
and we should not support DATABRICKS_TEST_SKIPLOCAL=true anymore, right?
Contributor
There was a problem hiding this comment.
This could be a follow up PR, given that it would be nice to merge this one quickly to minimize conflicts.
denik
approved these changes
Aug 7, 2026
Contributor
Approval status: pending
|
Uh oh!
There was an error while loading. Please reload this page.
radakam added a commit
that referenced
this pull request
Aug 7, 2026
The Local knob was removed in #6196; regenerate the two out.test.toml files this branch adds so they match the current snapshot format.
radakam added a commit
that referenced
this pull request
Aug 7, 2026
Local is no longer a test.toml key; keeping it fails config loading.
Sankalp-Mittal added a commit
that referenced
this pull request
Aug 7, 2026
#6196 removed the Local config field and rewrote the other test.toml files to drop it, but #5587 added this test.toml with Local = true around the same time, so it was not swept. The acceptance harness rejects undecoded keys, failing the test on config parse before any assertion runs. Remove the key. Co-authored-by: Isaac
Sankalp-Mittal added a commit
that referenced
this pull request
Aug 7, 2026
#6196 removed the Local config field and regenerated every out.test.toml snapshot, but this test was added on this branch in parallel, so its committed snapshot still carried Local = true. The post-test "no files changed" check regenerates it and fails on the diff. Regenerate to match. Co-authored-by: Isaac
eng-dev-ecosystem-bot
commented
Aug 7, 2026
Collaborator
Integration test reportCommit: aebca14
19 interesting tests: 6 FAIL, 6 flaky, 4 RECOVERED, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
eng-dev-ecosystem-bot
commented
Aug 9, 2026
Collaborator
Integration test reportCommit: aebca14
17 interesting tests: 7 RECOVERED, 6 FAIL, 2 flaky, 1 KNOWN, 1 SKIP
Top 50 slowest tests (at least 2 minutes):
|
radakam added a commit
that referenced
this pull request
Aug 10, 2026
Local is no longer a test.toml key; keeping it fails config loading.
yolocs pushed a commit
to yolocs/dbcli
that referenced
this pull request
Aug 11, 2026
## Changes Stop supporting `DATABRICKS_TEST_SKIPLOCAL=true` (which now skips every acceptance test) and delete the `test-update-aws` Taskfile task that used it. `DATABRICKS_TEST_SKIPLOCAL=withchanged` remains for PR cloud runs. ## Why Follow-up to databricks#6196. After dropping the `Local` knob, `=true` no longer means "skip tests with testserver coverage" — it skips everything — so the aws update path was a no-op.
yolocs pushed a commit
to yolocs/dbcli
that referenced
this pull request
Aug 11, 2026
…cks#6232) ## Summary Drop stale `Local = true` left by databricks#5846 after databricks#6196 removed the field from `TestConfig`. Unblocks `task test-pipelines` on main.
Sankalp-Mittal added a commit
that referenced
this pull request
Aug 12, 2026
#6196 removed the Local config field and regenerated every out.test.toml snapshot, but this test was added on this branch in parallel, so its committed snapshot still carried Local = true. The post-test "no files changed" check regenerates it and fails on the diff. Regenerate to match. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Drop the
Localtest.tomlknob. Every acceptance test already runs against the testserver, so local runs include all tests by default;Cloud/CloudSlowstill opt into real-workspace runs.DATABRICKS_TEST_SKIPLOCALkeeps skipping acceptance tests on cloud PR/integration runs (withwithchangedre-enabling touched tests).Most of the diff is mechanical
Local = trueremovals fromtest.toml/out.test.toml.After this merges, any
test.tomlthat still setsLocalwill fail config loading with an undecoded-key error. Drop theLocalline (tests always run locally now).Why
There are now no remaining
Local = falsetests. Keeping an opt-out that nobody uses invites new cloud-only gaps and forces everytest.toml/out.test.tomlto repeatLocal = true.