Skip to content

Introduce retries to databricks psql command - #3492

Merged
anton-107 merged 13 commits into
mainfrom
anton-107/psql-retries
Aug 29, 2025
Merged

Introduce retries to databricks psql command #3492
anton-107 merged 13 commits into
mainfrom
anton-107/psql-retries

Conversation

@anton-107

@anton-107anton-107 commented Aug 26, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Why

Sometimes the connection to the database instance might fail, which can be remediated with a retry. This change makes the command to retry the connection 3 times by default.

To disable the retries users can set a max-retries value to zero:

$ databricks psql my-instance --max-retries 0

Tests

Added a new acceptance test, modified and existing acceptance test; ran manual tests on Mac and Windows

@anton-107
anton-107 marked this pull request as ready for review August 26, 2025 14:18
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Aug 26, 2025•
edited
Loading

Copy link
Copy Markdown
Collaborator

Run: 17299828292

Env✅​pass🔄​flaky🙈​skip
🔄​aws linux2999504
🔄​aws windows3036503
✅​aws-ucws linux421401
✅​aws-ucws windows422400
🔄​azure linux3026503
🔄​azure windows3054502
🔄​azure-ucws linux4138400
✅​azure-ucws windows422399
🔄​gcp linux2989505
🔄​gcp windows3035504
36 failing tests:
Test Nameaws linuxaws windowsazure linuxazure windowsazure-ucws linuxgcp linuxgcp windows
TestAccept✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass
TestAccept/bundle/deploy/dashboard/simple✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky
TestAccept/bundle/deploy/pipeline/allow-duplicate-names✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/deploy/pipeline/allow-duplicate-names/DATABRICKS_CLI_DEPLOYMENT=terraform✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/deployment/bind/job/job-spark-python-task✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky
TestAccept/bundle/destroy/jobs-and-pipeline✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_CLI_DEPLOYMENT=terraform✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/resources/pipelines/update✅​pass✅​pass🔄​flaky✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/resources/pipelines/update/DATABRICKS_CLI_DEPLOYMENT=direct-exp🔄​flaky🔄​flaky✅​pass✅​pass✅​pass✅​pass🔄​flaky
TestAccept/bundle/resources/pipelines/update/DATABRICKS_CLI_DEPLOYMENT=terraform🔄​flaky🔄​flaky🔄​flaky✅​pass✅​pass🔄​flaky🔄​flaky
TestAccept/bundle/templates/default-python/combinations/classic🔄​flaky✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=yes🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=yes✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/templates/default-python/integration_classic✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.10✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.11🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.9✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.10✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.11🔄​flaky✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.12🔄​flaky🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.9✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass
TestAccept/selftest/record_cloud/pipeline-crud🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_CLI_DEPLOYMENT=direct-exp🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass
TestAccept/selftest/record_cloud/workspace-file-io✅​pass🔄​flaky✅​pass🔄​flaky✅​pass✅​pass✅​pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=direct-exp✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=terraform✅​pass🔄​flaky✅​pass✅​pass✅​pass✅​pass✅​pass
TestFsLsForNonexistingDir✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass
TestFsLsForNonexistingDir/uc-volumes🙈​skip🙈​skip🙈​skip🙈​skip🔄​flaky🙈​skip🙈​skip
TestFsLsOnEmptyDir✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass
TestFsLsOnEmptyDir/uc-volumes🙈​skip🙈​skip🙈​skip🙈​skip🔄​flaky🙈​skip🙈​skip
TestFsLsOnFile✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass
TestFsLsOnFile/uc-volumes🙈​skip🙈​skip🙈​skip🙈​skip🔄​flaky🙈​skip🙈​skip
TestFsLsWithAbsolutePaths✅​pass✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass
TestFsLsWithAbsolutePaths/uc-volumes🙈​skip🙈​skip🙈​skip🙈​skip🔄​flaky🙈​skip🙈​skip
TestGenerateFromExistingPipelineAndDeploy✅​pass✅​pass✅​pass🔄​flaky✅​pass✅​pass✅​pass
TestLockUnlockWithoutAllowsLockFileNotExist✅​pass✅​pass✅​pass✅​pass✅​pass✅​pass🔄​flaky

denik
denik reviewed Aug 26, 2025
View reviewed changes
Comment threadcmd/psql/psql.go Outdated
Comment threadlibs/lakebase/connect.go Outdated
Comment threadlibs/lakebase/connect.go Outdated
@anton-107
anton-107force-pushed the anton-107/psql-retries branch from 0879a63 to 7dbc8baCompareAugust 27, 2025 09:44
@anton-107
anton-107 requested a review from denikAugust 27, 2025 09:44
denik
denik reviewed Aug 27, 2025
View reviewed changes
Comment threadlibs/lakebase/connect.go Outdated
Comment threadlibs/lakebase/connect_test.go Outdated
Comment threadacceptance/cmd/psql/failing-connection/script
Comment threadlibs/lakebase/connect.go Outdated
Comment threadlibs/lakebase/connect.go
Comment threadlibs/lakebase/connect.go Outdated
@anton-107
anton-107 requested a review from denikAugust 27, 2025 11:55
@anton-107
anton-107 added this pull request to the merge queueAug 29, 2025
Merged via the queue into main with commit b2c1182Aug 29, 2025
13 checks passed
@anton-107
anton-107 deleted the anton-107/psql-retries branch August 29, 2025 11:20
deco-sdk-taggingBot added a commit that referenced this pull request Sep 3, 2025
@deco-sdk-tagging
[Release] Release v0.267.0
e6f94a3
## Release v0.267.0
### CLI
* Introduce retries to `databricks psql` command ([#3492](#3492))
* Add rule files for coding agents working on the CLI code base ([#3245](#3245))
### Dependency updates
* Upgrade TF provider to 1.88.0 ([#3529](#3529))
* Upgrade Go SDK to 0.82.0
### Bundles
* Update default-python template to make DB Connect work out of the box for unit tests, using uv to install dependencies ([#3254](#3254))
* Add support for `TaskRetryMode` for continuous jobs ([#3529](#3529))
* Add support for specifying database instance as an application resource ([#3529](#3529))
* Allow referencing job libraries outside bundle root without the need to specify sync root ([#2842](#2842))
* Add top level `run_as` support for Lakeflow Declarative Pipelines ([#3307](#3307))
denik pushed a commit that referenced this pull request May 20, 2026
@anton-107
Introduce retries todatabricks psqlcommand (#3492)
ec0a845
## Why
<!-- Why are these changes needed? Provide the context that the reviewer
might be missing.
For example, were there any decisions behind the change that are not
reflected in the code itself? -->
Sometimes the connection to the database instance might fail, which can
be remediated with a retry. This change makes the command to retry the
connection 3 times by default.
To disable the retries users can set a `max-retries` value to zero:
```
$ databricks psql my-instance --max-retries 0
```
## Tests
<!-- How have you tested the changes? -->
Added a new acceptance test, modified and existing acceptance test; ran
manual tests on Mac and Windows
<!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
denik pushed a commit that referenced this pull request May 20, 2026
@deco-sdk-tagging
[Release] Release v0.267.0
7bfd24d
## Release v0.267.0
### CLI
* Introduce retries to `databricks psql` command ([#3492](#3492))
* Add rule files for coding agents working on the CLI code base ([#3245](#3245))
### Dependency updates
* Upgrade TF provider to 1.88.0 ([#3529](#3529))
* Upgrade Go SDK to 0.82.0
### Bundles
* Update default-python template to make DB Connect work out of the box for unit tests, using uv to install dependencies ([#3254](#3254))
* Add support for `TaskRetryMode` for continuous jobs ([#3529](#3529))
* Add support for specifying database instance as an application resource ([#3529](#3529))
* Allow referencing job libraries outside bundle root without the need to specify sync root ([#2842](#2842))
* Add top level `run_as` support for Lakeflow Declarative Pipelines ([#3307](#3307))
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denikdenikdenik approved these changes
@pieternpieternAwaiting requested review from pietern
@andrewnesterandrewnesterAwaiting requested review from andrewnester
@shreyas-goenkashreyas-goenkaAwaiting requested review from shreyas-goenka

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anton-107@eng-dev-ecosystem-bot@denik