Skip to content

Add regression tests for CLI error output - #1566

Merged
pietern merged 5 commits into
databricks:mainfrom
kanterov:add-main-tests
Jul 10, 2024
Merged

Add regression tests for CLI error output#1566
pietern merged 5 commits into
databricks:mainfrom
kanterov:add-main-tests

Conversation

@kanterov

@kanterovkanterov commented Jul 3, 2024

Copy link
Copy Markdown
Collaborator

Changes

Add regression tests for #1563

We test 2 code paths:

  • if there is an error, we can print to stderr
  • if there is a valid output, we can print to stdout

We should also consider adding black-box tests that will run the CLI binary as a black box and inspect its output to stderr/stdout.

Tests

Unit tests

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also modify cobraTestRunner in internal/helpers.go to use the same?

Comment threadmain.go Outdated
Comment on lines +13 to +15
code := root.Execute(ctx, cmd.New(ctx))

os.Exit(code)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
code:=root.Execute(ctx, cmd.New(ctx))
os.Exit(code)
code:=root.Execute(ctx, cmd.New(ctx))
os.Exit(code)

Comment threadcmd/root/root_test.go Outdated
Comment on lines +1 to +2
package root_test // using 'root' will create circular import
import (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The circular import thing is implied. We use the same pattern in other places.

Suggested change
package root_test // using 'root' will create circular import
import (
package root_test
import (

@lennartkats-dblennartkats-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL at Pieter's comments, and please make the PR title a bit more descriptive for changelog purposes, otherwise LGTM

@kanterovkanterov changed the title Add tests for ExecuteAdd regression tests for CLI error outputJul 4, 2024
@kanterov

Copy link
Copy Markdown
CollaboratorAuthor

@pietern I didn't know cobraTestRunner existed. I've fixed its fidelity, now it uses root.Execute and logs to stderr. I think we don't need root_test.go anymore, because we can use cobra tests instead.

@pieternpietern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

I'm triggering an integration test run to confirm they pass with the modified helper.

@kanterov

Copy link
Copy Markdown
CollaboratorAuthor

@pietern is this good to merge?

@kanterov
kanterov requested a review from pieternJuly 8, 2024 12:05
@pietern

Copy link
Copy Markdown
Contributor

Yes, good to go!

@pietern
pietern enabled auto-merge July 10, 2024 06:36
@pietern
pietern added this pull request to the merge queueJul 10, 2024
Merged via the queue into databricks:main with commit 25737bbJul 10, 2024
andrewnester added a commit that referenced this pull request Jul 10, 2024
Bundles:
* Override complex variables with target overrides instead of merging ([#1567](#1567)).
* Rewrite local path for libraries in foreach tasks ([#1569](#1569)).
* Change SetVariables mutator to mutate dynamic configuration instead ([#1573](#1573)).
* Return early in bundle destroy if no deployment exists ([#1581](#1581)).
* Let notebook detection code use underlying metadata if available ([#1574](#1574)).
* Remove schema override for variable default value ([#1536](#1536)).
* Print diagnostics in 'bundle deploy' ([#1579](#1579)).
Internal:
* Update actions/upload-artifact to v4 ([#1559](#1559)).
* Use Go 1.22 to build and test ([#1562](#1562)).
* Move bespoke status call to main workspace files filer ([#1570](#1570)).
* Add new template ([#1578](#1578)).
* Add regression tests for CLI error output ([#1566](#1566)).
Dependency updates:
* Bump golang.org/x/mod from 0.18.0 to 0.19.0 ([#1576](#1576)).
* Bump golang.org/x/term from 0.21.0 to 0.22.0 ([#1577](#1577)).
@andrewnesterandrewnester mentioned this pull request Jul 10, 2024
github-merge-queueBot pushed a commit that referenced this pull request Jul 10, 2024
Bundles:
* Override complex variables with target overrides instead of merging
([#1567](#1567)).
* Rewrite local path for libraries in foreach tasks
([#1569](#1569)).
* Change SetVariables mutator to mutate dynamic configuration instead
([#1573](#1573)).
* Return early in bundle destroy if no deployment exists
([#1581](#1581)).
* Let notebook detection code use underlying metadata if available
([#1574](#1574)).
* Remove schema override for variable default value
([#1536](#1536)).
* Print diagnostics in 'bundle deploy'
([#1579](#1579)).
Internal:
* Update actions/upload-artifact to v4
([#1559](#1559)).
* Use Go 1.22 to build and test
([#1562](#1562)).
* Move bespoke status call to main workspace files filer
([#1570](#1570)).
* Add new template
([#1578](#1578)).
* Add regression tests for CLI error output
([#1566](#1566)).
Dependency updates:
* Bump golang.org/x/mod from 0.18.0 to 0.19.0
([#1576](#1576)).
* Bump golang.org/x/term from 0.21.0 to 0.22.0
([#1577](#1577)).
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Add regression tests for #1563
We test 2 code paths:
- if there is an error, we can print to stderr
- if there is a valid output, we can print to stdout
We should also consider adding black-box tests that will run the CLI
binary as a black box and inspect its output to stderr/stdout.
## Tests
Unit tests
denik pushed a commit that referenced this pull request May 20, 2026
Bundles:
* Override complex variables with target overrides instead of merging
([#1567](#1567)).
* Rewrite local path for libraries in foreach tasks
([#1569](#1569)).
* Change SetVariables mutator to mutate dynamic configuration instead
([#1573](#1573)).
* Return early in bundle destroy if no deployment exists
([#1581](#1581)).
* Let notebook detection code use underlying metadata if available
([#1574](#1574)).
* Remove schema override for variable default value
([#1536](#1536)).
* Print diagnostics in 'bundle deploy'
([#1579](#1579)).
Internal:
* Update actions/upload-artifact to v4
([#1559](#1559)).
* Use Go 1.22 to build and test
([#1562](#1562)).
* Move bespoke status call to main workspace files filer
([#1570](#1570)).
* Add new template
([#1578](#1578)).
* Add regression tests for CLI error output
([#1566](#1566)).
Dependency updates:
* Bump golang.org/x/mod from 0.18.0 to 0.19.0
([#1576](#1576)).
* Bump golang.org/x/term from 0.21.0 to 0.22.0
([#1577](#1577)).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kanterov@pietern@lennartkats-db