Deprecate `--progress-format` flag and remove progress logger types by pietern · Pull Request #3820 · databricks/cli

pietern · 2025-10-24T14:20:04Z

Changes

This change removes remaining references to the "progress logger".

All terminal I/O now goes through a single type in cmdio.

In-place mode was designed to update job progress in place using ANSI escape codes (e.g., showing 'PENDING' → 'RUNNING' → 'TERMINATED' on the same line). However, acceptance tests show jobs typically output only a single state transition: 'Run URL: <url>' followed by '[TIMESTAMP] "job-name" TERMINATED', suggesting the job completes before multiple states can be observed. The default mode selection logic required log-file to not be stderr AND stderr to be a terminal to enable in-place mode, which is an uncommon configuration. Additionally, only JobProgressEvent supported in-place updates while all other events (URLs, errors, pipeline events) fell back to append mode, making the feature inconsistent. The implementation added complexity with ANSI escape codes, terminal detection, and an IsInplaceSupported() interface method across all event types. Since the feature provided minimal practical value and likely was rarely (if ever) enabled by default, it has been removed in favor of the simpler append mode.

The JSON mode for progress logging was designed to output structured JSON events for machine parsing. However, this mode had several limitations: it prevented interactive prompts (Ask/AskSelect methods would error), required --auto-approve for destroy commands, and added complexity with JSON marshaling in the logger. The feature provided minimal practical value as most CLI usage is interactive, and the default append mode is sufficient for both human and machine consumption. Since the mode added unnecessary complexity without clear benefits, it has been removed in favor of the simpler append-only mode.

…te-to-cmdio

…ructure

eng-dev-ecosystem-bot · 2025-10-24T14:33:56Z

Run: 18870840512

	Env	❌FAIL	🟨KNOWN	💚RECOVERED	🙈SKIP	✅pass	🙈skip
💚	aws linux			1	2	319	582
💚	aws windows			1	2	320	581
💚	aws-ucws linux			2	1	439	478
💚	aws-ucws windows			2	1	440	477
💚	azure linux			1	2	319	581
💚	azure windows			1	2	320	580
🟨	azure-ucws linux		2		1	437	477
🟨	azure-ucws windows		2		1	438	476
❌	gcp linux	27	1		2	291	583
❌	gcp windows	20	1		2	299	582

45 failing tests:

Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
TestAccept	💚R	💚R	💚R	💚R	💚R	💚R	🟨K	🟨K	🟨K	🟨K
TestAccept/bundle/deployment/bind/dashboard	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/deployment/bind/job/generate-and-bind	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/deployment/bind/job/job-spark-python-task	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/deployment/bind/model-serving-endpoint	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/local_state_staleness	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/local_state_staleness/DATABRICKS_BUNDLE_ENGINE=direct-exp	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/resources/dashboards/detect-change	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/resources/jobs/double-underscore-keys	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/resources/jobs/double-underscore-keys/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/resources/jobs/fail-on-active-runs	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/resources/jobs/fail-on-active-runs/DATABRICKS_BUNDLE_ENGINE=direct-exp	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/resources/pipelines/lakeflow-pipeline	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/resources/pipelines/lakeflow-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/resources/synced_database_tables/basic	🙈S	🙈S	💚R	💚R	🙈S	🙈S	🟨K	🟨K	🙈S	🙈S
TestAccept/bundle/run/app-with-job	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
TestAccept/bundle/templates/default-python/combinations/classic	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	❌F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=no	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=yes	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=no	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=no	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=yes	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=no	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=yes	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=yes	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/integration_classic	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.10	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.11	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.12	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.13	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.9	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.10	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestFsCpFileToDirFileNotOverwritten	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestFsCpFileToDirFileNotOverwritten/dbfs_to_dbfs	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestSyncIncrementalSyncFileToPythonNotebook	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestExportWithFileFlag	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestImportFileFormatSource	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestImportFileUsingContentFormatAuto	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestImportFileUsingContentFormatSource	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p
TestFilerRecursiveDelete	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F
TestFilerRecursiveDelete/workspace_files	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F

denik

nice cleanup

…3820) ## Changes This change removes remaining references to the "progress logger". All terminal I/O now goes through a single type in `cmdio`. Builds on #3811, #3812, and #3818.

pietern added 7 commits October 23, 2025 15:49

Add entry to NEXT_CHANGELOG.md
8c3f4b7

Migrate progress logger functions to use cmdio directly
a48589f

Remove unused mutex field
50939cc

Merge remote-tracking branch 'origin/main' into progress-logger-migra…
f6d3f65
…te-to-cmdio

Deprecate --progress-format flag and remove progress logger infrast…
3dc83b7
…ructure

pietern requested review from andrewnester, anton-107, denik, jefferycheng1, kanterov, lennartkats-db and shreyas-goenka as code owners October 24, 2025 14:20

pietern temporarily deployed to test-trigger-is October 24, 2025 14:20 — with GitHub Actions Inactive

denik approved these changes Oct 24, 2025
View reviewed changes

andrewnester approved these changes Oct 24, 2025
View reviewed changes

Base automatically changed from progress-logger-migrate-to-cmdio to mainOctober 28, 2025 09:50

Merge remote-tracking branch 'origin/main' into progress-logger-rm
0a599f7

pietern temporarily deployed to test-trigger-is October 28, 2025 09:54 — with GitHub Actions Inactive

pietern merged commit c4a66fb into mainOct 28, 2025
12 of 13 checks passed

pietern deleted the progress-logger-rm branch October 28, 2025 10:51

Navigation Menu

Uh oh!

Deprecate `--progress-format` flag and remove progress logger types - #3820

Deprecate `--progress-format` flag and remove progress logger types#3820
pietern merged 8 commits into
mainfrom
progress-logger-rm

pietern commented Oct 24, 2025

Uh oh!

eng-dev-ecosystem-bot commented Oct 24, 2025•
edited
Loading

Uh oh!

denik left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pietern commented Oct 24, 2025

Changes

Uh oh!

eng-dev-ecosystem-bot commented Oct 24, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

denik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

eng-dev-ecosystem-bot commented Oct 24, 2025•
edited
Loading