Skip to content

Fix race condition in cmdio.LogString - #3868

Merged
pietern merged 1 commit into
mainfrom
generate-race
Nov 4, 2025
Merged

Fix race condition in cmdio.LogString#3868
pietern merged 1 commit into
mainfrom
generate-race

Conversation

@pietern

Copy link
Copy Markdown
Contributor

Changes

Make LogString write the string and newline in a single call to reduce the likelihood of interleaved output when called concurrently. This was atomic until #3818 where I removed the lock, assuming it wasn't necessary.

Note: This fix assumes io.WriteString provides atomic writes for small messages, which is not guaranteed.

Why

Fixes flaky test from: https://github.com/databricks/cli/actions/runs/18995883239/job/54255615389

Make LogString write the string and newline in a single call to reduce
the likelihood of interleaved output when called concurrently.
Note: This fix assumes io.WriteString provides atomic writes for small
messages, which is not guaranteed by the Go specification.
Fixes flaky test from: https://github.com/databricks/cli/actions/runs/18995883239/job/54255615389
Comment threadlibs/cmdio/compat.go
c := fromContext(ctx)
_, _ = io.WriteString(c.err, str)
_, _ = io.WriteString(c.err, "\n")
_, _ = io.WriteString(c.err, str+"\n")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This fix assumes io.WriteString provides atomic writes for small messages, which is not guaranteed.

if it's not guaranteed should we add the lock back?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other call sites where we write to the same output.

It deserves a better fix but that's more work than trying this out.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 3, 2025

Copy link
Copy Markdown
Collaborator

Run: 19034545837

Env❌​FAIL🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skip
💚​aws linux11326591
💚​aws windows11327590
💚​aws-ucws linux11449486
🔄​aws-ucws windows21449485
💚​azure linux11326590
💚​azure windows11327589
❌​azure-ucws linux241440485
💚​azure-ucws windows11446484
🔄​gcp linux611319592
🔄​gcp windows411322591
18 failing tests:
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
TestAccept💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f💚​R💚​R💚​R
TestAccept/bundle/resources/clusters/deploy/data_security_mode✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
TestAccept/bundle/resources/clusters/deploy/update-after-create✅​p✅​p✅​p✅​p✅​p✅​p❌​F✅​p✅​p✅​p
TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p✅​p✅​p✅​p❌​F✅​p✅​p✅​p
TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
TestAccept/bundle/resources/pipelines/auto-approve✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/pipelines/auto-approve/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/pipelines/lakeflow-pipeline✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/pipelines/lakeflow-pipeline/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=direct✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=terraform✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/resources/synced_database_tables/basic🙈​s🙈​s✅​p🔄​f🙈​s🙈​s✅​p✅​p🙈​s🙈​s
TestAccept/bundle/run/app-with-job🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=yes/PY=yes✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=no✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=yes✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=no✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f

@pietern
pietern added this pull request to the merge queueNov 4, 2025
Merged via the queue into main with commit 92029e1Nov 4, 2025
13 checks passed
@pietern
pietern deleted the generate-race branch November 4, 2025 12:28
denik pushed a commit that referenced this pull request May 20, 2026
## Changes
Make `LogString` write the string and newline in a single call to reduce
the likelihood of interleaved output when called concurrently. This was
atomic until #3818 where I removed the lock, assuming it wasn't
necessary.
Note: This fix assumes `io.WriteString` provides atomic writes for small
messages, which is not guaranteed.
## Why
Fixes flaky test from:
https://github.com/databricks/cli/actions/runs/18995883239/job/54255615389
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pietern@eng-dev-ecosystem-bot@denik