Skip to content

gh_report: include skipped tests in the report if they match known_failures.txt - #3767

Merged
denik merged 2 commits into
mainfrom
denik/gh_parse_KNOWN_SKIP
Oct 16, 2025
Merged

gh_report: include skipped tests in the report if they match known_failures.txt#3767
denik merged 2 commits into
mainfrom
denik/gh_parse_KNOWN_SKIP

Conversation

@denik

@denikdenik commented Oct 15, 2025

Copy link
Copy Markdown
Contributor

Why

When CloudSlow tests are added to known_failures.txt then TestAccept shows up as "RECOVERED" even though there are no subtests that actually recovered. This PR does not fix that but at least includes those skipped tests in the output so one can understand the reason for "RECOVERED" status.

…ilures.txt
When cloud slow tests are added to known_failures.txt then TestAccept
shows up as recovered even though there are no subtests that actually
recovered. This PR does not fix that but at least includes those skipped
tests in the output so one can understand the reason for 'RECOVERED'
@denik

denik commented Oct 15, 2025

Copy link
Copy Markdown
ContributorAuthor

Example output for #3766 (comment)

TestAccept/bundle/integration_whl/interactive_single_user is new addition

Env🔄flaky💚RECOVERED🙈SKIP✅pass🙈skip
💚aws linux11321544
💚aws windows11322543
💚aws-ucws linux11437440
💚aws-ucws windows11438439
💚azure linux11321543
💚azure windows11322542
💚azure-ucws linux11437439
💚azure-ucws windows11438438
🔄gcp linux311317545
🔄gcp windows311318544
8 failing tests:
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
TestAccept💚R💚R💚R💚R💚R💚R💚R💚R💚R💚R
TestAccept/bundle/deployment/bind/cluster✅p✅p✅p✅p✅p✅p✅p✅p🔄f✅p
TestAccept/bundle/deployment/bind/dashboard✅p✅p✅p✅p✅p✅p✅p✅p✅p🔄f
TestAccept/bundle/integration_whl/interactive_single_user🙈S🙈S🙈S🙈S🙈S🙈S🙈S🙈S🙈S🙈S
TestAccept/bundle/resources/pipelines/update✅p✅p✅p✅p✅p✅p✅p✅p✅p🔄f
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=direct-exp✅p✅p✅p✅p✅p✅p✅p✅p✅p🔄f
TestAccept/bundle/templates/default-python/integration_classic✅p✅p✅p✅p✅p✅p✅p✅p🔄f✅p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9✅p✅p✅p✅p✅p✅p✅p✅p🔄f✅p

@denik
denik enabled auto-merge October 15, 2025 11:21
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 15, 2025

Copy link
Copy Markdown
Collaborator

Run: 18527194329

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skip
💚​aws linux11321544
💚​aws windows11322543
🔄​aws-ucws linux911428440
🔄​aws-ucws windows911429439
💚​azure linux11321543
💚​azure windows11322542
🔄​azure-ucws linux711430439
🔄​azure-ucws windows711431438
🔄​gcp linux211318545
🔄​gcp windows51317544
17 failing tests:
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R🔄​f
TestAccept/bundle/deployment/bind/dashboard✅​p✅​p🔄​f🔄​f✅​p✅​p✅​p🔄​f✅​p✅​p
TestAccept/bundle/deployment/bind/dashboard/recreation✅​p✅​p🔄​f🔄​f✅​p✅​p✅​p🔄​f✅​p✅​p
TestAccept/bundle/integration_whl/interactive_single_user🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
TestAccept/bundle/resources/dashboards/detect-change✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f✅​p✅​p✅​p
TestAccept/bundle/resources/dashboards/generate_inplace✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f🔄​f✅​p✅​p
TestAccept/bundle/resources/dashboards/nested-folders✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f🔄​f✅​p✅​p
TestAccept/bundle/resources/dashboards/simple✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f🔄​f✅​p✅​p
TestAccept/bundle/resources/dashboards/simple_outside_bundle_root✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f🔄​f✅​p✅​p
TestAccept/bundle/resources/dashboards/simple_syncroot✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f✅​p✅​p✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=yes✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=yes✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p
TestAccept/bundle/templates/default-python/integration_classic✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestDashboardAssumptions_WorkspaceImport✅​p✅​p🔄​f🔄​f✅​p✅​p🔄​f🔄​f✅​p✅​p
TestFsCpFileToDir✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f
TestFsCpFileToDir/dbfs_to_local✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f

@denik
denik added this pull request to the merge queueOct 16, 2025
Merged via the queue into main with commit 7e5ad79Oct 16, 2025
13 checks passed
@denik
denik deleted the denik/gh_parse_KNOWN_SKIP branch October 16, 2025 15:20
denik added a commit that referenced this pull request May 20, 2026
…ilures.txt (#3767)
## Why
When CloudSlow tests are added to known_failures.txt then TestAccept
shows up as "RECOVERED" even though there are no subtests that actually
recovered. This PR does not fix that but at least includes those skipped
tests in the output so one can understand the reason for "RECOVERED"
status.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@eng-dev-ecosystem-bot@anton-107