Skip to content

branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases - #64693

Merged
morningman merged 1 commit into
apache:branch-4.0from
morningman:fix/deflake-p2-cases-198908
Jun 24, 2026
Merged

branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases#64693
morningman merged 1 commit into
apache:branch-4.0from
morningman:fix/deflake-p2-cases-198908

Conversation

@morningman

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Deflake several pre-existing flaky/failing cases in the branch-4.0 P2 regression suite (surfaced by an internal P2 run). None of these are caused by product behavior changes.

  1. opensky_p2count / avgDistance / totalDistance / mostBusyOrigin — backport of [fix](regression) fix p2 opensky failure #62447: the S3 source csv.gz was actually a tar.gz, so after it was corrected the loaded row count changed (+30 rows) while the committed .out still held the old values. Update the four .out files and re-enable the NumberTotalRows == NumberLoadedRows load assertion.

  2. segcompaction_p2/test_segcompaction_agg_keys — backport of [fix](regression) Fix case test_segcompaction_agg_keys #61026: the AGGREGATE REPLACE winner for a duplicate key is not deterministic. S3 load parallelizes into segments whose boundaries are non-deterministic (adaptive memtable flush / memory pressure) and the read-merge keeps the first row by segment order, so which of the duplicate col_0=47 rows wins varies between runs. Assert exactly one row plus the value being one of the legitimate outcomes, and delete the brittle .out.

  3. inverted_index_p2/test_show_data (test_show_data_with_compaction) — it compared index sizes with inverted_index_compaction_enable on vs off for exact equality. For identical data the on-disk size can differ slightly due to file/merge layout, so compare within a 10% tolerance instead, which still catches gross index bloat or corruption.

  4. compaction/test_base_compaction_with_dup_key_max_file_size_limit — the test builds a specific rowset layout via manual compactions and expects a manual base compaction to be rejected with E-808 (input rowset exceeds base_compaction_dup_key_max_file_size_mbytes). When the BE global disable_auto_compaction is false, background compaction races the manual steps and reshapes the rowsets, and an in-test short-circuit masked the result. Disable auto compaction cluster-wide for the duration of the test (restored in finally) and assert on the real response.

  5. compaction/test_single_replica_compaction — the waitForCompaction and getTabletStatus closures assigned code/out without def, which the regression framework's script-source guard rejects (defined global variables in script are not allowed: code). [fix](case) fix global vars used in cases #55933 declared only process; declare the remaining locals too.

Release note

None

🤖 Generated with Claude Code

https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF

Pre-existing flaky/failing cases in the branch-4.0 P2 regression suite
(surfaced by TeamCity build 198908), none caused by product changes:
1) opensky_p2 count / avgDistance / totalDistance / mostBusyOrigin: the
S3 source csv.gz was actually a tar.gz, so after it was corrected the
loaded row count changed (+30 rows), but the committed .out still held
the old values. Backport of apache#62447 (update the four .out files and
re-enable the NumberTotalRows == NumberLoadedRows load assertion).
2) segcompaction_p2/test_segcompaction_agg_keys: the AGGREGATE REPLACE
winner for a duplicate key is not deterministic. S3 load parallelizes
into segments whose boundaries are non-deterministic (adaptive memtable
flush / memory pressure), and the read-merge keeps the first row by
segment order, so which of the duplicate col_0=47 rows wins varies
between runs. Backport of apache#61026 (assert exactly one row plus the value
being one of the legitimate outcomes; delete the brittle .out).
3) inverted_index_p2/test_show_data (test_show_data_with_compaction):
it compared index sizes with inverted_index_compaction on vs off for
exact equality. For identical data the on-disk size can differ slightly
due to file/merge layout, so compare within a 10% tolerance instead,
which still catches gross index bloat or corruption.
4) compaction/test_base_compaction_with_dup_key_max_file_size_limit: the
test builds a specific rowset layout via manual compactions and expects
a manual base compaction to be rejected with E-808 (input rowset exceeds
base_compaction_dup_key_max_file_size_mbytes). When the BE global
disable_auto_compaction is false, background compaction races the manual
steps and reshapes the rowsets, and an in-test short-circuit masked the
result. Disable auto compaction cluster-wide for the duration of the
test (restored in finally) and assert on the real response.
5) compaction/test_single_replica_compaction: the waitForCompaction and
getTabletStatus closures assigned `code`/`out` without `def`, which the
regression framework's script-source guard rejects ("defined global
variables in script are not allowed: code"). apache#55933 declared only
`process`; declare the remaining locals too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningmanmorningman changed the title [fix](test) deflake several branch-4.0 P2 regression casesbranch-4.0: [fix](test) deflake several branch-4.0 P2 regression casesJun 22, 2026
@morningman

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morningman
morningman merged commit 6f1538c into apache:branch-4.0Jun 24, 2026
30 of 33 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@morningman@hello-stephen@hubgeter