[fix](regression) Isolate non-catalog Kerberos test database by suxiaogang223 · Pull Request #65825 · apache/doris

suxiaogang223 · 2026-07-20T14:52:58Z

What problem does this PR solve?

Issue Number: None

Problem Summary:

The Kerberos external regression suites can run concurrently against the same Hive Metastore. Both test_non_catalog_kerberos and test_two_hive_kerberos used the database name test_krb_hive_db.

Creating an external HMS database with IF NOT EXISTS performs a check followed by a create operation. When both suites create the database concurrently, both checks can observe that it is absent, and one request then fails with an HMS AlreadyExistsException.

Use a dedicated test_non_catalog_krb_hive_db database for the non-catalog Kerberos suite. This isolates its fixture from the two-Hive suite without changing the tested export, outfile, HDFS, or Kerberos behavior.

Release note

None

Check List (For Author)

Test: Static validation
- git diff --check
- Verified the two concurrent Kerberos suites no longer share the same HMS database name
Behavior changed: No
Does this need documentation: No

Use a dedicated Hive database for the non-catalog Kerberos suite so it does not race with test_two_hive_kerberos when both create fixtures concurrently.

hello-stephen · 2026-07-20T14:53:13Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

suxiaogang223 · 2026-07-20T14:55:38Z

run buildall

github-actions · 2026-07-20T15:03:11Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-07-20T15:03:13Z

PR approved by anyone and no changes requested.

suxiaogang223 · 2026-07-21T02:02:07Z

/review

github-actions · 2026-07-21T02:13:03Z

Codex automated review failed and did not complete.

Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Jul 25th, 2026 3:35 AM.
Workflow run: https://github.com/apache/doris/actions/runs/29794748785

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

## What changed - Backport the Hive baseline and incremental bootstrap workflow, startup health checks, and external Docker stability improvements. - Backport the ClickHouse startup stabilization and external regression stage timing summary. - Remove obsolete Hive transactional cases and move memory-intensive external cases from P0 to P2. - Remove the obsolete Parquet complex cross-page regression case. - Backport the lightweight Kerberos environment and isolate the non-catalog Kerberos test database. - Apply the minimal deterministic ordering fix from #63719 for Iceberg and Paimon initialization scripts. ## Why `branch-4.1` still uses the older external Docker initialization path and misses several follow-up fixes from `master`. This can leave Hive modules incompletely refreshed, make Iceberg/Paimon initialization order nondeterministic, and consume unnecessary resources in external regression jobs. This PR brings the related fixes into `branch-4.1` as one ordered backport so the external regression environment matches the stabilized behavior on `master`. ## Related PRs Related to #61671, #61869, #61852, #62103, #62664, #62690, #65251, #65460, #65459, #65564, #63719, and #65825. ## Validation - `git diff --check` - Bash syntax checks for all changed `.sh` and `.sh.tpl` files - `bash docker/thirdparties/test/run-thirdparties-docker-hive-bootstrap-groups-test.sh` - `python3 regression-test/pipeline/common/test_stage_timer.py` Compilation and local Docker startup were intentionally skipped because these changes are limited to Docker and regression infrastructure. The external regression pipeline will provide end-to-end validation.

This PR backports the external Docker startup improvements and the applicable follow-up fixes to `branch-4.0`. ### Included backports | PR | Backported scope | | --- | --- | | #61852 | Split Hive bootstrap data into common, Hive2-only, and Hive3-only groups | | #62103 | Hive baseline restore plus incremental module refresh | | #62664 | External Docker startup cleanup, diagnostics, and health handling | | #62690 | ClickHouse startup and health-check stabilization | | #63719 | Only the deterministic `find \| sort` changes required by Iceberg/Paimon initialization | | #61671 | Reduce duplicate Hive2/Hive3 regression runs | | #65460 | Remove obsolete Hive transactional cases and their initialization work | | #65564 | Backport the lightweight Kerberos environment | | #61869 | Add external regression stage timing and diagnostics | | #65459 | Move resource-intensive external cases and Docker components to P2 | | #65825 | Apply the follow-up fixes for the lightweight Kerberos environment | | #65251 | Remove the obsolete Parquet complex cross-page case and expected output | The later fixes made while validating the companion `branch-4.1` backport #65913 are also included: - Switch `test_hive_topn_rf_null` to Hive2 so it matches the Hive2-only `run81.hql` bootstrap group. - Change the location-backed `decimals_1_10` table in `run82.hql` to `CREATE TABLE IF NOT EXISTS`, avoiding deletion of preinstalled HDFS data during refresh. - Keep `HIVE_BASELINE_VERSION=20260415`, the same baseline version used by `master` and the companion 4.1 backport. ### Intentionally not included | PR | Reason | | --- | --- | | #63673 | Already present in `branch-4.0` | | #63542 | Already present in `branch-4.0` | | #64886 | `branch-4.0` still uses the older Iceberg/Spark layout; this cannot be picked independently | | #64671 | Contains a BE feature change together with new regression cases; it is outside this Docker backport | | #60262 | Not cherry-picked as a whole; only the final TopN Hive2 mapping needed by the bootstrap grouping is retained | ## Why `branch-4.0` still uses the older external Docker initialization path. Without these backports, Hive modules can remain incompletely refreshed, Iceberg/Paimon SQL execution order can be nondeterministic, and external jobs repeat unnecessary Hive2/Hive3 coverage. During conflict resolution, this backport preserves the `branch-4.0` Iceberg/Paimon 1.10 data archive and jar bootstrap while integrating the new startup and health-check framework. It does not introduce the newer Spark Thrift JDBC architecture. ## Validation - `git diff --check` - Conflict-marker scan across `docker`, `regression-test`, and `be/src` - Bash syntax checks for all changed `.sh` and `.sh.tpl` files - `bash docker/thirdparties/test/run-thirdparties-docker-hive-bootstrap-groups-test.sh` - `python3 regression-test/pipeline/common/test_stage_timer.py` - Confirmed the BE and new serde regression files from #64671 are absent from the final diff Compilation and local Docker startup were intentionally skipped. The final scope contains Docker, scripts, pipeline support, and regression-case organization only. --------- Co-authored-by: Chenjunwei <138805230+xylaaaaa@users.noreply.github.com>

…65825) The Kerberos external regression suites can run concurrently against the same Hive Metastore. Both `test_non_catalog_kerberos` and `test_two_hive_kerberos` used the database name `test_krb_hive_db`. Creating an external HMS database with `IF NOT EXISTS` performs a check followed by a create operation. When both suites create the database concurrently, both checks can observe that it is absent, and one request then fails with an HMS `AlreadyExistsException`. Use a dedicated `test_non_catalog_krb_hive_db` database for the non-catalog Kerberos suite. This isolates its fixture from the two-Hive suite without changing the tested export, outfile, HDFS, or Kerberos behavior.

[fix](regression) Isolate non-catalog Kerberos test database
e34f5b8
Use a dedicated Hive database for the non-catalog Kerberos suite so it does not race with test_two_hive_kerberos when both create fixtures concurrently.

suxiaogang223 marked this pull request as ready for review July 20, 2026 14:55

Gabriel39 approved these changes Jul 20, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 20, 2026

github-actionsBot added the reviewed label Jul 20, 2026

hello-stephen approved these changes Jul 21, 2026
View reviewed changes

hello-stephen merged commit ae88ed4 into apache:masterJul 21, 2026
35 checks passed

suxiaogang223 deleted the codex/fix-kerberos-database-race branch July 21, 2026 02:52

This was referenced Jul 22, 2026
branch-4.1: [fix](docker) Backport external Docker improvements #65913
Merged
branch-4.0: [fix](docker) Backport external Docker improvements #65939
Merged

morningman added dev/4.0.x dev/4.0.8-merged and removed dev/4.0.x labels Jul 23, 2026

github-actionsBot added the dev/4.0.x-conflict label Jul 23, 2026

Navigation Menu

Uh oh!

[fix](regression) Isolate non-catalog Kerberos test database - #65825

[fix](regression) Isolate non-catalog Kerberos test database#65825
hello-stephen merged 1 commit into
apache:masterfrom
suxiaogang223:codex/fix-kerberos-database-race

suxiaogang223 commented Jul 20, 2026

Uh oh!

hello-stephen commented Jul 20, 2026

Uh oh!

suxiaogang223 commented Jul 20, 2026

Uh oh!

github-actionsBot commented Jul 20, 2026

Uh oh!

github-actionsBot commented Jul 20, 2026

Uh oh!

suxiaogang223 commented Jul 21, 2026

Uh oh!

github-actionsBot commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

suxiaogang223 commented Jul 20, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Uh oh!

hello-stephen commented Jul 20, 2026

Uh oh!

suxiaogang223 commented Jul 20, 2026

Uh oh!

github-actionsBot commented Jul 20, 2026

Uh oh!

github-actionsBot commented Jul 20, 2026

Uh oh!

suxiaogang223 commented Jul 21, 2026

Uh oh!

github-actionsBot commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants