Skip to content

branch-4.1: [Fix](Regression) reduce duplicate hive2/hive3 regression runs #61671 - #61749

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
xylaaaaa:auto-pick-61671-branch-4.1
Mar 26, 2026
Merged

branch-4.1: [Fix](Regression) reduce duplicate hive2/hive3 regression runs #61671#61749
yiguolei merged 1 commit into
apache:branch-4.1from
xylaaaaa:auto-pick-61671-branch-4.1

Conversation

@xylaaaaa

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: #61671

Problem Summary: Cherry-pick #61671 to branch-4.1. This reduces duplicate hive2/hive3 regression execution by limiting the affected Hive regression suites to hive3-only on branch-4.1.

Release note

None

Check List (For Author)

  • Test: No need to test (regression script-only backport); verified git diff --check HEAD^ HEAD
  • Behavior changed: Yes (affected Hive regression suites no longer run duplicated hive2/hive3 coverage)
  • Does this need documentation: No

…e#61671)
- reduce a set of general external Hive regression suites from
hive2+hive3 dual-run to hive3-only
- keep the more version-sensitive Hive3-specific coverage separate
- shrink duplicate extern regression workload without removing dedicated
Hive3 feature cases
(cherry picked from commit 51c2609)
@xylaaaaa
xylaaaaa requested a review from yiguolei as a code ownerMarch 26, 2026 05:16
CopilotAI review requested due to automatic review settings March 26, 2026 05:16
@Thearas

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Cherry-picks #61671 onto branch-4.1 to reduce duplicated Hive regression workload by running a set of Hive suites against hive3 only (instead of hive2 + hive3), while preserving Hive3 coverage.

Changes:

  • Update multiple Hive-related regression suites to iterate only over ["hive3"] rather than ["hive2", "hive3"].
  • Keep suite logic intact while reducing duplicated execution time/cost.
  • Minor whitespace cleanup in a couple of large Hive TPCH suites.

Reviewed changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated no comments.

Show a summary per file
FileDescription
regression-test/suites/mtmv_p0/test_hive_rewrite_mtmv.groovyRun MTMV Hive rewrite regression only on hive3
regression-test/suites/mtmv_p0/test_hive_olap_mtmv.groovyRun MTMV Hive+OLAP regression only on hive3
regression-test/suites/mtmv_p0/test_hive_mtmv.groovyRun MTMV Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_wide_table.groovyRun wide-table Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_utf8_check.groovyRun UTF-8 Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_upper_case_column_name.groovyRun uppercase-column-name Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_truncate_char_or_varchar_columns.groovyRun truncate char/varchar Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_text_skip_header.groovyRun text skip-header Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_text_garbled_file.groovyRun garbled text file Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_string_dict_filter.groovyRun string dict filter Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_special_orc_formats.groovyRun special ORC formats Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_parquet_nested_types.groovyRun parquet nested types Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_parquet_bloom_filter.groovyRun parquet bloom filter Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_orc_nested_types.groovyRun ORC nested types Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_open_csv_serde.groovyRun open CSV serde Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_multi_delimit_serde.groovyRun multi-delimiter serde Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_mixed_par_locations.groovyRun mixed parquet locations Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_varbinary_type.groovyRun varbinary type Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_to_date.groovyRun hive to_date Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_to_array.groovyRun hive to_array Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_text_complex_type.groovyRun text complex type Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_tablesample_p0.groovyRun TABLESAMPLE Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistics_p0.groovyRun Hive statistics regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistics_all_type_p0.groovyRun Hive statistics (all types) regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistic_timeout.groovyRun Hive statistics timeout regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistic_clean.groovyRun Hive statistics cleanup regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistic_auto.groovyRun Hive statistics auto regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_statistic.groovyRun Hive statistics regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_schema_evolution.groovyRun Hive schema evolution regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_same_db_table_name.groovyRun same DB/table name Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_runtime_filter_partition_pruning.groovyRun runtime filter partition pruning regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_remove_partition.groovyRun remove partition Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_query_cache.groovyRun Hive query cache regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_partition_location.groovyRun partition location regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_partition_column_analyze.groovyRun partition column analyze regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_parquet_skip_page.groovyRun parquet skip-page regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_parquet_alter_column.groovyRun parquet alter-column regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_parquet.groovyRun parquet regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_other.groovyRun misc Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_orc.groovyRun ORC regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_get_schema_from_table.groovyRun get-schema-from-table regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_default_partition.groovyRun default partition regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_broker_scan.groovyRun broker scan regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_basic_type.groovyRun basic types regression only on hive3
regression-test/suites/external_table_p0/hive/test_hive_analyze_db.groovyRun analyze DB regression only on hive3
regression-test/suites/external_table_p0/hive/test_external_credit_data.groovyRun external credit data regression only on hive3
regression-test/suites/external_table_p0/hive/test_external_catalog_hive_partition.groovyRun external catalog partition regression only on hive3
regression-test/suites/external_table_p0/hive/test_external_catalog_hive.groovyRun external catalog Hive regression only on hive3
regression-test/suites/external_table_p0/hive/test_drop_expired_table_stats.groovyRun drop expired stats regression only on hive3
regression-test/suites/external_table_p0/hive/test_different_parquet_types.groovyRun different parquet types regression only on hive3
regression-test/suites/external_table_p0/hive/test_different_column_orders.groovyRun different column orders regression only on hive3
regression-test/suites/external_table_p0/hive/test_complex_types.groovyRun complex types regression only on hive3
regression-test/suites/external_table_p0/hive/hive_tpch_sf1_parquet.groovyRun TPCH parquet suite only on hive3 (+ whitespace cleanup)
regression-test/suites/external_table_p0/hive/hive_tpch_sf1_orc.groovyRun TPCH ORC suite only on hive3 (+ whitespace cleanup)
regression-test/suites/external_table_p0/hive/hive_json_basic_test.groovyRun Hive JSON basic suite only on hive3
Comments suppressed due to low confidence (2)

regression-test/suites/external_table_p0/hive/test_text_skip_header.groovy:30

  • catalog_name is set to "${hivePrefix}_test_utf8_check", which is the same catalog name used by test_utf8_check.groovy. Because regression suites can run in parallel, this shared catalog name can cause cross-suite interference/flakiness (each suite drops/creates the same catalog). Use a suite-specific catalog name here (e.g., based on test_hive_text_skip_header) to avoid collisions.
    regression-test/suites/external_table_p0/hive/test_hive_analyze_db.groovy:47
  • This suite uses catalog_name = "${hivePrefix}_test_hive_partition_column_analyze", which collides with test_hive_partition_column_analyze.groovy using the exact same catalog name. Since suites may execute concurrently, they can drop/recreate each other’s catalog and cause flaky failures. Rename this catalog to something unique for test_hive_analyze_db (and keep the drop/create isolated to this suite).
 for (String hivePrefix : ["hive3"]) {
String extHiveHmsHost = context.config.otherConfigs.get("externalEnvIp")
String extHiveHmsPort = context.config.otherConfigs.get(hivePrefix + "HmsPort")
String catalog_name = "${hivePrefix}_test_hive_partition_column_analyze"
sql """drop catalog if exists ${catalog_name};"""

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@xylaaaaa

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yiguolei
yiguolei merged commit 14753d2 into apache:branch-4.1Mar 26, 2026
31 of 34 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@xylaaaaa@Thearas@yiguolei