Skip to content

test - #41839

Closed
bobhan1 wants to merge 4 commits into
apache:branch-2.1from
bobhan1:run-CI-for-41784
Closed

test#41839
bobhan1 wants to merge 4 commits into
apache:branch-2.1from
bobhan1:run-CI-for-41784

Conversation

@bobhan1

@bobhan1bobhan1 commented Oct 15, 2024

Copy link
Copy Markdown
Contributor

based on #41784, resolve conflicts with the latest branch-2.1 to run branch-2.1 CI

@doris-robot

doris-robot commented Oct 15, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actionsBot reviewed Oct 15, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/exec/tablet_info.h
Comment threadbe/src/olap/partial_update_info.cpp
Comment threadbe/src/olap/partial_update_info.h
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/tablet.cpp
github-actions[bot]
github-actionsBot reviewed Oct 15, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
github-actions[bot]
github-actionsBot reviewed Oct 15, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
@bobhan1

bobhan1 commented Oct 15, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Oct 15, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.97% (9349/25988)
Line Coverage: 27.50% (76793/279232)
Region Coverage: 26.34% (39457/149817)
Branch Coverage: 23.11% (20066/86812)
Coverage Report: http://coverage.selectdb-in.cc/coverage/366a0271cfed2821f2e25adad3d3dd475a4935ec_366a0271cfed2821f2e25adad3d3dd475a4935ec/report/index.html

@bobhan1
tmp, pick [opt](partial update) Extract some common logic in partial …
c882bbf
…update apache#39619
pick [opt](partial update) Remove unnecessary lock and refactor some code for partial update (apache#40062)
1. apache#34112 let partial update fetch
rowsets in the initialization of RowsetBuilder rather than flush phase.
So we can remove that tablet header lock.
2. refactor some partial update code
fix compile
pick [Fix](partial update) Fix __DORIS_SEQUENCE_COL__ is not set for newly inserted rows in partial update apache#40272
picks apache#40272
pick [Cherry-pick](branch-2.1) Pick "[Featrue](default value) Support bitmap_empty default value (apache#40364)" (apache#40487)
Pick apache#40364
<!--Describe your changes.-->
pick [Feature](partial update) Support flexible partial update in stream load with json files (apache#39756)
This PR add the ability to update different columns for each row in one
stream load
Doc: apache/doris-website#1140
```sql
MySQL root@127.1:d1> CREATE TABLE t1 (
-> `k` int(11) NULL,
-> `v1` BIGINT NULL,
-> `v2` BIGINT NULL DEFAULT "9876",
-> `v3` BIGINT NOT NULL,
-> `v4` BIGINT NOT NULL DEFAULT "1234",
-> `v5` BIGINT NULL
-> ) UNIQUE KEY(`k`) DISTRIBUTED BY HASH(`k`) BUCKETS 1
-> PROPERTIES(
-> "replication_num" = "1",
-> "enable_unique_key_merge_on_write" = "true");
Query OK, 0 rows affected
Time: 0.013s
MySQL root@127.1:d1> insert into t1 select number, number, number, number, number, number from numbers("number" = "6");
Query OK, 6 rows affected
Time: 0.107s
MySQL root@127.1:d1> select * from t1;
+---+----+----+----+----+----+
| k | v1 | v2 | v3 | v4 | v5 |
+---+----+----+----+----+----+
| 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 2 | 2 | 2 | 2 | 2 |
| 3 | 3 | 3 | 3 | 3 | 3 |
| 4 | 4 | 4 | 4 | 4 | 4 |
| 5 | 5 | 5 | 5 | 5 | 5 |
+---+----+----+----+----+----+
```
test1.json:
```json
{"k": 1, "v1": 10}
{"k": 2, "v2": 20, "v5": 25}
{"k": 3, "v3": 30}
{"k": 4, "v4": 20, "v1": 43, "v3": 99}
{"k": 5, "v5": null}
{"k": 6, "v1": 999, "v3": 777}
{"k": 2, "v4": 222}
{"k": 1, "v2": 111, "v3": 111}
```
```bash
curl --location-trusted -u root: \
-H "strict_mode:false" \
-H "format:json" \
-H "read_json_by_line:true" \
-H "unique_key_update_mode:UPDATE_FLEXIBLE_COLUMNS" \
-T test1.json \
-XPUT http://<host>:<http_port>/api/d1/t1/_stream_load
```
```sql
MySQL root@127.1:d1> select * from t1;
+---+-----+------+-----+------+--------+
| k | v1 | v2 | v3 | v4 | v5 |
+---+-----+------+-----+------+--------+
| 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 10 | 111 | 111 | 1 | 1 |
| 2 | 2 | 20 | 2 | 222 | 25 |
| 3 | 3 | 3 | 30 | 3 | 3 |
| 4 | 43 | 4 | 99 | 20 | 4 |
| 5 | 5 | 5 | 5 | 5 | <null> |
| 6 | 999 | 9876 | 777 | 1234 | <null> |
+---+-----+------+-----+------+--------+
```
fix compile
pick [branch-2.1] Picks "[opt](partial update) Allow to only specify key columns in partial update apache#40736" (apache#40863)
picks apache#40736
fix
@bobhan1

bobhan1 commented Oct 15, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Oct 15, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.97% (9349/25988)
Line Coverage: 27.51% (76816/279253)
Region Coverage: 26.33% (39448/149826)
Branch Coverage: 23.11% (20069/86828)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2a506b1696e6ef42a0879526e94be725fde2ca2b_2a506b1696e6ef42a0879526e94be725fde2ca2b/report/index.html

zhannngchenand others added 2 commits October 16, 2024 14:34
@zhannngchen@bobhan1
pick [opt](merge-on-write) avoid to check delete bitmap while lookup …
705669f
…rowkey in some situation to reduce CPU cost (apache#41480) (apache#41439)
Issue Number: close #xxx
cherry-pick apache#41480
fix
@bobhan1
fix case
e3b9b2a
@bobhan1

bobhan1 commented Oct 16, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@bobhan1

bobhan1 commented Oct 16, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.96% (9348/25999)
Line Coverage: 27.49% (76799/279366)
Region Coverage: 26.32% (39443/149885)
Branch Coverage: 23.09% (20060/86882)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c21c60e703e1982114936a3ffd5b76e525af9e58_c21c60e703e1982114936a3ffd5b76e525af9e58/report/index.html

@bobhan1bobhan1 closed this Nov 14, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@yiguoleiyiguoleiAwaiting requested review from yiguoleiyiguolei will be requested when the pull request is marked ready for reviewyiguolei is a code owner

Assignees

No one assigned

Labels

meta-change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bobhan1@doris-robot@zhannngchen