Skip to content

test only - #41784

Closed
bobhan1 wants to merge 11 commits into
apache:branch-2.1from
bobhan1:branch-2.1-anta-pick-flexible
Closed

test only#41784
bobhan1 wants to merge 11 commits into
apache:branch-2.1from
bobhan1:branch-2.1-anta-pick-flexible

Conversation

@bobhan1

@bobhan1bobhan1 commented Oct 14, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

pick fleixble partial update related PRs to doris-2.1.6-rc04

other PR:

main PR:

@doris-robot

doris-robot commented Oct 14, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions

github-actionsBot commented Oct 14, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch 2 times, most recently from d96fe24 to 4e1b152CompareOctober 14, 2024 04:20
@github-actions

github-actionsBot commented Oct 14, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions

github-actionsBot commented Oct 14, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 4e1b152 to 797d9d5CompareOctober 14, 2024 05:30
@github-actions

github-actionsBot commented Oct 14, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/partial_update_info.cpp Outdated
@bobhan1
pick [opt](partial update) Remove unnecessary lock and refactor some …
166871c
…code for partial update (apache#40062)
1. apache#34112 let partial update fetch
rowsets in the initialization of RowsetBuilder rather than flush phase.
So we can remove that tablet header lock.
2. refactor some partial update code
@bobhan1
fix compile
aae575d
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 4fa2051 to aae575dCompareOctober 14, 2024 06:45
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/partial_update_info.cpp Outdated
@Yukang-Lian@bobhan1
pick [Cherry-pick](branch-2.1) Pick "[Featrue](default value) Support…
9a2de1c
… bitmap_empty default value (apache#40364)" (apache#40487)
Pick apache#40364
<!--Describe your changes.-->
@bobhan1

bobhan1 commented Oct 14, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/partial_update_info.cpp Outdated
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/exec/tablet_info.h
Comment threadbe/src/olap/partial_update_info.cpp
Comment threadbe/src/olap/partial_update_info.h
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 3c25fbf to 27e630dCompareOctober 14, 2024 08:49
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/tablet.cpp
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch 2 times, most recently from 1868432 to 07ca899CompareOctober 14, 2024 10:30
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/partial_update_info.cpp
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 07ca899 to f7a1bb0CompareOctober 14, 2024 10:40
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/segment_writer.cpp
github-actions[bot]
github-actionsBot reviewed Oct 14, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
Comment threadbe/src/olap/rowset/segment_v2/vertical_segment_writer.cpp
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch 2 times, most recently from bb69cea to fe069fdCompareOctober 15, 2024 03:11
@bobhan1bobhan1 mentioned this pull request Oct 15, 2024
Closed
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 5545303 to ccf869cCompareOctober 15, 2024 04:03
github-actions[bot]
github-actionsBot reviewed Oct 15, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/olap/tablet.cpp
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from 11236ee to f54ab54CompareOctober 16, 2024 03:43
bobhan1and others added 5 commits October 16, 2024 14:32
@bobhan1
pick [Feature](partial update) Support flexible partial update in str…
0fec929
…eam load with json files (apache#39756)
This PR add the ability to update different columns for each row in one
stream load
Doc: apache/doris-website#1140
```sql
MySQL root@127.1:d1> CREATE TABLE t1 (
-> `k` int(11) NULL,
-> `v1` BIGINT NULL,
-> `v2` BIGINT NULL DEFAULT "9876",
-> `v3` BIGINT NOT NULL,
-> `v4` BIGINT NOT NULL DEFAULT "1234",
-> `v5` BIGINT NULL
-> ) UNIQUE KEY(`k`) DISTRIBUTED BY HASH(`k`) BUCKETS 1
-> PROPERTIES(
-> "replication_num" = "1",
-> "enable_unique_key_merge_on_write" = "true");
Query OK, 0 rows affected
Time: 0.013s
MySQL root@127.1:d1> insert into t1 select number, number, number, number, number, number from numbers("number" = "6");
Query OK, 6 rows affected
Time: 0.107s
MySQL root@127.1:d1> select * from t1;
+---+----+----+----+----+----+
| k | v1 | v2 | v3 | v4 | v5 |
+---+----+----+----+----+----+
| 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 2 | 2 | 2 | 2 | 2 |
| 3 | 3 | 3 | 3 | 3 | 3 |
| 4 | 4 | 4 | 4 | 4 | 4 |
| 5 | 5 | 5 | 5 | 5 | 5 |
+---+----+----+----+----+----+
```
test1.json:
```json
{"k": 1, "v1": 10}
{"k": 2, "v2": 20, "v5": 25}
{"k": 3, "v3": 30}
{"k": 4, "v4": 20, "v1": 43, "v3": 99}
{"k": 5, "v5": null}
{"k": 6, "v1": 999, "v3": 777}
{"k": 2, "v4": 222}
{"k": 1, "v2": 111, "v3": 111}
```
```bash
curl --location-trusted -u root: \
-H "strict_mode:false" \
-H "format:json" \
-H "read_json_by_line:true" \
-H "unique_key_update_mode:UPDATE_FLEXIBLE_COLUMNS" \
-T test1.json \
-XPUT http://<host>:<http_port>/api/d1/t1/_stream_load
```
```sql
MySQL root@127.1:d1> select * from t1;
+---+-----+------+-----+------+--------+
| k | v1 | v2 | v3 | v4 | v5 |
+---+-----+------+-----+------+--------+
| 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 10 | 111 | 111 | 1 | 1 |
| 2 | 2 | 20 | 2 | 222 | 25 |
| 3 | 3 | 3 | 30 | 3 | 3 |
| 4 | 43 | 4 | 99 | 20 | 4 |
| 5 | 5 | 5 | 5 | 5 | <null> |
| 6 | 999 | 9876 | 777 | 1234 | <null> |
+---+-----+------+-----+------+--------+
```
fix compile
fix
@bobhan1
pick [branch-2.1] Picks "[opt](partial update) Allow to only specify …
f373f7e
…key columns in partial update apache#40736" (apache#40863)
picks apache#40736
@bobhan1
fix
763fa6a
@zhannngchen@bobhan1
pick [opt](merge-on-write) avoid to check delete bitmap while lookup …
dfbc0eb
…rowkey in some situation to reduce CPU cost (apache#41480) (apache#41439)
Issue Number: close #xxx
cherry-pick apache#41480
@bobhan1
fix case
532d4d0
@bobhan1
bobhan1force-pushed the branch-2.1-anta-pick-flexible branch from f54ab54 to 532d4d0CompareOctober 16, 2024 06:33
@bobhan1bobhan1 closed this Nov 14, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@yiguoleiyiguoleiAwaiting requested review from yiguoleiyiguolei will be requested when the pull request is marked ready for reviewyiguolei is a code owner

Assignees

No one assigned

Labels

meta-change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bobhan1@doris-robot@Yukang-Lian@zhannngchen