Skip to content

[feature](information_schema) add rowsets table into information_s… - #11266

Merged
yiguolei merged 13 commits into
apache:masterfrom
BePPPower:dev-segmentsTable
Aug 9, 2022
Merged

[feature](information_schema) add rowsets table into information_s…#11266
yiguolei merged 13 commits into
apache:masterfrom
BePPPower:dev-segmentsTable

Conversation

@BePPPower

@BePPPowerBePPPower commented Jul 27, 2022

Copy link
Copy Markdown
Contributor

…chema

Proposed changes

Issue Number: close #xxx

Problem Summary:

In some cases, we would like to check the status of rowsets. So I add the rowsets table into information_schema database .
The table schema of rowsets is:

MySQL [(none)]> desc information_schema.rowsets;
+------------------------+------------+------+-------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+------------+------+-------+---------+-------+
| BACKEND_ID | BIGINT | Yes | false | NULL | |
| ROWSET_ID | VARCHAR(*) | Yes | false | NULL | |
| TABLET_ID | BIGINT | Yes | false | NULL | |
| ROWSET_NUM_ROWS | BIGINT | Yes | false | NULL | |
| TXN_ID | BIGINT | Yes | false | NULL | |
| NUM_SEGMENTS | BIGINT | Yes | false | NULL | |
| START_VERSION | BIGINT | Yes | false | NULL | |
| END_VERSION | BIGINT | Yes | false | NULL | |
| INDEX_DISK_SIZE | BIGINT | Yes | false | NULL | |
| DATA_DISK_SIZE | BIGINT | Yes | false | NULL | |
| CREATION_TIME | BIGINT | Yes | false | NULL | |
| OLDEST_WRITE_TIMESTAMP | BIGINT | Yes | false | NULL | |
| NEWEST_WRITE_TIMESTAMP | BIGINT | Yes | false | NULL | |
+------------------------+------------+------+-------+---------+-------+

and then we can search rowsets info from rowsets table, like this:

MySQL [(none)]> select * from information_schema.rowsets where BACKEND_ID = 10004 limit 10;
+------------+--------------------------------------------------+-----------+-----------------+--------+--------------+---------------+-------------+-----------------+----------------+---------------+------------------------+------------------------+
| BACKEND_ID | ROWSET_ID | TABLET_ID | ROWSET_NUM_ROWS | TXN_ID | NUM_SEGMENTS | START_VERSION | END_VERSION | INDEX_DISK_SIZE | DATA_DISK_SIZE | CREATION_TIME | OLDEST_WRITE_TIMESTAMP | NEWEST_WRITE_TIMESTAMP |
+------------+--------------------------------------------------+-----------+-----------------+--------+--------------+---------------+-------------+-----------------+----------------+---------------+------------------------+------------------------+
| 10004 | 02000000000000994847fbd41a42297d7c7a57d3bcb46f8c | 10771 | 66850 | 6 | 1 | 3 | 3 | 2894 | 688855 | 1659964582 | 1659964581 | 1659964581 |
| 10004 | 020000000000008d4847fbd41a42297d7c7a57d3bcb46f8c | 10771 | 66850 | 2 | 1 | 2 | 2 | 2894 | 688855 | 1659964575 | 1659964574 | 1659964574 |
| 10004 | 02000000000000894847fbd41a42297d7c7a57d3bcb46f8c | 10771 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1659964567 | 1659964567 | 1659964567 |
| 10004 | 020000000000009a4847fbd41a42297d7c7a57d3bcb46f8c | 10773 | 66639 | 6 | 1 | 3 | 3 | 2897 | 686828 | 1659964582 | 1659964581 | 1659964581 |
| 10004 | 020000000000008e4847fbd41a42297d7c7a57d3bcb46f8c | 10773 | 66639 | 2 | 1 | 2 | 2 | 2897 | 686828 | 1659964575 | 1659964574 | 1659964574 |
| 10004 | 02000000000000884847fbd41a42297d7c7a57d3bcb46f8c | 10773 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1659964567 | 1659964567 | 1659964567 |
| 10004 | 02000000000000984847fbd41a42297d7c7a57d3bcb46f8c | 10757 | 66413 | 6 | 1 | 3 | 3 | 2893 | 685381 | 1659964582 | 1659964581 | 1659964581 |
| 10004 | 020000000000008c4847fbd41a42297d7c7a57d3bcb46f8c | 10757 | 66413 | 2 | 1 | 2 | 2 | 2893 | 685381 | 1659964575 | 1659964574 | 1659964574 |
| 10004 | 02000000000000874847fbd41a42297d7c7a57d3bcb46f8c | 10757 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1659964567 | 1659964567 | 1659964567 |
| 10004 | 020000000000009c4847fbd41a42297d7c7a57d3bcb46f8c | 10739 | 1698 | 8 | 1 | 3 | 3 | 454 | 86126 | 1659964582 | 1659964582 | 1659964582 |
+------------+--------------------------------------------------+-----------+-----------------+--------+--------------+---------------+-------------+-----------------+----------------+---------------+------------------------+------------------------+

Checklist(Required)

  1. Type of your changes:
    • Improvement
    • Fix
    • Feature-WIP
    • Feature
    • Doc
    • Refator
    • Others:
  2. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  3. Has unit tests been added:
    • Yes
    • No
    • No Need
  4. Has document been added or modified:
    • Yes
    • No
    • No Need
  5. Does it need to update dependencies:
    • Yes
    • No
  6. Are there any changes that cannot be rolled back:
    • Yes
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actionsgithub-actionsBot added the area/planner Issues or PRs related to the query planner label Jul 27, 2022

@yiguoleiyiguolei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will review this pr.

Comment threadfe/fe-core/src/main/java/org/apache/doris/planner/SingleNodePlanner.java Outdated
Comment threadbe/src/exec/schema_scanner/schema_segments_scanner.cpp Outdated
Comment threadbe/src/exec/schema_scanner/schema_segments_scanner.cpp
Comment threadbe/src/exec/schema_scanner/schema_segments_scanner.cpp Outdated
Comment threadfe/fe-core/src/main/java/org/apache/doris/catalog/SchemaTable.java Outdated
@BePPPowerBePPPower changed the title [feature](information_schema) add 'segments' table into information_s…[feature](information_schema) add rowsets table into information_s…Aug 9, 2022

@yiguoleiyiguolei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
yiguolei merged commit 169996d into apache:masterAug 9, 2022
924060929 added a commit that referenced this pull request May 9, 2025
…one backend's data (#50721)
fix backend_active_tasks table only return one backend's data, for
example:
```sql
select sum(SCAN_ROWS), sum(SCAN_BYTES)
from backend_active_tasks where QUERY_ID = 'd299cb2156ef4870-aea578938f703503'
```
the root cause is: the partition column `be_id` has be pruned, and can
not get scan ranges from multiple backends, this bug is introduced by #11266
github-actionsBot pushed a commit that referenced this pull request May 9, 2025
…one backend's data (#50721)
fix backend_active_tasks table only return one backend's data, for
example:
```sql
select sum(SCAN_ROWS), sum(SCAN_BYTES)
from backend_active_tasks where QUERY_ID = 'd299cb2156ef4870-aea578938f703503'
```
the root cause is: the partition column `be_id` has be pruned, and can
not get scan ranges from multiple backends, this bug is introduced by #11266
github-actionsBot pushed a commit that referenced this pull request May 9, 2025
…one backend's data (#50721)
fix backend_active_tasks table only return one backend's data, for
example:
```sql
select sum(SCAN_ROWS), sum(SCAN_BYTES)
from backend_active_tasks where QUERY_ID = 'd299cb2156ef4870-aea578938f703503'
```
the root cause is: the partition column `be_id` has be pruned, and can
not get scan ranges from multiple backends, this bug is introduced by #11266
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…one backend's data (apache#50721)
fix backend_active_tasks table only return one backend's data, for
example:
```sql
select sum(SCAN_ROWS), sum(SCAN_BYTES)
from backend_active_tasks where QUERY_ID = 'd299cb2156ef4870-aea578938f703503'
```
the root cause is: the partition column `be_id` has be pruned, and can
not get scan ranges from multiple backends, this bug is introduced by apache#11266
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plannerIssues or PRs related to the query planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@BePPPower@yiguolei