Skip to content

[improve](information schema) introduce routine load job system table (#48963) - #49286

Merged
yiguolei merged 2 commits into
apache:branch-2.1from
sollhui:2.1_job_sys_table
May 21, 2025
Merged

[improve](information schema) introduce routine load job system table (#48963)#49286
yiguolei merged 2 commits into
apache:branch-2.1from
sollhui:2.1_job_sys_table

Conversation

@sollhui

Copy link
Copy Markdown
Contributor

pick #48963

Part IV of #48511

doc apache/doris-website#2196

Introduce routine load job statistic system table:

mysql> show create table information_schema.routine_load_job\G
*************************** 1. row ***************************
Table: routine_load_job
Create Table: CREATE TABLE `routine_load_job` (
`JOB_ID` text NULL,
`JOB_NAME` text NULL,
`CREATE_TIME` text NULL,
`PAUSE_TIME` text NULL,
`END_TIME` text NULL,
`DB_NAME` text NULL,
`TABLE_NAME` text NULL,
`STATE` text NULL,
`CURRENT_TASK_NUM` text NULL,
`JOB_PROPERTIES` text NULL,
`DATA_SOURCE_PROPERTIES` text NULL,
`CUSTOM_PROPERTIES` text NULL,
`STATISTIC` text NULL,
`PROGRESS` text NULL,
`LAG` text NULL,
`REASON_OF_STATE_CHANGED` text NULL,
`ERROR_LOG_URLS` text NULL,
`USER_NAME` text NULL,
`CURRENT_ABORT_TASK_NUM` int NULL,
`IS_ABNORMAL_PAUSE` boolean NULL
) ENGINE=SCHEMA;
1 row in set (0.00 sec)

There are some benefits to empower job with SQL query capability for statistical information:

SELECT JOB_NAME
FROM information_schema.routine_load_job_statistics
WHERE CURRENT_ABORT_TASK_NUM > 0
OR IS_ABNORMAL_PAUSE = TRUE;
  • User can use the select * from information_schema.routine_load_job instead of the show routine load. The advantage is that the show routine load can only be searched by name, but SQL can be very flexible in locating jobs

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@sollhui
sollhui requested a review from yiguolei as a code ownerMarch 20, 2025 03:12
@sollhui

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 0.00% (0/130) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage37.37% (9846/26350)
Line Coverage29.05% (82459/283883)
Region Coverage27.66% (42307/152969)
Branch Coverage24.40% (21538/88254)

@sollhui

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/130) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage37.38% (9850/26350)
Line Coverage29.08% (82562/283910)
Region Coverage27.67% (42334/152984)
Branch Coverage24.41% (21548/88266)

@sollhui

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels May 5, 2025
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 0.00% (0/130) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage38.85% (10275/26447)
Line Coverage29.90% (85370/285495)
Region Coverage28.54% (44003/154162)
Branch Coverage25.26% (22508/89088)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/130) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage60.94% (15879/26057)
Line Coverage53.16% (150824/283702)
Region Coverage50.78% (91139/179487)
Branch Coverage44.64% (44974/100750)

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/130) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage60.94% (15879/26057)
Line Coverage53.16% (150824/283702)
Region Coverage50.78% (91139/179487)
Branch Coverage44.64% (44974/100750)

@yiguolei
yiguolei merged commit 16b3e5f into apache:branch-2.1May 21, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@sollhui@doris-robot@hello-stephen@yiguolei