Skip to content

[fix](job-manager) cancelTaskById should not be blocked by unrelated streaming jobs - #62940

Merged
liaoxin01 merged 1 commit into
apache:masterfrom
JNSimba:fix/job-scheduler-streaming-type-handling
May 14, 2026
Merged

[fix](job-manager) cancelTaskById should not be blocked by unrelated streaming jobs#62940
liaoxin01 merged 1 commit into
apache:masterfrom
JNSimba:fix/job-scheduler-streaming-type-handling

Conversation

@JNSimba

@JNSimbaJNSimba commented Apr 29, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Problem Summary:

JobManager.cancelTaskById rejects all CANCEL TASK calls when any streaming job exists in jobMap. The streaming-type check is placed before the jobName match, so it fires on every iteration:

for (Tjob : jobMap.values()) {
if (job.getJobConfig().getExecuteType().equals(JobExecuteType.STREAMING)) {
thrownewJobException("streaming job not support cancel task by id");
}
if (job.getJobName().equals(jobName)) { ... }
}

If a single streaming job exists in jobMap, CANCEL TASK FOR <any-non-streaming-job> throws "streaming job not support cancel task by id" before ever matching the actual target.

Fix

Move the streaming-type check inside the jobName match, so it only fires when the matched job is actually a streaming job.

Release note

Fix CANCEL TASK on non-streaming jobs incorrectly rejected when an unrelated streaming job exists in the job map.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes. CANCEL TASK on non-streaming jobs is no longer rejected when an unrelated streaming job exists in the job map.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document

@JNSimba

Copy link
Copy Markdown
MemberAuthor

/review

@JNSimba

Copy link
Copy Markdown
MemberAuthor

run buildall

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two STREAMING-vs-RECURRING behavioral inconsistencies in the FE job scheduling layer: resuming STREAMING jobs now triggers an immediate scheduler refresh (like RECURRING), and CANCEL TASK ... is no longer incorrectly rejected due to unrelated streaming jobs present in the job map.

Changes:

  • Allow JobScheduler.cycleTimerJobScheduler(T job) to proceed for both RECURRING and STREAMING execute types.
  • Fix JobManager.cancelTaskById to only reject cancel-by-id when the matched job is STREAMING (instead of rejecting if any streaming job exists).
  • Add a unit test to cover the cancelTaskById regression.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

FileDescription
fe/fe-core/src/test/java/org/apache/doris/job/manager/JobManagerTest.javaAdds unit test ensuring cancel-by-id for a RECURRING job is not blocked by an unrelated STREAMING job.
fe/fe-core/src/main/java/org/apache/doris/job/scheduler/JobScheduler.javaExtends the cycle scheduler gate to include STREAMING jobs.
fe/fe-core/src/main/java/org/apache/doris/job/manager/JobManager.javaMoves STREAMING-type rejection inside the jobName match to avoid false rejections.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadfe/fe-core/src/main/java/org/apache/doris/job/scheduler/JobScheduler.java Outdated
@JNSimba
JNSimbaforce-pushed the fix/job-scheduler-streaming-type-handling branch from cf32b25 to 36723fdCompareApril 29, 2026 09:20
@JNSimbaJNSimba changed the title [fix](job) handle streaming type in cycleTimerJobScheduler and cancelTaskById[fix](job-manager) cancelTaskById should not be blocked by unrelated streaming jobsApr 29, 2026
@JNSimba

Copy link
Copy Markdown
MemberAuthor

run buildall

@JNSimba

Copy link
Copy Markdown
MemberAuthor

run cloud_p0

@JNSimba

Copy link
Copy Markdown
MemberAuthor

run feut

@JNSimba

Copy link
Copy Markdown
MemberAuthor

/review

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review result: no blocking issues found.

Critical checkpoint conclusions:

  • Goal and proof: The change correctly narrows the STREAMING rejection in JobManager.cancelTaskById to the matched job name, fixing cancellation of non-streaming jobs when unrelated streaming jobs exist. The added unit test covers both the fixed path and the preserved streaming-job rejection.
  • Scope: The modification is small and focused on the incorrect conditional placement.
  • Concurrency: jobMap is a ConcurrentHashMap; this PR does not introduce new shared state, locking, or lock-order changes. The iteration behavior remains consistent with existing lookup-by-name methods.
  • Lifecycle/static initialization: No lifecycle, persistence format, or static initialization changes.
  • Configuration/compatibility/protocol: No new configuration, serialization format, FE-BE protocol, or rolling-upgrade compatibility concern.
  • Parallel paths: The changed path is the generic scheduler-job cancel-by-task-id path used by CancelJobTaskCommand; MTMV has a separate manager path and is not affected by this bug fix.
  • Conditional checks: The moved STREAMING check now applies only after the target job is identified, which matches the intended error condition.
  • Tests: A focused FE unit test was added. I did not run it locally because thirdparty/installed/bin/protoc is absent in this runner, and FE build/test instructions require stopping when that prerequisite is missing.
  • Observability: No new observability is needed for this narrow control-flow fix.
  • Transactions/persistence/data correctness: No transaction visibility, edit-log, storage, or data-write semantics are changed beyond the existing logUpdateOperation() after successful cancellation.
  • Performance: No meaningful performance regression; the loop and complexity remain unchanged.

Existing review context: I saw the prior inline thread about duplicate scheduling in JobScheduler.java; this PR's current diff does not modify that file, and I did not re-raise that already-known concern.

User focus: No additional user-provided review focus was specified.

@liaoxin01liaoxin01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liaoxin01
liaoxin01 merged commit a789c99 into apache:masterMay 14, 2026
33 checks passed
github-actionsBot pushed a commit that referenced this pull request May 14, 2026
…streaming jobs (#62940)
### What problem does this PR solve?
Problem Summary:
`JobManager.cancelTaskById` rejects all `CANCEL TASK` calls when any
streaming job exists in `jobMap`. The streaming-type check is placed
before the `jobName` match, so it fires on every iteration:
```java
for (T job : jobMap.values()) {
if (job.getJobConfig().getExecuteType().equals(JobExecuteType.STREAMING)) {
throw new JobException("streaming job not support cancel task by id");
}
if (job.getJobName().equals(jobName)) { ... }
}
```
If a single streaming job exists in `jobMap`, `CANCEL TASK FOR
<any-non-streaming-job>` throws "streaming job not support cancel task
by id" before ever matching the actual target.
### Fix
Move the streaming-type check inside the `jobName` match, so it only
fires when the matched job is actually a streaming job.
### Release note
Fix `CANCEL TASK` on non-streaming jobs incorrectly rejected when an
unrelated streaming job exists in the job map.
JNSimba added a commit that referenced this pull request May 19, 2026
…streaming jobs (#62940)
### What problem does this PR solve?
Problem Summary:
`JobManager.cancelTaskById` rejects all `CANCEL TASK` calls when any
streaming job exists in `jobMap`. The streaming-type check is placed
before the `jobName` match, so it fires on every iteration:
```java
for (T job : jobMap.values()) {
if (job.getJobConfig().getExecuteType().equals(JobExecuteType.STREAMING)) {
throw new JobException("streaming job not support cancel task by id");
}
if (job.getJobName().equals(jobName)) { ... }
}
```
If a single streaming job exists in `jobMap`, `CANCEL TASK FOR
<any-non-streaming-job>` throws "streaming job not support cancel task
by id" before ever matching the actual target.
### Fix
Move the streaming-type check inside the `jobName` match, so it only
fires when the matched job is actually a streaming job.
### Release note
Fix `CANCEL TASK` on non-streaming jobs incorrectly rejected when an
unrelated streaming job exists in the job map.
yiguolei pushed a commit that referenced this pull request May 20, 2026
…y unrelated streaming jobs #62940 (#63262)
Cherry-picked from #62940
---------
Co-authored-by: wudi <wudi@selectdb.com>
zhaorongsheng pushed a commit to zhaorongsheng/doris that referenced this pull request Jun 4, 2026
…streaming jobs (apache#62940)
### What problem does this PR solve?
Problem Summary:
`JobManager.cancelTaskById` rejects all `CANCEL TASK` calls when any
streaming job exists in `jobMap`. The streaming-type check is placed
before the `jobName` match, so it fires on every iteration:
```java
for (T job : jobMap.values()) {
if (job.getJobConfig().getExecuteType().equals(JobExecuteType.STREAMING)) {
throw new JobException("streaming job not support cancel task by id");
}
if (job.getJobName().equals(jobName)) { ... }
}
```
If a single streaming job exists in `jobMap`, `CANCEL TASK FOR
<any-non-streaming-job>` throws "streaming job not support cancel task
by id" before ever matching the actual target.
### Fix
Move the streaming-type check inside the `jobName` match, so it only
fires when the matched job is actually a streaming job.
### Release note
Fix `CANCEL TASK` on non-streaming jobs incorrectly rejected when an
unrelated streaming job exists in the job map.
@yiguoleiyiguolei mentioned this pull request Jun 14, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@JNSimba@liaoxin01@yiguolei