Skip to content

YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned on - #8686

Open
susheelgupta7 wants to merge 3 commits into
apache:trunkfrom
susheelgupta7:YARN-11985
Open

YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned on#8686
susheelgupta7 wants to merge 3 commits into
apache:trunkfrom
susheelgupta7:YARN-11985

Conversation

@susheelgupta7

@susheelgupta7susheelgupta7 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Contains content generated by Cursor.

Description of PR

When creating a placement rule in the CS UI, the parent queue dropdown did not show queues that had dynamic queue creation enabled but no static child queues.
Now a queue is considered a parent if it either has child queues or has dynamic queue creation enabled.

How was this patch tested?

Tested manually on the cluster.

For code changes:

  • Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: Have the integration tests been executed and the endpoint
    declared according to the connector-specific documentation? Note: Automated CI
    testing doesn't cover all cases so manual testing with cloud storage is still
    required.
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

…ent although Dynamic Queue Creation is turned on
@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 34sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚mvninstall41m 10strunk passed
+1 💚shadedclient69m 46sbranch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚mvninstall1m 27sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚shadedclient27m 26spatch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚asflicense0m 46sThe patch does not generate ASF License warnings.
101m 57s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/1/artifact/out/Dockerfile
GITHUB PR#8686
Optional Testsdupname asflicense shadedclient codespell detsecrets
unameLinux 08ead813ddc7 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/bin/hadoop.sh
git revisiontrunk / 8a94d52
Max. process+thread count633 (vs. ulimit of 10000)
modulesC: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console outputhttps://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/1/console
versionsgit=2.43.0 maven=3.9.15
Powered byApache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@slfan1989slfan1989 changed the title YARN-11985: Placement rule does not allow to select Leaf Queue as Par…YARN-11985: Placement rule does not allow to select Leaf Queue as Parent although Dynamic Queue Creation is turned onAug 15, 2026

@slfan1989slfan1989 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on YARN-11985. The overall approach makes sense and allows queues with Dynamic Queue Creation enabled to be selected as placement-rule parents even when they do not currently have child queues.

I found one issue where the live autoCreationEligibility takes precedence over a staged disable, which can leave an invalid queue in the parent dropdown. Please address the inline comment and add focused unit tests for live legacy/flexible eligibility, staged enable, and staged disable.

eligibility === AUTO_CREATION_PROPS.ELIGIBILITY_FLEXIBLE ||
eligibility === AUTO_CREATION_PROPS.ELIGIBILITY_LEGACY
) {
return true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This early return prevents the staged configuration from taking effect. If the queue is currently flexible, but the user stages both auto-creation properties to false, the queue will still be included in the parent queue dropdown.

Could we read both values from getQueuePropertyValue first and let staged values take precedence, falling back to autoCreationEligibility only when neither property has a staged override?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @slfan1989 for the review. I’ve fixed the precedence issue.

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec12m 53sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚mvninstall40m 48strunk passed
+1 💚shadedclient69m 33sbranch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚mvninstall1m 28sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚shadedclient27m 26spatch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚asflicense0m 45sThe patch does not generate ASF License warnings.
114m 3s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/2/artifact/out/Dockerfile
GITHUB PR#8686
Optional Testsdupname asflicense shadedclient codespell detsecrets
unameLinux 22ffbaf36e04 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/bin/hadoop.sh
git revisiontrunk / a27c2f0
Max. process+thread count613 (vs. ulimit of 10000)
modulesC: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console outputhttps://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/2/console
versionsgit=2.43.0 maven=3.9.15
Powered byApache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@K0K0V0K

Copy link
Copy Markdown
Contributor

Thanks @susheelgupta7 for the fix!

FYI, the fix can be also validated with the built in dev server and mock data.
Before the change the root.marketing.test queue is not visible on the Add Placement Rule view, but after the change it is visible.

LGTM

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 34sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚mvninstall41m 58strunk passed
+1 💚shadedclient70m 36sbranch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚mvninstall1m 25sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚shadedclient27m 24spatch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌asflicense0m 45s/results-asflicense.txtThe patch generated 1 ASF License warnings.
102m 31s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/3/artifact/out/Dockerfile
GITHUB PR#8686
Optional Testsdupname asflicense shadedclient codespell detsecrets
unameLinux ac1c53ffb1ad 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/bin/hadoop.sh
git revisiontrunk / b8ff710
Max. process+thread count639 (vs. ulimit of 10000)
modulesC: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console outputhttps://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8686/3/console
versionsgit=2.43.0 maven=3.9.15
Powered byApache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@susheelgupta7@hadoop-yetus@K0K0V0K@slfan1989@brumi1024