Custom config handler #6508

[ccoffline]

Proposed changes

Support custom config handler callback and types.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)
• Code refactor (Modify the code structure, format the code, etc...)
• Optimization. Including functional usability improvements and performance improvements.
• Dependency. Such as changes related to third-party components.
• Other.

Checklist

• I have created an issue on (Fix[Feature] Custom config handler, support callback when set mutable config at FE and BE #6508) and described the bug/feature there in detail
• Compiling and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• If these changes need document changes, I have updated the document
• Any dependent changes have been merged

[morningman]

Could your explain more about query traffic limit in your issue #6508 ?

[ccoffline]

Could your explain more about query traffic limit in your issue #6508 ?

I didn't put query traffic limit in this pr. This is still experimental on one of our clusters.

Some users want to limit query more flexibly than maxQueryInstances#6159. For example, all query should be rejected when queryInstances of this user reach 30000, and big queries that queryInstances more than 1000 should be rejected when queryInstances of this user reach 20000. Limiting rules like this will be complex and need to be modified at any time.

So we add a config that support this feature, using SpEL to evaluate whether to limit a query or not. The limiting expressions will be reset at any time, and the Expression needed to be compile as soon as setting the config. This is what this pr for, I think more configs will need callback in the future.

We are using 0.13 and #6159 is not supported yet. In master, there is a similar feature sql block rule#6192 which can support query traffic limit, maybe add a limit property. So I won't push our traffic limit pr to master, but just add config handler to support config setting callback.

[ccoffline]

I just looked into #6192 and sql block rule cannot support traffic limit, but I think traffic limit is better set in the user properties.

[morningman]

LGTM

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.