Skip to content

branch-4.0: [fix](job) set cluster when routine load job replay from image #59135 - #59163

Merged
yiguolei merged 1 commit into
branch-4.0from
auto-pick-59135-branch-4.0
Dec 23, 2025
Merged

branch-4.0: [fix](job) set cluster when routine load job replay from image #59135#59163
yiguolei merged 1 commit into
branch-4.0from
auto-pick-59135-branch-4.0

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Cherry-picked from #59135

@Thearas

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Thearas

Copy link
Copy Markdown
Contributor

run buildall

### What problem does this PR solve?
routine load job pause and can not resume when update:
```
mysql> show routine load\G;
*************************** 1. row ***************************
Id: 1765800763597
Name: lineitem_dup_persistent_label
CreateTime: 2025-12-15 20:18:51
PauseTime: 2025-12-15 20:33:34
EndTime: NULL
DbName: regression_test_stress_load_release_routine_load
TableName: lineitem_dup_persistent
IsMultiTable: false
State: PAUSED
DataSourceType: KAFKA
CurrentTaskNum: 0
JobProperties: {"max_batch_rows":"300000","timezone":"Asia/Shanghai","send_batch_parallelism":"1","load_to_single_tablet":"false","column_separator":"','","line_delimiter":"\n","delete":"*","current_concurrent_number":"96","partial_columns":"false","merge_type":"APPEND","exec_mem_limit":"2147483648","strict_mode":"false","jsonpaths":"","max_batch_interval":"20","max_batch_size":"209715200","fuzzy_parse":"false","escape":"0","enclose":"0","partitions":"*","columnToColumnExpr":"","whereExpr":"*","desired_concurrent_number":"256","precedingFilter":"*","format":"csv","max_error_number":"0","max_filter_ratio":"1.0","sequence_col":"*","json_root":"","strip_outer_array":"false","num_as_string":"false"}
DataSourceProperties: {"topic":"test-release-topic-persistent690131793","currentKafkaPartitions":"0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95","brokerList":"172.20.48.94:9092"}
CustomProperties: {"kafka_default_offsets":"OFFSET_BEGINNING","group.id":"test-consumer-group","client.id":"test-client-id"}
Statistic: {"receivedBytes":955557901,"runningTxns":[],"errorRows":0,"committedTaskNum":0,"loadedRows":5000000,"loadRowsRate":5657,"abortedTaskNum":0,"errorRowsAfterResumed":0,"totalRows":5000000,"unselectedRows":0,"receivedBytesRate":1081251,"taskExecuteTimeMs":883752}
Progress: {"0":"50620","1":"55664","2":"51135","3":"52169","4":"55413","5":"55355","6":"50806","7":"52994","8":"51368","9":"50639","10":"52749","11":"54719","12":"51991","13":"52286","14":"53822","15":"52359","16":"50858","17":"55055","18":"52334","19":"54346","20":"48332","21":"53804","22":"50136","23":"52355","24":"50488","25":"53128","26":"53798","27":"54857","28":"54381","29":"51395","30":"53017","31":"51399","32":"49111","33":"51818","34":"52621","35":"52366","36":"51532","37":"54548","38":"53237","39":"54158","40":"48448","41":"48682","42":"48340","43":"53605","44":"50907","45":"58149","46":"52260","47":"55176","48":"51573","49":"52380","50":"52152","51":"52939","52":"52225","53":"52897","54":"51117","55":"53323","56":"51467","57":"51690","58":"52769","59":"50265","60":"54767","61":"54841","62":"50346","63":"48053","64":"56053","65":"53854","66":"54136","67":"48582","68":"53609","69":"50954","70":"55356","71":"51986","72":"45904","73":"50689","74":"57569","75":"49008","76":"51408","77":"50222","78":"50540","79":"55616","80":"50095","81":"52915","82":"49424","83":"50399","84":"49945","85":"50468","86":"49964","87":"50477","88":"50415","89":"54023","90":"49280","91":"49895","92":"51864","93":"48190","94":"53446","95":"52084"}
Lag: {"0":5671,"1":0,"2":0,"3":0,"4":0,"5":0,"6":0,"7":0,"8":0,"9":0,"10":0,"11":0,"12":0,"13":0,"14":0,"15":0,"16":0,"17":0,"18":0,"19":0,"20":0,"21":0,"22":0,"23":0,"24":0,"25":0,"26":0,"27":0,"28":0,"29":0,"30":0,"31":0,"32":0,"33":0,"34":0,"35":0,"36":0,"37":0,"38":0,"39":0,"40":0,"41":0,"42":0,"43":0,"44":0,"45":0,"46":0,"47":0,"48":0,"49":0,"50":0,"51":0,"52":0,"53":0,"54":0,"55":1,"56":0,"57":0,"58":0,"59":0,"60":0,"61":0,"62":0,"63":0,"64":0,"65":0,"66":0,"67":0,"68":0,"69":0,"70":0,"71":0,"72":0,"73":0,"74":0,"75":0,"76":0,"77":0,"78":0,"79":0,"80":0,"81":0,"82":0,"83":0,"84":0,"85":0,"86":0,"87":0,"88":0,"89":0,"90":0,"91":0,"92":0,"93":0,"94":0,"95":0}
ReasonOfStateChanged: ErrorReason{code=errCode = 103, msg='failed to allocate task: Cannot invoke "Object.hashCode()" because "key" is null'}
ErrorLogUrls: OtherMsg: failed to allocate task: Cannot invoke "Object.hashCode()" because "key" is null
User: root
Comment: 1 row in set (0.01 sec)
```
introduced by #52911, root cause is
missing setting cluster when routine load job replay from image.
@yiguolei
yiguoleiforce-pushed the auto-pick-59135-branch-4.0 branch from 661af49 to 6c1af94CompareDecember 19, 2025 06:18
@yiguolei

Copy link
Copy Markdown
Contributor

run buildall

@yiguolei
yiguolei merged commit 3011f24 into branch-4.0Dec 23, 2025
24 of 29 checks passed
@github-actions
github-actionsBot deleted the auto-pick-59135-branch-4.0 branch December 23, 2025 10:14
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Thearas@yiguolei@dataroaring@sollhui