Skip to content

Make call SchedulerServer::new once in ballista-scheduler process - #1537

Merged
alamb merged 3 commits into
apache:masterfrom
Ted-Jiang:issue_1531
Jan 13, 2022
Merged

Make call SchedulerServer::new once in ballista-scheduler process#1537
alamb merged 3 commits into
apache:masterfrom
Ted-Jiang:issue_1531

Conversation

@Ted-Jiang

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes#1531 .

avoid tokio::spawn(async move { state_clone.synchronize_job_status_loop(). multi-times

@Ted-Jiang

Copy link
Copy Markdown
MemberAuthor

@houqp@alamb Plz have a review.

@houqp
houqp requested review from alamb and andygroveJanuary 11, 2022 03:40
Comment threadballista/rust/scheduler/src/main.rs Outdated
"Ballista v{} Scheduler listening on {:?}",
BALLISTA_VERSION, addr
);
//there should be only one SchedulerServer in process

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how this code ensures just one SchedulerServer exists per process.

It seems to me what this code does is create a new SchedulerServer on a port that the operating system picks.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps using something like lazy_static (https://docs.rs/lazy_static/1.4.0/lazy_static/, already a dependency) might be worth looking into

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, sorry for my wrong annotation. it should be only call SchedulerServer::new() once in process ,
is this right ?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its ok to clone() SchedulerServer in multi-thread without call tokio::spawn(async move { state_clone.synchronize_job_status_loop().await });

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SchedulerServer::new creates the scheduler loop that runs in the background, that's the thing that's causing the issue.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb Could you review it again, Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry -- I don't really know what this code is doing, so I just had an uninformed question; Sorry if this delayed merging unnecessarily.

@Ted-Jiang
Ted-Jiang requested a review from alambJanuary 12, 2022 13:53
@alamb

Copy link
Copy Markdown
Contributor

If it fixes your issues @Ted-Jiang then it sounds good to me 👍

@alamb
alamb merged commit cf76969 into apache:masterJan 13, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ballista scheduler dead loop in loadtest

3 participants

@Ted-Jiang@alamb@houqp