[18.0] queue_job: refactor job acquisition

[sbidoul]

In this PR we cleanly separate the job acquisition (i.e. verifying the job is in the exepected state, marking it started and locking it) from job execution.

We also avoid trying to start the job if it is already locked by using SKIP LOCKED and exiting early. Indeed, in such situations, the job is likely already being handled by another worker so there is no point trying to start it, so we exit early and let it be handled either by the other worker or the dead job requeuer.

Following-up on #859 (comment)

maybe fixes#858

[OCA-git-bot]

Hi @guewen,
some modules you are maintaining are being modified, check this out!

[sbidoul]

Moved to JobCommonCase

[sbidoul]

This is ready to review.

It can be tested with a small number of workers and a root channel capacity greater than the number of workers and test jobs of duration >= 10 sec.

If the test graph has enough parallelism, you will see warnings about dead jobs being requeued (the jobs in state enqueued that are not picked up by workers in due time), but each job should execute only once.

Example: odoo --workers=3, root:10 and http://odoo/queue_job/create_test_job?size=20&job_duration=10.

The _acquire_job method is now simple enough that it can be unit tested. Next I may try refactoring the perform job part buy its a bit scarier :) But first, merging 19.0.

[hoangtrann]

Code LGTM

[hoangtrann]

I've also tested this locally and and it seems to work as intended

[sbidoul]

Thanks for the review and testing @hoangtrann !

I added a couple of commits to split out the job execution logic to class method so it is reusable from outside the controller and open the door to experiments such as #871.

[sbidoul]

/ocabot merge patch

[OCA-git-bot]

On my way to merge this fine PR!
Prepared branch 18.0-ocabot-merge-pr-866-by-sbidoul-bump-patch, awaiting test results.

[OCA-git-bot]

Congratulations, your PR was merged at 48f5707. Thanks a lot for contributing to OCA. ❤️