[18.0][IMP] queue_job: add index for efficient autovacuum

[florentx]

This query was part of the slowest.

Before:

# EXPLAIN ANALYZESELECT id FROM"queue_job"WHERE (("date_done"<='2025-07-01'OR"date_cancelled"<='2025-07-01') AND"channel"='root.custom.purchase')
ORDER BY"date_created"DESC, "date_done"DESCLIMIT1000;
QUERY PLAN ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------Limit (cost=212724.34..212841.01 rows=1000 width=20) (actual time=1949.277..1966.346 rows=1000 loops=1)
-> Gather Merge (cost=212724.34..220532.68 rows=66924 width=20) (actual time=1939.068..1956.102 rows=1000 loops=1)
Workers Planned: 2
Workers Launched: 2-> Sort (cost=211724.32..211807.97 rows=33462 width=20) (actual time=1880.715..1880.798 rows=779 loops=3)
Sort Key: date_created DESC, date_done DESC
Sort Method: top-N heapsort Memory: 179kB
Worker 0: Sort Method: top-N heapsort Memory: 180kB
Worker 1: Sort Method: top-N heapsort Memory: 183kB
-> Parallel Bitmap Heap Scan on queue_job (cost=944.61..209889.63 rows=33462 width=20) (actual time=16.823..1873.439 rows=27435 loops=3)
Recheck Cond: ((channel)::text='root.custom.purchase'::text)
Rows Removed by Index Recheck: 51603
Filter: ((date_done <='2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <='2025-07-01 00:00:00'::timestamp without time zone))
Rows Removed by Filter: 224
Heap Blocks: exact=13666 lossy=12625-> Bitmap Index Scan on queue_job_channel_index (cost=0.00..924.54 rows=83748 width=0) (actual time=13.932..13.932 rows=82979 loops=1)
Index Cond: ((channel)::text='root.custom.purchase'::text)
Planning Time: 1.395 ms
JIT:
Functions: 19
Options: Inlining false, Optimization false, Expressions true, Deforming true
Timing: Generation 4.315 ms, Inlining 0.000 ms, Optimization 3.396 ms, Emission 37.965 ms, Total 45.676 ms
Execution Time: 1968.386 ms
(23 rows)

---

After:

# EXPLAIN ANALYZESELECT id FROM"queue_job"WHERE (("date_done"<='2025-07-01'OR"date_cancelled"<='2025-07-01') AND"channel"='root.custom.purchase')
ORDER BY"date_created"DESC, "date_done"DESCLIMIT1000;
QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Limit (cost=0.43..66770.92 rows=1000 width=20) (actual time=0.750..93.965 rows=1000 loops=1)
-> Index Scan Backward using queue_job_date_created_date_done_index on queue_job (cost=0.43..5362271.99 rows=80309 width=20) (actual time=0.748..93.895 rows=1000 loops=1)
Filter: (((channel)::text='root.custom.purchase'::text) AND ((date_done <='2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <='2025-07-01 00:00:00'::timestamp without time zone)))
Rows Removed by Filter: 40686
Planning Time: 0.312 ms
Execution Time: 94.057 ms
(6 rows)

[OCA-git-bot]

Hi @guewen,
some modules you are maintaining are being modified, check this out!

[guewen]

Thanks @florentx!

The index is scanned backwards because the query sorts by DESC and the index goes ASC.
We could use an index on (date_created DESC, date_done DESC), but changing the ordering in the autovacuum's query to sort by ASC - aside of speeding up the index - would drop the oldest jobs to the newest jobs which actually makes more sense, what do you think?

[florentx]

You're right @guewen .

I've forced an order= argument, and changed the index.

# EXPLAIN ANALYZE SELECT id FROM"queue_job"WHERE (("date_done"<='2025-07-01'OR"date_cancelled"<='2025-07-01') AND"channel"='root.custom.purchase')
ORDER BY date_done, date_created LIMIT1000;
QUERY PLAN -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------Limit (cost=0.55..3756.69 rows=1000 width=20) (actual time=0.138..0.877 rows=1000 loops=1)
-> Index Scan using queue_job_channel_date_done_date_created_index on queue_job (cost=0.55..301651.84 rows=80309 width=20) (actual time=0.137..0.842 rows=1000 loops=1)
Index Cond: ((channel)::text='root.custom.purchase'::text)
Filter: ((date_done <='2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <='2025-07-01 00:00:00'::timestamp without time zone))
Planning Time: 0.200 ms
Execution Time: 0.918 ms
(6 rows)

[guewen]

Great!

[guewen]

/ocabot merge minor

[OCA-git-bot]

This PR looks fantastic, let's merge it!
Prepared branch 18.0-ocabot-merge-pr-798-by-guewen-bump-minor, awaiting test results.

[OCA-git-bot]

Congratulations, your PR was merged at 45dabc7. Thanks a lot for contributing to OCA. ❤️