Fix JOIN motion type selection for join quals containing outer refs

[reshke]

I found a planner bug while backporting 14.8 - 14.9 range in REL_2_STABLE #1843

In some cases planner failed to mark join qual restrict clauses as ones referring outer query levels. This makes join motion planning code to make wrong query: it uses redistribute motion in cases it shouldn't.
In this simple reproduces this leads to motion plan node rescan (this is execute-time ERROR normally). Note that for generate_series(1,1) t1 it would work, because our parametrized plan would execute only once.

CREATE TABLE t2(i int);
CREATE TABLE t3(i int);
INSERT INTO t2 SELECT generate_series(1,10);
INSERT INTO t3 SELECT generate_series(1,10);
select * from generate_series(1,2) t1, lateral (select t3.i from t2 join t3 on t2.i = t3.i + t1 order by 1) z;
ERROR: illegal rescan of motion node: invalid plan (nodeMotion.c:1368) (seg1 slice1 127.0.1.1:7003 pid=879693) (nodeMotion.c:1368)
HINT: Likely caused by bad NL-join, try setting enable_nestloop to off
reshke=# explain select * from generate_series(1,1) t1, lateral (select t3.i from t2 join t3 on t2.i = t3.i + t1 order by 1) z;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=10000623579.03..10000785868.61 rows=9273690 width=8)
-> Nested Loop (cost=10000623579.03..10000662219.41 rows=3091230 width=8)
-> Function Scan on generate_series t1 (cost=0.00..0.01 rows=1 width=4)
-> Materialize (cost=623579.02..646763.25 rows=3091230 width=4)
-> Sort (cost=623579.02..631307.10 rows=3091230 width=4)
Sort Key: t3.i
-> Hash Join (cost=756.25..290348.30 rows=3091230 width=4)
Hash Cond: ((t3.i + t1.t1) = t2.i)
-> Redistribute Motion 3:3 (slice2; segments: 3) (cost=0.00..997.00 rows=32100 width=4)
Hash Key: (t3.i + t1.t1)
-> Seq Scan on t3 (cost=0.00..355.00 rows=32100 width=4)
-> Hash (cost=355.00..355.00 rows=32100 width=4)
-> Seq Scan on t2 (cost=0.00..355.00 rows=32100 width=4)
Optimizer: Postgres query optimizer
(14 rows)

As we can see, Redistribute Motion uses Hash Key: (t3.i + t1.t1) which is parametrized by outer rel (t1).

With fix:

reshke=# explain select * from generate_series(1,1) t1, lateral (select t3.i from t2 join t3 on t2.i = t3.i + t1 order by 1) z;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=10000000117.94..10000000129.99 rows=963 width=8)
-> Function Scan on generate_series t1 (cost=0.00..0.01 rows=1 width=4)
-> Materialize (cost=117.94..125.16 rows=963 width=4)
-> Sort (cost=117.94..120.34 rows=963 width=4)
Sort Key: t3.i
-> Hash Join (cost=33.90..70.21 rows=963 width=4)
Hash Cond: (t2.i = (t3.i + t1.t1))
-> Materialize (cost=0.00..21.87 rows=963 width=4)
-> Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..17.05 rows=963 width=4)
-> Seq Scan on t2 (cost=0.00..4.21 rows=321 width=4)
-> Hash (cost=21.87..21.87 rows=963 width=4)
-> Materialize (cost=0.00..21.87 rows=963 width=4)
-> Gather Motion 3:1 (slice2; segments: 3) (cost=0.00..17.05 rows=963 width=4)
-> Seq Scan on t3 (cost=0.00..4.21 rows=321 width=4)
Optimizer: Postgres query optimizer
(15 rows)

[reshke]

@yjhjstz maybe interesting for you

[yjhjstz]

The Nested Loop sits directly above the Gather Motion 3:1, so the whole join runs serially on the coordinator (QD) instead of segment-local. Is that intended, or should the join happen below the Motion so it stays parallel across segments?

[reshke]

Yep this is intended. The before-plan is better in term of performance, expect it is not valid ;). We receive executor-time error becuase of motion rescan. The after-fix plan is worse, but can be executed correctly. In fact, the sole thing this PR do is correctly use infrastructure committed at 00e25afe119c

[reshke]

in fact, I think pushing down join below the motion is possible, so some types of plans. But this is separate problem, in this PR I merely try to fix ERROR: illegal rescan of motion node:

[yjhjstz]

OK，add a fixme for the plan.

[reshke]

I think these two link can help in understanding

https://github.com/apache/cloudberry/blob/main/src/backend/cdb/cdbpath.c#L1417-L1421

https://github.com/apache/cloudberry/blob/main/src/backend/cdb/cdbpath.c#L1585-L1594

[reshke]

Fixme added