Generate query jumble automaticly by chenjinbao1989 · Pull Request #1827 · apache/cloudberry

chenjinbao1989 · 2026-06-22T21:45:09Z

Fixes #ISSUE_Number

What does this PR do?

Type of Change

Bug fix (non-breaking change)
New feature (non-breaking change)
Breaking change (fix or feature with breaking changes)
Documentation update

Breaking Changes

Test Plan

Unit tests added/updated
Integration tests added/updated
Passed make installcheck
Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Followed contribution guide
Added/updated documentation
Reviewed code for security implications
This PR contains AI-assisted code generation
Requested review from cloudberry committers

Additional Context

CI Skip Instructions

yjhjstz · 2026-06-24T02:08:34Z


 if (get_dirent_type(path, de, false, ERROR) != PGFILETYPE_LNK)
- ereport(allow_in_place_tablespaces ? WARNING : PANIC,
+ ereport(allow_in_place_tablespaces ? WARNING : WARNING,


PANIC is need to abort transcation.

This check is intended to restrict the internal tablespace to testing purposes only, but throwing a panic directly is too strict.

how about use ERROR?

yjhjstz · 2026-06-24T02:11:55Z

+ * code that executes the command in't prepared for a NULL.
+ */
+	if (conninfo == NULL)
+ conninfo = pstrdup("");


why not elog(ERROR) here ?

In this context, "" is considered a valid value. Therefore, we only need to convert the format.

yjhjstz · 2026-06-24T02:16:17Z


 my $last_nodetag = 'WindowObjectData';
-my $last_nodetag_no = 557;
+my $last_nodetag_no = 584;


how to decide last_nodetag_no?

Although gen_node_support.pl calculates this value automatically, manual modification is still required. I don't quite understand why it was designed this way.

leborchuk

Yeah, let's clear the mess. We have 2 functions with the same name:

src/backend/nodes/queryjumblefuncs.c:JumbleQuery(Query *query)
src/backend/utils/misc/queryjumble.c:static void JumbleQueryInternal(JumbleState *jstate, Query *query);

Should use only one - those which was moved to the src/backend/nodes

leborchuk · 2026-07-01T08:54:07Z

It will be great to add good PR description there. I asked Copilot for review changes, one could get the base description from Copilot output

Copilot

Pull request overview

This PR refactors Cloudberry’s node infrastructure to generate query jumble and node support code automatically (including nodetag generation and updated node read/write/copy behavior), and updates regression expected files to match the new serialized node output.

Changes:

Move/modernize query jumble interfaces under src/include/nodes/ and switch callers to the updated API.
Extend gen_node_support.pl + build rules to generate/ship nodetags.h and additional generated node support outputs (including query-jumble support).
Update multiple node struct definitions/attributes and refresh regression expected outputs accordingly.

Reviewed changes

Copilot reviewed 47 out of 51 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/test/singlenode_regress/expected/partition.out	Updates expected partition template serialization output.
src/test/regress/expected/xml.out	Adjusts expected XML serialization formatting.
src/test/regress/expected/partition_append.out	Updates expected partition template serialization output.
src/test/regress/expected/expressions.out	Updates expected output for `NOT IN (...) NULL` semantics.
src/include/utils/queryenvironment.h	Adds node attribute annotation for generated support behavior.
src/include/parser/analyze.h	Switches queryjumble include to the new header path.
src/include/nodes/tidbitmap.h	Refactors bitmap word array sizing constant.
src/include/nodes/queryjumble.h	Moves/updates query jumble public header and API surface.
src/include/nodes/primnodes.h	Updates node attributes / renames anonymous structs.
src/include/nodes/plannodes.h	Adds node attributes for generated copy/read/write behavior.
src/include/nodes/pathnodes.h	Adds/adjusts node attributes for generated support behavior.
src/include/nodes/parsenodes.h	Adds/adjusts node attributes and a few struct typedef names.
src/include/nodes/nodes.h	Replaces large inline NodeTag enum with generated `nodetags.h` include.
src/include/nodes/altertablenodes.h	Adds node attributes for generated support behavior.
src/include/Makefile	Installs generated `nodes/nodetags.h`; cleans generated node headers.
src/include/executor/execdesc.h	Adds node attributes / renames anonymous struct typedef.
src/include/commands/explain_gp.h	New header extracting EXPLAIN GP structs.
src/include/catalog/heap.h	Adds read/write-ignore defaults via node attributes.
src/include/access/tupdesc.h	Renames typedef + adds node attributes.
src/backend/utils/misc/queryjumble.c	Removes legacy query jumble implementation.
src/backend/utils/misc/Makefile	Stops building the removed `queryjumble.o`.
src/backend/utils/misc/guc_tables.c	Updates includes to new queryjumble header location.
src/backend/postmaster/postmaster.c	Updates includes to new queryjumble header location.
src/backend/parser/analyze.c	Switches to new `JumbleQuery()` call signature.
src/backend/nodes/read.c	Changes error severity for token parsing failure.
src/backend/nodes/nodeFuncs.c	Adjusts mutator behavior for certain node tags.
src/backend/nodes/Makefile	Adds generated-header rules and new node support objects.
src/backend/nodes/gen_node_support.pl	Major generator extensions (out/read fast + queryjumble generation).
src/backend/nodes/copyfuncs.switch.c	Removes generated switch file (now regenerated differently).
src/backend/nodes/copyfuncs.c	Adjusts pointer-copy macro + adds custom copy routines.
src/backend/nodes/.gitignore	Ignores newly generated node-support artifacts.
src/backend/Makefile	Re-enables node header generation step.
src/backend/executor/execProcnode.c	Removes references to a no-longer-present planstate tag.
src/backend/executor/execAmi.c	Removes references to a no-longer-present planstate tag.
src/backend/commands/subscriptioncmds.c	Normalizes `conninfo` to avoid NULL vs empty-string ambiguity.
src/backend/commands/explain.c	Switches to new `JumbleQuery()` call signature.
src/backend/commands/explain_gp.c	Uses new extracted header for EXPLAIN GP stats structs.
src/backend/access/transam/xlogrecovery.c	Changes severity handling for unexpected `pg_tblspc/` entries.
gpcontrib/pg_hint_plan/pg_hint_plan.c	Switches to new `JumbleQuery()` call signature.
gpcontrib/gp_stats_collector/src/stat_statements_parser/pg_stat_statements_parser.c	Updates includes to new queryjumble header location.
contrib/pax_storage/src/test/regress/expected/xml.out	Mirrors XML expected-output change for pax_storage tests.
contrib/pax_storage/src/test/regress/expected/partition.out	Mirrors partition expected-output change for pax_storage tests.
contrib/pax_storage/src/test/regress/expected/partition_optimizer.out	Mirrors partition optimizer expected-output change for pax_storage tests.
contrib/pax_storage/src/test/regress/expected/expressions.out	Mirrors expressions expected-output change for pax_storage tests.
contrib/pax_storage/src/test/regress/expected/bfv_partition.out	Mirrors partition template serialization changes for BFV output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+ elsif($t eq 'bytea*')
+ {
+ print $ofaf 
+ "\tWRITE_BYTEA_FIELD($f);\n";
+ print $rfaf
+ "\tREAD_BYTEA_FIELD($f);\n"
+ unless $no_read;
+ }


+ if (defined $read_as_field)
+ {
+ print $rff "\tlocal_node->$f = $read_as_field;\n" unless $no_read;
+
+ # for out and read fast
+ print $rff "\tlocal_node->$f = $read_as_field;\n" unless $no_read;
+ }


 if (jumblequery)
 {
- jstate = JumbleQueryDirect(query, query_str);
+ jstate = JumbleQuery(query);


+	/*
+ * conninfo can be an empty string, but the serialization
+ * doesn't distinguish an empty string from NULL. The
+ * code that executes the command in't prepared for a NULL.


 else
 {
- elog(ERROR, "unrecognized token: \"%.*s\"", tok_len, token);
+ elog(PANIC, "unrecognized token: \"%.*s\"", tok_len, token);



 if (get_dirent_type(path, de, false, ERROR) != PGFILETYPE_LNK)
- ereport(allow_in_place_tablespaces ? WARNING : PANIC,
+ ereport(allow_in_place_tablespaces ? WARNING : WARNING,


+
+

tuhaihe · 2026-07-16T02:59:58Z

Hi, how about the progress? Could we start to resolve the comments and try to merge this PR for PG16 FIXME?

chenjinbao1989 added 5 commits June 22, 2026 10:59

Generage copy/equal/out/read functions in nodes
a1d61e9

Generate query jumble
98e9fbb

remove src/backend/utils/misc/queryjumble.c
9617a10

Fix some compile errors
48bbad7

test
d30025a

yjhjstz reviewed Jun 24, 2026
View reviewed changes

tuhaihe added this to Merge postgres 16 Jun 26, 2026

tuhaihe moved this to In Progress in Merge postgres 16 Jun 26, 2026

leborchuk approved these changes Jun 29, 2026
View reviewed changes

leborchuk requested a review from Copilot July 1, 2026 08:52

Copilotstarted reviewing on behalf of leborchuk July 1, 2026 09:02 View session

CopilotAI reviewed Jul 1, 2026
View reviewed changes

Navigation Menu

Uh oh!

Generate query jumble automaticly - #1827

Generate query jumble automaticly#1827
chenjinbao1989 wants to merge 5 commits into
apache:mainfrom
chenjinbao1989:generate_queryjumble

chenjinbao1989 commented Jun 22, 2026

Uh oh!

yjhjstzJun 24, 2026

Uh oh!

chenjinbao1989Jul 6, 2026

Uh oh!

yjhjstzJul 6, 2026

Uh oh!

yjhjstzJun 24, 2026

Uh oh!

chenjinbao1989Jul 5, 2026

Uh oh!

yjhjstzJun 24, 2026

Uh oh!

chenjinbao1989Jul 5, 2026

Uh oh!

leborchuk left a comment

Uh oh!

leborchuk commented Jul 1, 2026

Uh oh!

CopilotAI left a comment

Uh oh!

tuhaihe commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

chenjinbao1989 commented Jun 22, 2026

What does this PR do?

Type of Change

Breaking Changes

Test Plan

Impact

Checklist

Additional Context

CI Skip Instructions

Uh oh!

yjhjstzJun 24, 2026

Choose a reason for hiding this comment

Uh oh!

chenjinbao1989Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

yjhjstzJul 6, 2026

Choose a reason for hiding this comment

Uh oh!

yjhjstzJun 24, 2026

Choose a reason for hiding this comment

Uh oh!

chenjinbao1989Jul 5, 2026

Choose a reason for hiding this comment

Uh oh!

yjhjstzJun 24, 2026

Choose a reason for hiding this comment

Uh oh!

chenjinbao1989Jul 5, 2026

Choose a reason for hiding this comment

Uh oh!

leborchuk left a comment

Choose a reason for hiding this comment

Uh oh!

leborchuk commented Jul 1, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

tuhaihe commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants