[SQL] Alter Catalog and Table statements by ahmedabu98 · Pull Request #36571 · apache/beam

ahmedabu98 · 2025-10-20T20:19:06Z

Addresses #35637

Adds the following features:

Modify a catalog's properties

ALTER CATALOG my_catalog [ RESET ('prop1', 'prop2', ...) ] [ SET ('prop5'='val5', 'prop6'='val6', ...) ]

Modify a table's properties, schema, or partition spec

ALTERTABLE my_table
[ DROP COLUMNS (col1, col2, ...) ]
[ ADD COLUMNS (col5 VARCHAR COMMENT 'some-comment', col6 INTEGER, ...) ]
[ DROP PARTITIONS ('col1', 'month(col4)', ...) ]
[ ADD PARTITIONS ('truncate(col5, 3)', 'col4', ...) ]
[ RESET ('prop1', 'prop2', ...) ] [ SET ('prop5'='val5', 'prop6'='val6', ...) ]

Limitations:

Does not support adding a required/non-null column to an Iceberg table (introduced in V3). Iceberg requires such a column to specify a default value (so that existing rows can adapt that value to their new column), but Beam Schemas do not support a "default value"

github-actions · 2025-11-11T20:37:57Z

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

ahmedabu98 · 2025-11-13T22:54:52Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces support for ALTER CATALOG and ALTER TABLE statements in Beam SQL, which is a significant feature addition. The changes are extensive, spanning the SQL parser, DDL execution logic, and the Iceberg IO connector. The implementation is well-structured and accompanied by a comprehensive set of tests for the new functionality.

My review has identified a functional bug in the unparse logic for ALTER TABLE, a potential thread-safety issue in the Iceberg catalog configuration, and an opportunity for code simplification by removing duplicated logic. After addressing these points, this pull request will be in excellent shape.

github-actions · 2025-11-21T12:15:52Z

Reminder, please take a look at this pr: @Abacn

Abacn · 2025-11-21T20:39:40Z

assign to next reviewer

github-actions · 2025-11-21T20:40:48Z

Assigning reviewers:

R: @kennknowles for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions · 2025-11-29T12:14:39Z

Reminder, please take a look at this pr: @kennknowles

github-actions · 2025-12-03T12:17:21Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

…r_sql

ahmedabu98 · 2025-12-10T16:38:59Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces ALTER CATALOG and ALTER TABLE statements to Beam SQL, enhancing DDL capabilities. The changes include new parser rules, DDL node classes, and interface methods to support these operations. Specific implementations are provided for Iceberg catalogs and tables, as well as a test table provider. The new functionality allows modifying catalog properties, and for tables, it supports updating properties, schema (adding/dropping columns), and partition specifications. Comprehensive unit tests have been added to validate the new features, including a specific test for the limitation of not adding required columns to existing Iceberg tables. The code is well-structured and follows good practices for extensibility.

github-actions · 2025-12-18T12:17:29Z

Reminder, please take a look at this pr: @Abacn

github-actions · 2025-12-22T12:17:58Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions · 2025-12-29T12:18:05Z

Reminder, please take a look at this pr: @chamikaramj

github-actions · 2026-01-01T12:17:31Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions · 2026-01-08T12:18:42Z

Reminder, please take a look at this pr: @Abacn

github-actions · 2026-01-13T12:18:26Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions · 2026-01-20T12:19:31Z

Reminder, please take a look at this pr: @kennknowles

github-actions · 2026-01-22T12:19:59Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions · 2026-01-29T12:22:49Z

Reminder, please take a look at this pr: @chamikaramj

github-actions · 2026-02-02T12:24:20Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

Abacn

Thanks!

Abacn · 2026-02-02T19:48:33Z

+import org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.ImmutableList;
+import org.checkerframework.checker.nullness.qual.Nullable;
+
+public class SqlAlterCatalog extends SqlAlter implements BeamSqlParser.ExecutableStatement {


Just a question - these DDLs are added in the main sql module. Are they now supported by Beam SQL in general, or is just supported for Icebeg Catalog and Tables?

This class just funnels the properties down to the TableProvider implementation.
Currently, only the InMemory provider and Iceberg providers support Alter operations. Other providers will throw an UnsupportedOperationException

codecov · 2026-02-04T19:58:21Z

Codecov Report

❌ Patch coverage is 77.97834% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 36.33%. Comparing base (0274331) to head (9fd7054).
⚠️ Report is 234 commits behind head on master.

Files with missing lines	Patch %	Lines
.../sdk/extensions/sql/impl/parser/SqlAlterTable.java	73.33%	19 Missing and 13 partials ⚠️
...dk/extensions/sql/impl/parser/SqlAlterCatalog.java	74.60%	14 Missing and 2 partials ⚠️
...ions/sql/meta/provider/test/AlterTestTableOps.java	88.09%	3 Missing and 2 partials ⚠️
...am/sdk/extensions/sql/impl/parser/SqlDdlNodes.java	82.60%	0 Missing and 4 partials ⚠️
...dk/extensions/sql/meta/provider/TableProvider.java	0.00%	2 Missing ⚠️
...k/extensions/sql/meta/store/InMemoryMetaStore.java	50.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@## master #36571 +/- ##
============================================
- Coverage 40.38% 36.33% -4.05% + Complexity 3476 1690 -1786 
============================================
Files 1226 1066 -160 Lines 188553 167215 -21338 Branches 3607 1227 -2380 ============================================
- Hits 76138 60758 -15380 + Misses 109012 104259 -4753 + Partials 3403 2198 -1205

Flag	Coverage Δ
java	`67.32% <77.97%> (-3.33%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Abacn

Thanks!

alter catalog
86f0896

github-actionsBot added java extensions sql labels Oct 20, 2025

alter table
e6d1f3e

github-actionsBot added the io label Nov 11, 2025

ahmedabu98 added 2 commits November 11, 2025 21:50

spotless
b87867e

spotless
717ef9c

ahmedabu98 marked this pull request as ready for review November 11, 2025 20:12

github-actionsBot added the Next Action: Reviewers label Nov 11, 2025

gemini-code-assistBot reviewed Nov 13, 2025
View reviewed changes

github-actionsBot added the slow-review label Nov 21, 2025

github-actionsBot removed the slow-review label Nov 21, 2025

github-actionsBot added the slow-review label Nov 29, 2025

github-actionsBot added reassigned-reviewers and removed slow-review labels Dec 3, 2025

ahmedabu98 mentioned this pull request Dec 9, 2025
[Task]: Polish Beam SQL user experience #35637
Closed
17 tasks

ahmedabu98 added 4 commits December 9, 2025 14:26

fix unparse method, cleanup
35548c8

Merge branch 'master' ofhttps://github.com/ahmedabu98/beaminto alte…
5e9f462
…r_sql

add more tests
cbe071b

add deps
47c6934

gemini-code-assistBot reviewed Dec 10, 2025
View reviewed changes

Comment thread...src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/iceberg/IcebergMetastore.java Outdated

github-actionsBot added the slow-review label Dec 18, 2025

github-actionsBot removed the slow-review label Dec 22, 2025

github-actionsBot added the slow-review label Dec 29, 2025

github-actionsBot removed the slow-review label Jan 1, 2026

github-actionsBot added the slow-review label Jan 8, 2026

github-actionsBot removed the slow-review label Jan 13, 2026

github-actionsBot added the slow-review label Jan 20, 2026

github-actionsBot removed the slow-review label Jan 22, 2026

github-actionsBot added the slow-review label Jan 29, 2026

github-actionsBot removed the slow-review label Feb 2, 2026

Abacn self-requested a review February 2, 2026 14:27

Abacn reviewed Feb 2, 2026
View reviewed changes

address comments: add java docs; add unit tests for unparsing; fix nits
9fd7054

ahmedabu98 requested a review from Abacn February 4, 2026 19:24

spotless
20d25d7

Abacn approved these changes Feb 9, 2026
View reviewed changes

ahmedabu98 merged commit f96db37 into apache:masterFeb 9, 2026
25 of 27 checks passed

Uh oh!

Conversation

ahmedabu98 commented Oct 20, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Modify a catalog's properties

Modify a table's properties, schema, or partition spec

Uh oh!

github-actionsBot commented Nov 11, 2025

Uh oh!

ahmedabu98 commented Nov 13, 2025

Uh oh!

gemini-code-assistBot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Nov 21, 2025

Uh oh!

Abacn commented Nov 21, 2025

Uh oh!

github-actionsBot commented Nov 21, 2025

Uh oh!

github-actionsBot commented Nov 29, 2025

Uh oh!

github-actionsBot commented Dec 3, 2025

Uh oh!

ahmedabu98 commented Dec 10, 2025

Uh oh!

gemini-code-assistBot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

github-actionsBot commented Dec 18, 2025

Uh oh!

github-actionsBot commented Dec 22, 2025

Uh oh!

github-actionsBot commented Dec 29, 2025

Uh oh!

github-actionsBot commented Jan 1, 2026

Uh oh!

github-actionsBot commented Jan 8, 2026

Uh oh!

github-actionsBot commented Jan 13, 2026

Uh oh!

github-actionsBot commented Jan 20, 2026

Uh oh!

github-actionsBot commented Jan 22, 2026

Uh oh!

github-actionsBot commented Jan 29, 2026

Uh oh!

github-actionsBot commented Feb 2, 2026

Uh oh!

Abacn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AbacnFeb 2, 2026

Choose a reason for hiding this comment

Uh oh!

ahmedabu98Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecovBot commented Feb 4, 2026

Codecov Report

Uh oh!

Abacn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

ahmedabu98 commented Oct 20, 2025•
edited
Loading