Skip to content

Add concurrency safety validation checks - #3049

Merged
Fokko merged 3 commits into
apache:mainfrom
gabeiglio:validation-checks
Feb 20, 2026
Merged

Add concurrency safety validation checks#3049
Fokko merged 3 commits into
apache:mainfrom
gabeiglio:validation-checks

Conversation

@gabeiglio

@gabeigliogabeiglio commented Feb 16, 2026

Copy link
Copy Markdown
Contributor

Closes#1930
Closes#1931

Rationale for this change

Now that we have an implementation of DeleteFileIndex we can move ahead and implement the last two safety concurrency validations mentioned in this #819

Are these changes tested?

Yes

Are there any user-facing changes?

No

@gabeiglio
gabeiglio marked this pull request as ready for review February 16, 2026 23:51
geruh
geruh reviewed Feb 17, 2026
View reviewed changes
Comment threadpyiceberg/table/update/validate.py Outdated
Comment threadpyiceberg/table/update/validate.py Outdated
Comment threadtests/table/test_validate.py
Comment threadpyiceberg/table/delete_file_index.py Outdated
Comment threadpyiceberg/table/update/validate.py Outdated
Comment threadpyiceberg/table/update/validate.py Outdated
@lawofcycleslawofcycles mentioned this pull request Feb 17, 2026
Open
tusharchou
tusharchou reviewed Feb 17, 2026
View reviewed changes
Comment threadpyiceberg/table/update/validate.py
parent_snapshot: Ending snapshot on the branch being validated
"""
# If there is no current state, or no files has been added
if parent_snapshot is None or table.format_version < 2:

@tusharchoutusharchouFeb 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've verified this logic locally by mocking a concurrent conflict. The table.format_version < 2 guard correctly prevents unnecessary overhead for V1 tables, and the use of the DeleteFileIndex ensures we are only blocking commits when there is a real overlap in data files (avoiding 'lazy' global locks).

Fokko
Fokko reviewed Feb 18, 2026
View reviewed changes
Comment threadpyiceberg/table/update/validate.py Outdated
@Fokko
Fokko merged commit d587aab into apache:mainFeb 20, 2026
13 checks passed
@Fokko

Fokko commented Feb 20, 2026

Copy link
Copy Markdown
Contributor

Thanks, @gabeiglio, for fixing this. Let's move this forward. Thanks @tusharchou and @geruh for the review 🙌

This was referenced May 2, 2026
Open
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@geruhgeruhgeruh left review comments
@FokkoFokkoFokko approved these changes
+1 more reviewer
@tusharchoutusharchoutusharchou left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

4 participants

@gabeiglio@Fokko@geruh@tusharchou