Skip to content

PHOENIX-6032: When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped - #949

Merged
ChinmaySKulkarni merged 1 commit into
apache:4.xfrom
ChinmaySKulkarni:PHOENIX-6032
Nov 4, 2020
Merged

PHOENIX-6032: When phoenix.allow.system.catalog.rollback=true, a view still sees data from a column that was dropped#949
ChinmaySKulkarni merged 1 commit into
apache:4.xfrom
ChinmaySKulkarni:PHOENIX-6032

Conversation

@ChinmaySKulkarni

Copy link
Copy Markdown
Contributor

No description provided.

@ChinmaySKulkarni

Copy link
Copy Markdown
ContributorAuthor

Please review @yanxinyi@jpisaac@gjacoby126@twdsilva

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 5sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall11m 41s4.x passed
+1 💚compile1m 0s4.x passed
+1 💚checkstyle1m 4s4.x passed
+1 💚javadoc0m 46s4.x passed
+0 🆗spotbugs3m 9sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall6m 15sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
-1 ❌checkstyle1m 4sphoenix-core: The patch generated 4 new + 2127 unchanged - 0 fixed = 2131 total (was 2127)
-1 ❌whitespace0m 0sThe patch 7 line(s) with tabs.
+1 💚javadoc0m 45sthe patch passed
+1 💚spotbugs3m 21sthe patch passed
_ Other Tests _
-1 ❌unit171m 22sphoenix-core in the patch failed.
+1 💚asflicense0m 28sThe patch does not generate ASF License warnings.
205m 29s
ReasonTests
Failed junit testsphoenix.end2end.UpsertSelectIT
phoenix.end2end.SkipScanQueryIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#949
JIRA IssuePHOENIX-6032
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux af08208ec8c5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 2aaf2e2
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespacehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/whitespace-tabs.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/testReport/
Max. process+thread count6075 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall12m 3s4.x passed
+1 💚compile1m 0s4.x passed
+1 💚checkstyle1m 5s4.x passed
+1 💚javadoc0m 47s4.x passed
+0 🆗spotbugs3m 8sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall6m 16sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
-1 ❌checkstyle1m 5sphoenix-core: The patch generated 4 new + 2127 unchanged - 0 fixed = 2131 total (was 2127)
-1 ❌whitespace0m 0sThe patch 7 line(s) with tabs.
+1 💚javadoc0m 44sthe patch passed
+1 💚spotbugs3m 21sthe patch passed
_ Other Tests _
-1 ❌unit174m 14sphoenix-core in the patch failed.
+1 💚asflicense0m 29sThe patch does not generate ASF License warnings.
208m 43s
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#949
JIRA IssuePHOENIX-6032
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux b9bf4f9c9ca0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 2aaf2e2
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespacehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/whitespace-tabs.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/testReport/
Max. process+thread count5842 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ChinmaySKulkarni

Copy link
Copy Markdown
ContributorAuthor

Fixed whitespace and checkStyle warnings. Also, test failures are not failing locally.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 0sDocker mode activated.
-1 ❌patch0m 4s#949 does not apply to 4.x. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
SubsystemReport/Notes
GITHUB PR#949
JIRA IssuePHOENIX-6032
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/3/console
versionsgit=2.17.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ChinmaySKulkarni

Copy link
Copy Markdown
ContributorAuthor

Thanks for the +1 Xinyi. Any one else want to review this patch? @virajjasani@gjacoby126 ?

@ChinmaySKulkarni

Copy link
Copy Markdown
ContributorAuthor

Deleted last build to avoid noise. Waiting for latest build

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 29sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall10m 39s4.x passed
+1 💚compile0m 55s4.x passed
+1 💚checkstyle1m 21s4.x passed
+1 💚javadoc0m 44s4.x passed
+0 🆗spotbugs2m 52sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 20sthe patch passed
+1 💚compile0m 56sthe patch passed
+1 💚javac0m 56sthe patch passed
+1 💚checkstyle1m 24sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 43sthe patch passed
+1 💚spotbugs3m 7sthe patch passed
_ Other Tests _
-1 ❌unit124m 44sphoenix-core in the patch failed.
+1 💚asflicense0m 35sThe patch does not generate ASF License warnings.
156m 30s
ReasonTests
Failed junit testsphoenix.end2end.OrphanViewToolIT
phoenix.end2end.index.MutableIndexExtendedIT
phoenix.end2end.ConcurrentMutationsExtendedIT
phoenix.end2end.NullIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#949
JIRA IssuePHOENIX-6032
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 23e271dbea11 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 7cbebb4
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/testReport/
Max. process+thread count6990 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/5/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few minor nits, +1 (non-binding) for source changes with my understanding

@gjacoby126gjacoby126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few nits

testUtil.getHBaseAdmin().enableTable(systemCatalog);
} catch (DoNotRetryIOException e) {
// table is not splittable
assert (e.getMessage().contains("NOT splittable"));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we require that we get this exception? If so should be an Assert.fail() after enableTable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should use JUnit assertions rather than Java assert keyword

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I know this is a copied test, but good to do the simple fixes while we're at it)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I'm not sure. This diff is just a result of copy pasting from the original test SystemCatalogIT Let me try to find out. Will change to using JUnit assert.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like that test wasn't reliably trying to split SYSCAT at all. I will open a follow-up Jira to fix this since it is unrelated to 6032.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

… still sees data from a column that was dropped
@ChinmaySKulkarni

Copy link
Copy Markdown
ContributorAuthor

Thanks for your reviews @virajjasani@gjacoby126 I have addressed all your review comments. Can I get a +1?

@stoty

stoty commented Nov 3, 2020

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 20sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall11m 44s4.x passed
+1 💚compile1m 0s4.x passed
+1 💚checkstyle1m 5s4.x passed
+1 💚javadoc0m 47s4.x passed
+0 🆗spotbugs3m 9sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall6m 11sthe patch passed
+1 💚compile1m 0sthe patch passed
+1 💚javac1m 0sthe patch passed
-1 ❌checkstyle1m 5sphoenix-core: The patch generated 1 new + 2141 unchanged - 1 fixed = 2142 total (was 2142)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 46sthe patch passed
+1 💚spotbugs3m 22sthe patch passed
_ Other Tests _
-1 ❌unit213m 12sphoenix-core in the patch failed.
+1 💚asflicense0m 45sThe patch does not generate ASF License warnings.
253m 7s
ReasonTests
Failed junit testsphoenix.end2end.index.GlobalMutableTxIndexIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#949
JIRA IssuePHOENIX-6032
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux a82b46abca17 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / a8a9699
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/testReport/
Max. process+thread count5434 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/6/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

stoty commented Nov 3, 2020

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 31sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall10m 55s4.x passed
+1 💚compile0m 57s4.x passed
+1 💚checkstyle1m 22s4.x passed
+1 💚javadoc0m 43s4.x passed
+0 🆗spotbugs2m 51sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 18sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
+1 💚checkstyle1m 24sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 41sthe patch passed
+1 💚spotbugs3m 3sthe patch passed
_ Other Tests _
-1 ❌unit135m 10sphoenix-core in the patch failed.
+1 💚asflicense0m 34sThe patch does not generate ASF License warnings.
171m 9s
ReasonTests
Failed junit testsphoenix.end2end.TenantSpecificViewIndexIT
phoenix.end2end.RowValueConstructorOffsetIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.util.CoprocessorHConnectionTableFactoryIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#949
JIRA IssuePHOENIX-6032
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux aea7b97e7e90 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 8aa243d
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/testReport/
Max. process+thread count6967 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-949/7/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@gjacoby126gjacoby126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@ChinmaySKulkarni
ChinmaySKulkarni merged commit 11575c0 into apache:4.xNov 4, 2020
@ChinmaySKulkarni
ChinmaySKulkarni deleted the PHOENIX-6032 branch November 4, 2020 02:03
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ChinmaySKulkarni@stoty@gjacoby126@yanxinyi@virajjasani