Skip to content

PHOENIX-6171 Child views should not be allowed to override the parent view PHOENIX_TTL attribute. - #916

Closed
jpisaac wants to merge 3 commits into
apache:4.xfrom
jpisaac:PHOENIX-6171-PR
Closed

PHOENIX-6171 Child views should not be allowed to override the parent view PHOENIX_TTL attribute.#916
jpisaac wants to merge 3 commits into
apache:4.xfrom
jpisaac:PHOENIX-6171-PR

Conversation

@jpisaac

Copy link
Copy Markdown
Contributor

No description provided.

@jpisaac

Copy link
Copy Markdown
ContributorAuthor

@stoty

stoty commented Oct 9, 2020

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 28sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall27m 2s4.x passed
+1 💚compile0m 58s4.x passed
+1 💚checkstyle2m 45s4.x passed
+1 💚javadoc0m 43s4.x passed
+0 🆗spotbugs2m 52sphoenix-core in 4.x has 957 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall22m 57sthe patch passed
+1 💚compile0m 55sthe patch passed
+1 💚javac0m 55sthe patch passed
-1 ❌checkstyle2m 49sphoenix-core: The patch generated 27 new + 5445 unchanged - 4 fixed = 5472 total (was 5449)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 43sthe patch passed
+1 💚spotbugs3m 3sthe patch passed
_ Other Tests _
-1 ❌unit124m 4sphoenix-core in the patch failed.
+1 💚asflicense0m 37sThe patch does not generate ASF License warnings.
192m 37s
ReasonTests
Failed junit testsphoenix.end2end.RowValueConstructorOffsetIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.end2end.ConcurrentMutationsExtendedIT
phoenix.end2end.DerivedTableIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#916
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux b072fbaa2ce8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / fc3d25c
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/testReport/
Max. process+thread count6934 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

PTable table = getTableFromCache(cacheKey, clientTimeStamp, clientVersion);

// Disallow if the parent has PHOENIX_TTL set.
if (parentTable != null && parentTable.getPhoenixTTL() != PHOENIX_TTL_NOT_DEFINED) {

@yanxinyiyanxinyiOct 13, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just want to confirm that we don't support more than 2 level case. For example, we have a global view V1 of the TTL set. The V2 is a view on top of the V1. If we create a new V3 on of the v2 with a TTL value, this won't catch and allow the V3 view to have a TTL value, which might conflict with V1 TTL.

better to document somewhere to notice other ppl

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yanxinyi Currently the PHOENIX_TTL value is stored with every view and at every level - so in this case V2 would have the same PHOENIX_TTL value as of V1. So when creating V3 the parentTable.getPhoenixTTL() != PHOENIX_TTL_NOT_DEFINED.

But realized as was going thru this that I need an additional check to ensure that the PHOENIX_TTL is being altered

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 16sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall26m 20s4.x passed
+1 💚compile0m 56s4.x passed
+1 💚checkstyle2m 44s4.x passed
+1 💚javadoc0m 44s4.x passed
+0 🆗spotbugs2m 47sphoenix-core in 4.x has 956 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall22m 52sthe patch passed
+1 💚compile0m 55sthe patch passed
+1 💚javac0m 55sthe patch passed
-1 ❌checkstyle2m 33sphoenix-core: The patch generated 32 new + 5445 unchanged - 4 fixed = 5477 total (was 5449)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 44sthe patch passed
+1 💚spotbugs3m 45sthe patch passed
_ Other Tests _
-1 ❌unit127m 26sphoenix-core in the patch failed.
+1 💚asflicense0m 36sThe patch does not generate ASF License warnings.
199m 22s
ReasonTests
Failed junit testsphoenix.end2end.AlterTableWithViewsIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#916
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 3b7e04256ddc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 2359f54
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/2/testReport/
Max. process+thread count7321 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@jpisaac

Copy link
Copy Markdown
ContributorAuthor

@yanxinyi@ChinmaySKulkarni rebased and fixed conflicts

+ "WHERE %s AND TABLE_SCHEM = '%s' AND TABLE_NAME = '%s' AND TABLE_TYPE = '%s'";

private static final String
ALTER_PHOENIX_TTL_SQL =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Modify formatting to avoid extra line

"ALTER VIEW \"%s\".\"%s\" set PHOENIX_TTL=%s";

private static final String ALTER_PHOENIX_TTL_SQL = "ALTER VIEW %s.%s set PHOENIX_TTL=%d";
private static final String

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Modify formatting to avoid extra line

RowFilter
schemaNameFilter =
new RowFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(schemaName));
QualifierFilter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar here. Can you please check if your IDE has the Phoenix IDE settings?

final SchemaBuilder schemaBuilder = createLevel2TenantViewWithGlobalLevelTTL(null, null);

String tenantId = schemaBuilder.getDataOptions().getTenantId();
String

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Same formatting issues here potentially.

String.format(ALTER_SQL_WITH_NO_TTL, schemaName, globalViewName, "COL_30");
stmt.execute(sql);
}
} catch (SQLException e) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This catch block is unnecessary, you can remove it.

// If a child view exists and the parent previously had a PHOENIX_TTL value set then that
// implies that the child view too has a valid PHOENIX_TTL (non zero).
// In this case we do not allow for ALTER of the parent view PHOENIX_TTL value.
if (!childViews.isEmpty()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we're setting the TTL for the first time for a parent view i.e. it was PHOENIX_TTL_NOT_DEFINED first and now we're setting it for the first time ever. This check will disallow that too. Is that intended?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change the comment to be less confusing.

}

// Since we do not allow propagation of PHOENIX_TTL values during ALTER for now.
// If a child view exists and the parent previously had a PHOENIX_TTL value set then that

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't checking the "and the parent previously had a PHOENIX_TTL value set " part. Shouldn't we do that too?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, changing the comment will bring more clarity. The checks are for "this" view's parent and its children

}

// Checks whether a non-zero PHOENIX_TTL value is being set.
private boolean settingNewPhoenixTTLAttribute(List<Mutation> tableMetadata, byte[] phoenixTtlBytes) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rename this to settingNewPhoenixTTLAttributeForView() to indicate it only applies to views and then also add the check for the same inside the method?

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 18sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall12m 11s4.x passed
+1 💚compile1m 3s4.x passed
+1 💚checkstyle2m 37s4.x passed
+1 💚javadoc0m 59s4.x passed
+0 🆗spotbugs3m 53sphoenix-core in 4.x has 954 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall7m 22sthe patch passed
+1 💚compile1m 12sthe patch passed
+1 💚javac1m 12sthe patch passed
-1 ❌checkstyle2m 29sphoenix-core: The patch generated 28 new + 5288 unchanged - 0 fixed = 5316 total (was 5288)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 54sthe patch passed
+1 💚spotbugs3m 56sthe patch passed
_ Other Tests _
-1 ❌unit130m 2sphoenix-core in the patch failed.
+1 💚asflicense0m 36sThe patch does not generate ASF License warnings.
175m 25s
ReasonTests
Failed junit testsphoenix.end2end.CaseStatementIT
phoenix.end2end.OrphanViewToolIT
phoenix.end2end.OrderByWithServerClientSpoolingDisabledIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#916
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 76c0bc3a23a7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 135692c
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/testReport/
Max. process+thread count7005 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@jpisaac

Copy link
Copy Markdown
ContributorAuthor

@ChinmaySKulkarni@yanxinyi Addressed review comments and rebased to resolve merge conflicts.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall10m 50s4.x passed
+1 💚compile0m 56s4.x passed
+1 💚checkstyle2m 14s4.x passed
+1 💚javadoc0m 46s4.x passed
+0 🆗spotbugs2m 56sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 59sthe patch passed
+1 💚compile1m 12sthe patch passed
+1 💚javac1m 12sthe patch passed
-1 ❌checkstyle2m 23sphoenix-core: The patch generated 57 new + 5253 unchanged - 34 fixed = 5310 total (was 5287)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 51sthe patch passed
+1 💚spotbugs4m 5sthe patch passed
_ Other Tests _
-1 ❌unit133m 21sphoenix-core in the patch failed.
+1 💚asflicense0m 33sThe patch does not generate ASF License warnings.
169m 20s
ReasonTests
Failed junit testsphoenix.end2end.index.GlobalMutableNonTxIndexIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#916
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 9345c7de816f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / ac0538b
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/3/testReport/
Max. process+thread count6942 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/3/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 46sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall11m 16s4.x passed
+1 💚compile0m 55s4.x passed
+1 💚checkstyle2m 22s4.x passed
+1 💚javadoc0m 44s4.x passed
+0 🆗spotbugs3m 6sphoenix-core in 4.x has 953 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 41sthe patch passed
+1 💚compile0m 54sthe patch passed
+1 💚javac0m 54sthe patch passed
-1 ❌checkstyle2m 45sphoenix-core: The patch generated 21 new + 5287 unchanged - 0 fixed = 5308 total (was 5287)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 42sthe patch passed
+1 💚spotbugs3m 1sthe patch passed
_ Other Tests _
-1 ❌unit123m 38sphoenix-core in the patch failed.
+1 💚asflicense0m 38sThe patch does not generate ASF License warnings.
159m 9s
ReasonTests
Failed junit testsphoenix.end2end.UpsertSelectIT
phoenix.end2end.AlterTableWithViewsIT
phoenix.end2end.DropIndexedColsIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#916
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 09a9e4931dde 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / ac0538b
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/4/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/4/testReport/
Max. process+thread count7250 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-916/4/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@gjacoby126

Copy link
Copy Markdown
Contributor

JIRA appears to be closed as fixed. Closing PR.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jpisaac@stoty@gjacoby126@ChinmaySKulkarni@yanxinyi