Skip to content

PHOENIX-5974 add queryId for phoenix queries and propagate to hbase - #1250

Open
kiran-maturi wants to merge 2 commits into
apache:4.16from
kiran-maturi:PHOENIX-5974
Open

PHOENIX-5974 add queryId for phoenix queries and propagate to hbase#1250
kiran-maturi wants to merge 2 commits into
apache:4.16from
kiran-maturi:PHOENIX-5974

Conversation

@kiran-maturi

Copy link
Copy Markdown
Contributor

No description provided.

@kiran-maturi
kiran-maturi marked this pull request as draft June 17, 2021 12:33
@kiran-maturikiran-maturi changed the title PHOENIX-5974 add queryId for phoenix queries and propagate to hbasePHOENIX-5974 add queryId for phoenix queries and propagate to hbase - WIPJun 17, 2021
@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 31sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.16 Compile Tests _
+1 💚mvninstall13m 57s4.16 passed
+1 💚compile1m 1s4.16 passed
+1 💚checkstyle1m 37s4.16 passed
+1 💚javadoc0m 45s4.16 passed
+0 🆗spotbugs2m 59sphoenix-core in 4.16 has 944 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 46sthe patch passed
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
-1 ❌checkstyle1m 39sphoenix-core: The patch generated 54 new + 2834 unchanged - 30 fixed = 2888 total (was 2864)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 44sthe patch passed
+1 💚spotbugs3m 7sthe patch passed
_ Other Tests _
-1 ❌unit1m 18sphoenix-core in the patch failed.
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
36m 2s
ReasonTests
Failed junit testsphoenix.compile.HavingCompilerTest
phoenix.compile.JoinQueryCompilerTest
phoenix.schema.MetaDataClientTest
phoenix.query.ParallelIteratorsSplitTest
phoenix.expression.NullValueTest
phoenix.compile.CreateTableCompilerTest
phoenix.index.VerifySingleIndexRowTest
phoenix.index.IndexUpgradeToolTest
phoenix.util.ScanUtilTest
phoenix.execute.MutationStateTest
phoenix.mapreduce.util.IndexColumnNamesTest
phoenix.expression.RoundFloorCeilExpressionsTest
phoenix.query.ScannerLeaseRenewalTest
phoenix.index.IndexMaintainerTest
phoenix.query.PropertyPolicyProviderTest
phoenix.util.PhoenixEncodeDecodeTest
phoenix.query.QueryPlanTest
phoenix.execute.UnnestArrayPlanTest
phoenix.jdbc.PhoenixPreparedStatementTest
phoenix.mapreduce.index.IndexScrutinyTableOutputTest
phoenix.compile.WhereCompilerTest
phoenix.compile.WhereOptimizerTest
phoenix.hbase.index.covered.NonTxIndexBuilderTest
phoenix.iterate.RowKeyOrderedAggregateResultIteratorTest
phoenix.compile.QueryMetaDataTest
phoenix.compile.SelectStatementRewriterTest
phoenix.query.KeyRangeClipTest
phoenix.compile.ViewCompilerTest
phoenix.compile.StatementHintsCompilationTest
phoenix.query.ExplainPlanTextTest
phoenix.schema.MutationTest
phoenix.util.csv.StringToArrayConverterTest
phoenix.iterate.AggregateResultScannerTest
phoenix.filter.SkipScanBigFilterTest
phoenix.compile.QueryOptimizerTest
phoenix.jdbc.PhoenixDriverTest
phoenix.util.json.JsonUpsertExecutorTest
phoenix.query.OrderByTest
phoenix.compile.LimitCompilerTest
phoenix.util.csv.CsvUpsertExecutorTest
phoenix.schema.RowKeyValueAccessorTest
phoenix.schema.RowKeySchemaTest
phoenix.execute.LiteralResultIteratorPlanTest
phoenix.compile.CursorCompilerTest
phoenix.query.ConnectionlessTest
phoenix.compile.TenantSpecificViewIndexCompileTest
phoenix.index.IndexScrutinyMapperTest
phoenix.util.PhoenixRuntimeTest
phoenix.mapreduce.util.PhoenixConfigurationUtilTest
phoenix.index.PrepareIndexMutationsForRebuildTest
phoenix.compile.PostIndexDDLCompilerTest
phoenix.jdbc.PhoenixResultSetMetadataTest
phoenix.compile.QueryCompilerTest
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1250
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 71bb7a7f22f0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.16 / 7cb4a05
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/1/testReport/
Max. process+thread count523 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 19sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.16 Compile Tests _
+1 💚mvninstall14m 9s4.16 passed
+1 💚compile1m 2s4.16 passed
+1 💚checkstyle1m 38s4.16 passed
+1 💚javadoc0m 44s4.16 passed
+0 🆗spotbugs3m 7sphoenix-core in 4.16 has 944 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall5m 38sthe patch passed
+1 💚compile1m 2sthe patch passed
+1 💚javac1m 2sthe patch passed
-1 ❌checkstyle1m 39sphoenix-core: The patch generated 30 new + 2860 unchanged - 4 fixed = 2890 total (was 2864)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 44sthe patch passed
+1 💚spotbugs3m 14sthe patch passed
_ Other Tests _
+1 💚unit191m 52sphoenix-core in the patch passed.
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
231m 8s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1250
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 02a4971e2413 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.16 / 7cb4a05
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/2/testReport/
Max. process+thread count5014 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 40sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ 4.16 Compile Tests _
+1 💚mvninstall20m 17s4.16 passed
+1 💚compile1m 6s4.16 passed
+1 💚checkstyle1m 13s4.16 passed
+1 💚javadoc0m 47s4.16 passed
+0 🆗spotbugs3m 15sphoenix-core in 4.16 has 944 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall11m 47sthe patch passed
+1 💚compile1m 6sthe patch passed
+1 💚javac1m 6sthe patch passed
-1 ❌checkstyle1m 13sphoenix-core: The patch generated 49 new + 2834 unchanged - 30 fixed = 2883 total (was 2864)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 47sthe patch passed
+1 💚spotbugs3m 29sthe patch passed
_ Other Tests _
+1 💚unit180m 15sphoenix-core in the patch passed.
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
232m 2s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1250
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 1b652c55fe85 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.16 / 9caa23c
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/3/testReport/
Max. process+thread count5352 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1250/3/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@kiran-maturikiran-maturi changed the title PHOENIX-5974 add queryId for phoenix queries and propagate to hbase - WIPPHOENIX-5974 add queryId for phoenix queries and propagate to hbaseJul 1, 2021
@kiran-maturi
kiran-maturi marked this pull request as ready for review July 1, 2021 05:49
@abhishek-chouhan

Copy link
Copy Markdown
Contributor

Have we looked at the scanId that we already have in BaseResultIterators? Seems to be scarcely used. However, we would want to have a single notion of a queryId, unless needed otherwise.

It also looks like the clients need to set this explicitly? Ideally we would not want the clients going through the internal implementations of phoenix (PhoenixStatement and their derivates in this case). Having phoenix generate the queryId implicitly makes more sense IMO.

The notion also seems useful not only for tracing but also for debugging. At the moment, logging in phoenix is scattered and hard to correlate with the actual query. @ArkenKiran

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kiran-maturi@stoty@abhishek-chouhan