PHOENIX-5224 Change 'Statement' to 'PreparedStatement' for better performance by bd2019us · Pull Request #474 · apache/phoenix

bd2019us · 2019-04-01T22:40:03Z

Fix: #PHOENIX-5224. When the same SQL query is intensively used in a loop, it should be replaced with PreparedStatement, which is a big gain on performance.

…formance

sync with base

swaroopak · 2019-04-05T19:03:57Z

- stmt.executeUpdate(
- "UPSERT INTO " + tableName + " VALUES('" + getRandomOrgId(maxOrgId) + "'," + i
- + "," + (i + 1) + "," + (i + 2) + ")");
+ stmt.setString(1, getRandomOrgId(maxOrgId);


missing closing parenthesis.

swaroopak · 2019-04-05T19:05:43Z

 conn.createStatement().execute("CREATE "+(localIndex? "LOCAL " : "")+"INDEX " + indexName + " ON " + fullTableName + "(val)");

 conn.createStatement().execute("UPSERT INTO " + fullTableName + " VALUES (NEXT VALUE FOR " + seqName + ",1)");
+ PreparedStatement stmt = comm.prepareStatement("UPSERT INTO " + fullTableName + " SELECT NEXT VALUE FOR " + seqName + ", val FROM " + fullTableName);


incorrect variable name.

swaroopak

Thank you for your PR. This 'll be helpful once committed.
Although I see there are a few things missing that could have been caught by compiler/IDE. Does this PR compile properly?

swaroopak · 2019-04-05T19:09:18Z

@@ -104,9 +105,9 @@ public void testUpsertSelectDoesntSeeUpsertedData() throws Exception {
 conn.createStatement().execute("CREATE "+(localIndex? "LOCAL " : "")+"INDEX " + indexName + " ON " + fullTableName + "(val)");


while you are at it, Could you please logically separate this line of code? Lots of things happening in one line.
Thanks.

bd2019us · 2019-04-07T16:23:32Z

@swaroopak Thanks for your reply. Sorry for my mistake and the compiling error is fixed, after which I run mvn compile and the output is shown below.

[INFO] Apache Phoenix ..................................... SUCCESS [ 0.564 s]
[INFO] Phoenix Core ....................................... SUCCESS [ 20.598 s]
[INFO] Phoenix - Pherf .................................... FAILURE [ 0.157 s]
[INFO] Phoenix Client ..................................... SKIPPED
[INFO] Phoenix Server ..................................... SKIPPED
[INFO] Phoenix Assembly ................................... SKIPPED
[INFO] Phoenix - Tracing Web Application .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.960 s
[INFO] Finished at: 2019-04-07T11:17:44-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project phoenix-pherf: Could not resolve dependencies forproject org.apache.phoenix:phoenix-pherf:jar:5.1.0-HBase-2.0-SNAPSHOT: Failure to find org.apache.phoenix:phoenix-core:jar:tests:5.1.0-HBase-2.0-SNAPSHOTin https://repository.apache.org/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of apache release has elapsed or updates are forced -> [Help 1]

Since I only changed the code in "Phoenix-core", I have no idea about the failure in "Phoenix - Pherf" , Could you kindly help. Thanks.

swaroopak · 2019-04-09T23:10:33Z

make sure to have imported all the packages and run mvn clean install

bd2019us · 2019-04-12T03:49:22Z

The compilation error is fixed. Could your please check?

[INFO] stopped o.e.j.s.h.ContextHandler{/webjars,file:/home/jiajun/empirical_pr/phoenix/}
[INFO] stopped o.e.j.s.h.ContextHandler{/classpath,file:/home/jiajun/empirical_pr/phoenix/}
[INFO] stopped o.e.j.s.h.ContextHandler{/,file:/home/jiajun/empirical_pr/phoenix/}
[INFO] stopped o.e.j.s.h.ContextHandler{/spec,file:/home/jiajun/empirical_pr/phoenix/}
[INFO] stopped o.e.j.s.h.ContextHandler{/src,file:/home/jiajun/empirical_pr/phoenix/}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Phoenix 5.1.0-HBase-2.0-SNAPSHOT:
[INFO]
[INFO] Apache Phoenix ..................................... SUCCESS [ 2.114 s]
[INFO] Phoenix Core ....................................... SUCCESS [01:54 min]
[INFO] Phoenix - Pherf .................................... SUCCESS [ 13.054 s]
[INFO] Phoenix Client ..................................... SUCCESS [ 0.999 s]
[INFO] Phoenix Server ..................................... SUCCESS [ 2.929 s]
[INFO] Phoenix Assembly ................................... SUCCESS [ 7.306 s]
[INFO] Phoenix - Tracing Web Application .................. SUCCESS [ 11.253 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:35 min
[INFO] Finished at: 2019-04-11T15:32:37-05:00
[INFO] ------------------------------------------------------------------------

swaroopak · 2019-04-19T21:52:35Z

@bd2019us could you as well submit the patch. So that nightly tests run on your patch?

bd2019us · 2019-04-24T19:45:16Z

Sure @swaroopak , I updated the patch on JIRA

gjacoby126 · 2019-09-09T18:00:39Z

@bd2019us - changes look good - could you please rebase so that they can be committed?

bd2019us · 2019-09-11T17:03:44Z

@gjacoby126 - thanks, we have updated the PR with rebase correspondingly, please check it

stoty · 2021-08-10T10:56:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	23m 51s	master passed
+0	hbaserecompile	31m 1s	HBase recompiled.
+1 💚	compile	1m 4s	master passed
+1 💚	checkstyle	1m 2s	master passed
+1 💚	javadoc	0m 52s	master passed
+0 🆗	spotbugs	3m 22s	phoenix-core in master has 973 extant spotbugs warnings.
-0 ⚠️	patch	3m 30s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	17m 2s	the patch passed
+0	hbaserecompile	29m 7s	HBase recompiled.
+1 💚	compile	1m 5s	the patch passed
+1 💚	javac	1m 5s	the patch passed
+1 💚	checkstyle	1m 3s	the patch passed
-1 ❌	whitespace	0m 0s	The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	javadoc	0m 51s	the patch passed
+1 💚	spotbugs	3m 34s	the patch passed
		_ Other Tests _
+1 💚	unit	121m 13s	phoenix-core in the patch passed.
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		193m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-474/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#474
Optional Tests	dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname	Linux 625cebae4600 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / `c4d297d`
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
whitespace	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-474/1/artifact/yetus-general-check/output/whitespace-eol.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-474/1/testReport/
Max. process+thread count	11400 (vs. ulimit of 30000)
modules	C: phoenix-core U: phoenix-core
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-474/1/console
versions	git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

bd2019usand others added 3 commits April 1, 2019 17:25

PHOENIX-5224 Change 'Statement' to 'PreparedStatement' for better per…
da28588
…formance

Merge pull request#1from apache/master
f2a54c0
sync with base

resolve conflicts
cd37470

swaroopak reviewed Apr 5, 2019
View reviewed changes

swaroopak requested changes Apr 5, 2019
View reviewed changes

swaroopak reviewed Apr 5, 2019
View reviewed changes

fix compiling errors
6e1faac

fix compilation errors
782619b

bd2019us added 5 commits September 11, 2019 11:15

Merge branch 'master' ofhttps://github.com/bd2019us/phoenix
e773ad3

rebase merge
3a0a2aa

Merge branch 'master' ofhttps://github.com/apache/phoenix
2609458

rebase merge
771a2b6

recover removal of classes before rebase
480b0ee

Navigation Menu

Uh oh!

PHOENIX-5224 Change 'Statement' to 'PreparedStatement' for better performance - #474

PHOENIX-5224 Change 'Statement' to 'PreparedStatement' for better performance#474
bd2019us wants to merge 10 commits into
apache:masterfrom
bd2019us:PHOENIX-5224-PATCH

bd2019us commented Apr 1, 2019

Uh oh!

swaroopakApr 5, 2019

Uh oh!

swaroopakApr 5, 2019

Uh oh!

swaroopak left a comment •
edited
Loading

Uh oh!

swaroopakApr 5, 2019

Uh oh!

bd2019us commented Apr 7, 2019•
edited
Loading

Uh oh!

swaroopak commented Apr 9, 2019

Uh oh!

bd2019us commented Apr 12, 2019

Uh oh!

swaroopak commented Apr 19, 2019

Uh oh!

bd2019us commented Apr 24, 2019

Uh oh!

gjacoby126 commented Sep 9, 2019

Uh oh!

bd2019us commented Sep 11, 2019

Uh oh!

stoty commented Aug 10, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -104,9 +105,9 @@ public void testUpsertSelectDoesntSeeUpsertedData() throws Exception {
		conn.createStatement().execute("CREATE "+(localIndex? "LOCAL " : "")+"INDEX " + indexName + " ON " + fullTableName + "(val)");

Uh oh!

Conversation

bd2019us commented Apr 1, 2019

Uh oh!

swaroopakApr 5, 2019

Choose a reason for hiding this comment

Uh oh!

swaroopakApr 5, 2019

Choose a reason for hiding this comment

Uh oh!

swaroopak left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swaroopakApr 5, 2019

Choose a reason for hiding this comment

Uh oh!

bd2019us commented Apr 7, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swaroopak commented Apr 9, 2019

Uh oh!

bd2019us commented Apr 12, 2019

Uh oh!

swaroopak commented Apr 19, 2019

Uh oh!

bd2019us commented Apr 24, 2019

Uh oh!

gjacoby126 commented Sep 9, 2019

Uh oh!

bd2019us commented Sep 11, 2019

Uh oh!

stoty commented Aug 10, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

swaroopak left a comment •
edited
Loading

bd2019us commented Apr 7, 2019•
edited
Loading