Uh oh!
There was an error while loading. Please reload this page.
PHOENIX-6454: Add feature to SchemaTool to get the DDL in specificati… - #1217
Merged
Conversation
stoty
commented
Apr 30, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
| @Test | ||
| public void testCreateTableStatement_addColumn() throws Exception { | ||
| String expected = "CREATE TABLE IF NOT EXISTS TEST.SAMPLE_TABLE" |
Contributor
There was a problem hiding this comment.
nit: each of these would be easier to read if there was a comment saying what the difference was, such as "Adds RELATED_COMMAND" in this test
| */ | ||
| package org.apache.phoenix.schema; | ||
| public class SchemaProcessor { |
Contributor
There was a problem hiding this comment.
Why a class and not an interface?
| getCreateIndexStatement((CreateIndexStatement) createStatement, alterStatement); | ||
| return getCreateIndexSQL(newCreateIndexStmt); | ||
| } | ||
| return ""; |
Contributor
There was a problem hiding this comment.
What about CreateSequenceStatement and CreateFunctionStatement? And if none of the above, shouldn't we get an error of some kind?
Contributor
There was a problem hiding this comment.
@swaroopak, This is a great start. As @gjacoby126 pointed out, there are more cases to cover. I think we need a design doc on this to explain how the design covers all the cases and explain how this tool will be used. I also think we can add add a future to Phoenix to self verify/test DDL statements. The place to add such a feature would be MetaDataClient. After every DDL, we can verify if DDL is correctly implemented by Phoenix.
ContributorAuthor
There was a problem hiding this comment.
Thank you @kadirozde.
I started the doc per your suggestion. Meanwhile, do you think if this PR can be reviewed/merged for the initial scenarios (would be easier on reviews and incremental addition of code)? Or do you recommend not to merge until all of that is implemented? Please let me know.
Contributor
There was a problem hiding this comment.
As far as I understand, this PR significantly refactors an existing tool. However, it does not cover some use cases. If so, refactoring should not reduce the functionality. So, I suggest a quick design review and a complete refactoring.
ContributorAuthor
There was a problem hiding this comment.
This PR adds a new feature in SchemaTool with SYNTH mode. I added the design doc in the Jira with the current implementation and future extensions.
As per offline discussions, I'll change the input format from 2 files to 1 file. Thanks, @kadirozde
Uh oh!
There was an error while loading. Please reload this page.
| return newCreateIndexStmt; | ||
| } | ||
| private CreateTableStatement getCreateTableStatement(DropColumnStatement alterStatement, |
Contributor
There was a problem hiding this comment.
nit: up to now we've been doing parameters as Create, Alter, and this switches to Alter, Create. Good to be consistent.
| finalProps = | ||
| ArrayListMultimap.<String, Pair<String, Object>>create(); | ||
| for (Map.Entry<String, Object> entry : oldPropMap.entrySet()) { | ||
| finalProps.put("", Pair.newPair(entry.getKey(), entry.getValue())); |
Contributor
There was a problem hiding this comment.
What's the purpose of a hard-coded ["" -> [key -> value]] instead of just [key -> value]? Can't be duplicates, because oldPropMap was already a multi-map, right?
ContributorAuthor
There was a problem hiding this comment.
Property multimap is formed with String and Pair as a key-value. Key for all properties pair is same "" but the value(pair of property key, value) is different. I had to stick to this to create a new CreateTableStatement. Not sure if I misunderstood your suggestion. LMK.
| return finalProps; | ||
| } | ||
| private String getCreateTableSQL(CreateTableStatement createStmt) { |
Contributor
There was a problem hiding this comment.
Good to extract these SQL-building helper functions into their own utility class that can be composed / injected. (For one thing, we're probably going to be adding the ability to output Statements to Avro schemas instead of SQL really soon now over at PHOENIX-6227)
stoty
commented
May 4, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
swaroopakforce-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
May 5, 2021 00:08
ee2a953 to
d046736Comparestoty
commented
May 5, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
stoty
commented
May 5, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
swaroopakforce-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
May 5, 2021 17:21
4a11cfc to
090d37eComparestoty
commented
May 5, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
stoty
commented
May 6, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
swaroopakforce-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
May 7, 2021 23:24
d5c93bd to
d6a924bCompareswaroopak
commented
May 7, 2021
ContributorAuthor
@gjacoby126@kadirozde thank you for the review. I updated the PR please take a look. Thanks. |
stoty
commented
May 8, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
stoty
commented
May 8, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
gjacoby126
commented
May 11, 2021
Contributor
@swaroopak - phoenix-tools unit tests are failing because of test coverage: and there's an ASF license warning for SchemaSQLUtil.java |
gjacoby126
requested changes
May 11, 2021
| @@ -0,0 +1,151 @@ | |||
| package org.apache.phoenix.schema; | |||
| .append("\nCONSTRAINT "+createStmt.getPrimaryKeyConstraint().getName()+" PRIMARY KEY") | ||
| .append(" ("+createStmt.getPrimaryKeyConstraint().toString()+"))" | ||
| .replaceAll(",", ",\n")); | ||
| if(createStmt.getTableType().equals(PTableType.VIEW)) { |
Contributor
There was a problem hiding this comment.
nit: space between if and (
| import java.util.List; | ||
| import java.util.Map; | ||
| public class SchemaSQLUtil { |
Contributor
There was a problem hiding this comment.
Does this utility class belong in tools? (that is, is it only useful to a particular tool or tools?)
ContributorAuthor
There was a problem hiding this comment.
I don't know where else it would be used at this time. Would be happy to move it else where as need arises.
swaroopak
commented
May 12, 2021
ContributorAuthor
@gjacoby126 updated. Also, checked locally that the new IT class has 98% coverage of the class. |
kadirozde
approved these changes
May 12, 2021
kadirozde
left a comment
Contributor
There was a problem hiding this comment.
Looks good to me. Please open one or more Jira (if you have not done yet) to cover the remaining DDL statements.
stoty
commented
May 12, 2021
Contributor
💔 -1 overall
This message was automatically generated. |
gjacoby126
approved these changes
May 12, 2021
gjacoby126
left a comment
Contributor
There was a problem hiding this comment.
+1, thanks for the changes, @swaroopak
swaroopak added a commit
to swaroopak/phoenix
that referenced
this pull request
May 13, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
swaroopak added a commit
to swaroopak/phoenix
that referenced
this pull request
May 13, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
swaroopak added a commit
that referenced
this pull request
May 21, 2021
…on mode (#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
richardantal pushed a commit
to richardantal/phoenix-1
that referenced
this pull request
Jul 22, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
richardantal pushed a commit
that referenced
this pull request
Jul 29, 2021
…on mode (#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…on mode