Uh oh!
There was an error while loading. Please reload this page.
PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table - #995
Merged
Merged
Conversation
tkhurana
commented
Dec 1, 2020
ContributorAuthor
stoty
commented
Dec 2, 2020
Contributor
💔 -1 overall
This message was automatically generated. |
stoty
commented
Dec 2, 2020
Contributor
💔 -1 overall
This message was automatically generated. |
gokceni
reviewed
Dec 2, 2020
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| // Run -v AFTER and check it doesn't fix the extra rows and the job fails | ||
| IndexTool indexTool = IndexToolIT.runIndexTool(false, false, schemaName, dataTableName, | ||
| indexTableName, null, -1, IndexVerifyType.AFTER, "-fi"); |
Contributor
There was a problem hiding this comment.
So this means -fi only works with BEFORE option is that correct?
Is there any reason why it doesn't work with AFTER?
I would expect it to build correctly independent of verify option and Verify is just verification not affecting how we build the index.
ContributorAuthor
There was a problem hiding this comment.
@Gokcen AFTER first rebuilds the index using the expected mutations from the data table. So any extra rows in the index table will remain after the rebuild since they are not present in the data table. Then when we verify, those rows will be reported as extra. As a result, the AFTER job will fail.
Uh oh!
There was an error while loading. Please reload this page.
| indexScan.setRaw(true); | ||
| indexScan.setMaxVersions(); | ||
| indexScan.setCacheBlocks(false); | ||
| try (RegionScanner regionScanner = region.getScanner(indexScan)) { |
Contributor
There was a problem hiding this comment.
Does the below code work if there is no index row?
ContributorAuthor
There was a problem hiding this comment.
@gokceni I am not sure I understood your concern here ?
Contributor
There was a problem hiding this comment.
If the index is truncated for some reason and this code is run, the below do while code, will it work since you seem to be checking the hasMore in the while part of the loop?
ContributorAuthor
There was a problem hiding this comment.
@gokceni hasMore will be false in that case and the code should exit the loop. This is a standard pattern used in multiple places in the index rebuild code.
stoty
commented
Dec 5, 2020
Contributor
💔 -1 overall
This message was automatically generated. |
tkhurana
commented
Dec 7, 2020
ContributorAuthor
The failure in StoreNullsIT is because the feature branch is missing this commit #939 |
gokceni
approved these changes
Dec 7, 2020
swaroopak
approved these changes
Dec 9, 2020
swaroopak pushed a commit
that referenced
this pull request
Dec 17, 2020
…d PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback
swaroopak pushed a commit
that referenced
this pull request
Dec 17, 2020
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
tkhurana added a commit
to tkhurana/phoenix
that referenced
this pull request
Jan 29, 2021
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
stoty pushed a commit
to stoty/phoenix
that referenced
this pull request
Feb 1, 2021
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
stoty pushed a commit
to stoty/phoenix
that referenced
this pull request
Feb 1, 2021
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
stoty pushed a commit
to stoty/phoenix
that referenced
this pull request
Feb 2, 2021
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
asfgit pushed a commit
that referenced
this pull request
Feb 2, 2021
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file This port to the master branch also includes a fix for * PHOENIX-6356 missing row.clear() for dummy row in GlobalIndexRegionScanner
asfgit pushed a commit
that referenced
this pull request
Feb 2, 2021
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file This port to the master branch also includes a fix for * PHOENIX-6356 missing row.clear() for dummy row in GlobalIndexRegionScanner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For index verification when using index table as the source