Uh oh!
There was an error while loading. Please reload this page.
[SPARK-18097][SQL] Add exception catch to handle corrupted metadata scenario while dropping the table - #16083
Conversation
Update from original
Handling the drop table command for meta data corrupted scenario.
thomastechs
commented
Nov 30, 2016
This patch would be catching the exception happening while the metadatacheck(due to getting corrupted) and continue with a warning log and continue with the drop operation... |
| case _ => | ||
| } | ||
| } catch { | ||
| case e: QueryExecutionException => log.warn(e.toString, e) |
There was a problem hiding this comment.
Hi @gatorsmile, I tested this change against all existing test cases. Also @davies mentioned in the jira itself, we cannot purposefully corrupt the meta data. So I tested and verified, this change will not break any existing test cases. And we are expecting QueryExecutionException when the meta data is get corrupted in some way.
There was a problem hiding this comment.
The existing test cases do not cover this case. I am afraid this might not work as expected. We need to test it.
There was a problem hiding this comment.
@gatorsmile and @davies ; In that case, any suggestions to mock the metadata corrupt scenario?
There was a problem hiding this comment.
I do not know why we can hit QueryExecutionException in getTableMetadata.
There was a problem hiding this comment.
Maybe you can manually modify the metadata stored in the database and check whether it fails as expected.
HyukjinKwon
commented
Feb 9, 2017
Hi @jayadevanmurali, are you still working on this? It seems inactive for few months. Maybe, it might better be closed for now if you are currently not able to proceed further. |
jayadevanmurali
commented
Feb 10, 2017
Hi @HyukjinKwon , |
HyukjinKwon
commented
Feb 11, 2017
Actually, I don't know (or heard of) how the schema is corrupt. I guess any committer should verify this before merging it. How about asking this to dev/user mailing list if you are not sure? If it is expected to be open without the reproducible steps for a long time (like few weeks or months), I guess we maybe should better close this for now and then open it latter. |
AmplabJenkins
commented
Mar 8, 2017
Can one of the admins verify this patch? |
Closesapache#16819Closesapache#13467Closesapache#16083Closesapache#17135Closesapache#8785Closesapache#16278Closesapache#16997Closesapache#17073Closesapache#17220
Closesapache#16819Closesapache#13467Closesapache#16083Closesapache#17135Closesapache#8785Closesapache#16278Closesapache#16997Closesapache#17073Closesapache#17220 Added: Closesapache#12059Closesapache#12524Closesapache#12888Closesapache#16061 Author: Sean Owen <sowen@cloudera.com> Closesapache#17386 from srowen/StalePRs.
What changes were proposed in this pull request?
Add exception catch to handle corrupted meta data scenario while dropping the table
How was this patch tested?
Unit tests
Please review http://spark.apache.org/contributing.html before opening a pull request.