Skip to content

[SPARK-44577][SQL] Fix INSERT BY NAME returns nonsensical error message - #42220

Closed
Hisoka-X wants to merge 7 commits into
apache:masterfrom
Hisoka-X:SPARK-44577_insert_by_name_bug_fix
Closed

[SPARK-44577][SQL] Fix INSERT BY NAME returns nonsensical error message#42220
Hisoka-X wants to merge 7 commits into
apache:masterfrom
Hisoka-X:SPARK-44577_insert_by_name_bug_fix

Conversation

@Hisoka-X

@Hisoka-XHisoka-X commented Jul 29, 2023•
edited
Loading

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Fix INSERT BY NAME returns nonsensical error message on v1 datasource.
eg:

CREATETABLE bug(c1 INT);
INSERTINTO bug BYNAMESELECT1AS c2;
==>Multi-part identifier cannot be empty.

After PR:

[INCOMPATIBLE_DATA_FOR_TABLE.CANNOT_FIND_DATA] Cannot write incompatible data for the table `spark_catalog`.`default`.`bug`:Cannot find data for the output column `c1`.

Also fixed the same issue when throwing other INCOMPATIBLE_DATA_FOR_TABLE type errors

Why are the changes needed?

Fix the error msg nonsensical.

Does this PR introduce any user-facing change?

Yes, the error msg in v1 insert by name will be changed.

How was this patch tested?

add new test.

@Hisoka-X

Hisoka-X commented Jul 29, 2023

Copy link
Copy Markdown
MemberAuthor

cc @cloud-fan

@cloud-fan

cloud-fan commented Jul 31, 2023

Copy link
Copy Markdown
Contributor

can we update the PR description for the new changes?

@Hisoka-X

Hisoka-X commented Jul 31, 2023

Copy link
Copy Markdown
MemberAuthor

can we update the PR description for the new changes?

Done

@cloud-fan

cloud-fan commented Aug 1, 2023

Copy link
Copy Markdown
Contributor
[INSERT_COLUMN_ARITY_MISMATCH.NOT_ENOUGH_DATA_COLUMNS] Cannot write to `spark_catalog`.`default`.`bug`, the reason is not enough data columns:
Table columns: `c1`.
Data columns: `c2`.

For by-name insert, shall we just say "incompatible data columns"? The number of columns does match here.

@Hisoka-X

Hisoka-X commented Aug 1, 2023

Copy link
Copy Markdown
MemberAuthor
[INSERT_COLUMN_ARITY_MISMATCH.NOT_ENOUGH_DATA_COLUMNS] Cannot write to `spark_catalog`.`default`.`bug`, the reason is not enough data columns:
Table columns: `c1`.
Data columns: `c2`.

For by-name insert, shall we just say "incompatible data columns"? The number of columns does match here.

Oh, let me align v1 and v2 errors.

MaxGekk
MaxGekk reviewed Aug 7, 2023
View reviewed changes
Comment threadsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala Outdated
Comment threadsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala Outdated
@cloud-fan

cloud-fan commented Aug 8, 2023

Copy link
Copy Markdown
Contributor

@MaxGekk does this PR align with your plan of unifying the assignment errors?

@MaxGekk

MaxGekk commented Aug 8, 2023

Copy link
Copy Markdown
Member

@cloud-fan In my PR #42393, I am going to touch non-BY NAME functionality so far.

@Hisoka-X

Hisoka-X commented Aug 17, 2023

Copy link
Copy Markdown
MemberAuthor

Hi @cloud-fan@MaxGekk Shall we continue push on this PR? Thanks.

cloud-fan
cloud-fan reviewed Aug 31, 2023
View reviewed changes
Comment threadcommon/utils/src/main/resources/error/error-classes.json Outdated
cloud-fan
cloud-fan reviewed Aug 31, 2023
View reviewed changes
Comment threadcommon/utils/src/main/resources/error/error-classes.json Outdated
},
"EXTRA_FIELDS" : {
"message" : [
"Cannot write extra fields <colName>."

@cloud-fancloud-fanAug 31, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's one column or multiple columns?

@Hisoka-XHisoka-XAug 31, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple columns, so I do some change, please check it again. Thanks.

@cloud-fan

cloud-fan commented Sep 1, 2023

Copy link
Copy Markdown
Contributor

thanks, merging to master/3.5!

cloud-fan pushed a commit that referenced this pull request Sep 1, 2023
@Hisoka-X@cloud-fan
[SPARK-44577][SQL] Fix INSERT BY NAME returns nonsensical error message
b41fd4c
### What changes were proposed in this pull request?
Fix INSERT BY NAME returns nonsensical error message on v1 datasource.
eg:
```scala
CREATE TABLE bug(c1 INT);
INSERT INTO bug BY NAME SELECT 1 AS c2;
==> Multi-part identifier cannot be empty.
```
After PR:
```scala
[INCOMPATIBLE_DATA_FOR_TABLE.CANNOT_FIND_DATA] Cannot write incompatible data for the table `spark_catalog`.`default`.`bug`: Cannot find data for the output column `c1`.
```
Also fixed the same issue when throwing other INCOMPATIBLE_DATA_FOR_TABLE type errors
### Why are the changes needed?
Fix the error msg nonsensical.
### Does this PR introduce _any_ user-facing change?
Yes, the error msg in v1 insert by name will be changed.
### How was this patch tested?
add new test.
Closes#42220 from Hisoka-X/SPARK-44577_insert_by_name_bug_fix.
Authored-by: Jia Fan <fanjiaeminem@qq.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit 00f6699)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@Hisoka-X

Hisoka-X commented Sep 1, 2023

Copy link
Copy Markdown
MemberAuthor

Thanks @cloud-fan and @MaxGekk

@Hisoka-X
Hisoka-X deleted the SPARK-44577_insert_by_name_bug_fix branch September 1, 2023 12:35
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MaxGekkMaxGekkMaxGekk left review comments
@cloud-fancloud-fancloud-fan approved these changes

Assignees

No one assigned

Labels

DOCSSQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Hisoka-X@cloud-fan@MaxGekk