[SPARK-25135][SQL] Insert datasource table may all null when select from view by wangyum · Pull Request #22124 · apache/spark

wangyum · 2018-08-16T18:44:01Z

What changes were proposed in this pull request?

How to reproduce:

valpath="/tmp/spark/parquet"valcnt=30
spark.range(cnt).selectExpr("id as col1").write.mode("overwrite").parquet(path)
spark.sql(s"CREATE TABLE table1(col1 bigint) using parquet location '$path'")
spark.sql("create view view1 as select col1 from table1 where col1 > -20")
// The column name of table2 is inconsistent with the column name of view1.
spark.sql("create table table2 (COL1 BIGINT) using parquet")
// When querying the view, ensure that the column name of the query matches the column name of the target table.
spark.sql("insert overwrite table table2 select COL1 from view1")
// At this time, the column name of the target table is uppercase, but the column name of the Parquet file is consistent with the case of the previous view(It's lower case). so all nulls.
spark.table("table2").show

The root cause is when insert a table contains view. for example:

insert overwrite table table2 select COL1 from view1

the execution plan optimized to:

=== Applying Rule org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases ===
InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-c0
Database: default Database: default
Table: table2 Table: table2
Owner: yumwang Owner: yumwang
Created Time: Sun Aug 1922:54:11 PDT 2018 Created Time: Sun Aug 1922:54:11 PDT 2018
Last Access: Wed Dec 3116:00:00 PST 1969 Last Access: Wed Dec 3116:00:00 PST 1969
Created By: Spark 2.4.0-SNAPSHOT Created By: Spark 2.4.0-SNAPSHOT
Type: MANAGED Type: MANAGED
Provider: parquet Provider: parquet
Table Properties: [transient_lastDdlTime=1534744451] Table Properties: [transient_lastDdlTime=1534744451]
Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-c044 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-c0444fd8-772b-4841-9182-3c
Serde Library: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe Serde Library: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
InputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat InputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat OutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Storage Properties: [serialization.format=1] Storage Properties: [serialization.format=1]
Schema: root Schema: root
-- COL1: long (nullable = true) |-- COL1: long (nullable = true)
!), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@5c0e0003, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@5c0e0003, [col1#5L]
!+- Project [col1#5L AS col1#6L] +- Project [col1#5L]+- Filter (col1#5L > -20) +- Filter (col1#5L > -20)+- Relation[col1#5L] parquet

At this time the outputColumns is col1#5L. So spark will use col1 as the column name of the parquet.

Before SPARK-22834. The allColumns is queryExecution.analyzed.output, it's not optimized.

I have three ways to solve this issue:

Read the column name in the view with the actual column name. but it is difficult to handle all cases. Please see the third commit.
Do not remove redundant alias if plan is Command. Because alias may be useful.
Change this line from case a if resolver(a.name, name) => a.withName(name) to case a if resolver(a.name, name) => a. But this change will cause some test failures:

[info] - order-by-nulls-ordering.sql ***FAILED*** (4 seconds, 789 milliseconds)
[info] Expected"struct<[COL1:int,COL2:int,COL]3:int>", but got "struct<[col1:int,col2:int,col]3:int>"Schema did not matchfor query #6
[info] SELECTCOL1, COL2, COL3FROM spark_10747 ORDERBYCOL3ASCNULLSFIRST, COL2 (SQLQueryTestSuite.scala:246)

This pr use the second way to fix this issue.

How was this patch tested?

unit tests

wangyum · 2018-08-16T18:51:33Z

cc @gengliangwang

SparkQA · 2018-08-16T20:41:14Z

Test build #94861 has finished for PR 22124 at commit 276879c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-17T03:55:59Z

 expected.name == actual.name &&
- expected.metadata == actual.metadata) {
+ expected.metadata == actual.metadata &&
+ expected.exprId.id == actual.exprId.id) {


why does this fix the problem?

This is not a correct change. please ignore this.

wangyum · 2018-08-17T07:55:16Z

 // If the projection list contains Stars, expand it.
 case p: Project if containsStar(p.projectList) =>
 p.copy(projectList = buildExpandedProjectList(p.projectList, p.child))
+ case p @ Project(projectList, _ @ SubqueryAlias(_, view: View))


Replace [COL1#10L, COL2#11L] to col1#10L,col2#11L. thus DDLPreprocessingUtils.castAndRenameQueryOutput will take effect and rename column name same to target table column name.
Then parquet column name same to table column name and we can read it.

cloud-fan · 2018-08-17T08:12:53Z

can you explain how this bug happens and what's the root cause?

wangyum · 2018-08-17T09:13:00Z

Thanks @cloud-fan I updated it in description.

SparkQA · 2018-08-17T09:23:20Z

Test build #94882 has finished for PR 22124 at commit bb93ca0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-17T12:59:21Z

So DDLPreprocessingUtils.castAndRenameQueryOutput() can matched, will not do any rename.
So the target table parquet file column is lower case.

why the parquet file columns is lower-cased? the root project has names upper-cased, doesn't it?

wangyum · 2018-08-18T10:30:25Z

The root project should be consistent with the schema of the target table. But it is inconsistent now.

Before this PR:
dataColumns:
col1#8L,col2#9L
plan:

*(1) Project [col1#8L, col2#9L]
+- *(1) Filter (isnotnull(col1#8L) && (col1#8L > -20))
+- *(1) FileScan parquet default.table1[col1#8L,col2#9L] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/tmp/yumwang/spark/parquet], PartitionFilters: [], PushedFilters: [IsNotNull(col1), GreaterThan(col1,-20)], ReadSchema: struct<col1:bigint,col2:bigint>

After this PR:
dataColumns:
COL1#14L,COL2#15L
plan:

*(1) Project [col1#8L AS COL1#14L, col2#9L AS COL2#15L]
+- *(1) Filter (isnotnull(col1#8L) && (col1#8L > -20))
+- *(1) FileScan parquet default.table1[col1#8L,col2#9L] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/tmp/yumwang/spark/parquet], PartitionFilters: [], PushedFilters: [IsNotNull(col1), GreaterThan(col1,-20)], ReadSchema: struct<col1:bigint,col2:bigint>

Before SPARK-22834
dataColumns:
COL1#19L,COL2#20L

queryExecution:

== Parsed Logical Plan ==
Project [COL1#19L, COL2#20L]
+- SubqueryAlias view1
+- View (`default`.`view1`, [col1#19L,col2#20L])
+- Project [col1#15L, col2#16L]
+- Filter (col1#15L > cast(-20 as bigint))
+- SubqueryAlias table1
+- Relation[col1#15L,col2#16L] parquet
== Analyzed Logical Plan ==
COL1: bigint, COL2: bigint
Project [COL1#19L, COL2#20L]
+- SubqueryAlias view1
+- View (`default`.`view1`, [col1#19L,col2#20L])
+- Project [cast(col1#15L as bigint) AS col1#19L, cast(col2#16L as bigint) AS col2#20L]
+- Project [col1#15L, col2#16L]
+- Filter (col1#15L > cast(-20 as bigint))
+- SubqueryAlias table1
+- Relation[col1#15L,col2#16L] parquet
== Optimized Logical Plan ==
Filter (isnotnull(col1#15L) && (col1#15L > -20))
+- Relation[col1#15L,col2#16L] parquet
== Physical Plan ==
*Project [col1#15L, col2#16L]
+- *Filter (isnotnull(col1#15L) && (col1#15L > -20))
+- *FileScan parquet default.table1[col1#15L,col2#16L] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/tmp/yumwang/spark/parquet], PartitionFilters: [], PushedFilters: [IsNotNull(col1), GreaterThan(col1,-20)], ReadSchema: struct<col1:bigint,col2:bigint>

SparkQA · 2018-08-18T14:11:54Z

Test build #94921 has finished for PR 22124 at commit 9b16ff0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-18T15:37:51Z

But it is inconsistent now.

Can you point out in the codebase where the inconsistency comes from?

wangyum · 2018-08-18T15:46:02Z

Comes from: [SPARK-22834][SQL] Make insertion commands have real children to fix UI issues.

gengliangwang · 2018-08-19T17:06:05Z

Hi @wangyum , thanks for working on this.
Can you simplify the reproducing case? E.g. Select only one column should be enough.
Also, in the PR description, somehow there are column names SITE_ID and LEAF_CATEG_ID comes from nowhere.

cloud-fan · 2018-08-20T03:12:18Z

@wangyum I know it's from #20020, but do you know which line of the code/which method cause it? We must fully understand the bug before fixing it.

cloud-fan · 2018-08-20T08:44:30Z

+ def apply(plan: LogicalPlan): LogicalPlan = {
+ plan match {
+ case c: Command => c
+ case _ => removeRedundantAliases(plan, AttributeSet.empty)


I don't get it. For the query
*(1) Project [col1#8L AS COL1#14L, col2#9L AS COL2#15L] +- *(1) Filter (isnotnull(col1#8L) && (col1#8L > -20)) +- *(1) FileScan parquet default.table1[col1#8L,col2#9L] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/tmp/yumwang/spark/parquet], PartitionFilters: [], PushedFilters: [IsNotNull(col1), GreaterThan(col1,-20)], ReadSchema: struct<col1:bigint,col2:bigint>
Why is the alias treated as redundant? The name does change, isn't it?

Yes, this is correct. Without this PR, RemoveRedundantAliases works like this:
===ApplyingRule org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-ae504f50-9543-49fb-a InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-ae504f50-9543-49fb-acf Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2003:03:52PDT2018CreatedTime:MonAug2003:03:52PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: hive Provider: hive TableProperties: [transient_lastDdlTime=1534759432] TableProperties: [transient_lastDdlTime=1534759432] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-ae504f50-9543-49fb-acf0-8b2736665d26/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-ae504f50-9543-49fb-acf0-8b2736665d26/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] PartitionProvider:CatalogPartitionProvider:CatalogSchema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) --COL2: long (nullable =true) |--COL2: long (nullable =true) !), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@60582d55, [COL1#10L, COL2#11L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@60582d55, [col1#8L, col2#9L] !+-Project [col1#8LAS col1#10L, col2#9LAS col2#11L] +-Project [col1#8L, col2#9L] +-Filter (col1#8L>-20) +-Filter (col1#8L>-20) +-Relation[col1#8L,col2#9L] parquet +-Relation[col1#8L,col2#9L] parquet

For example:
valpath="/tmp/spark/parquet"valcnt=30 spark.range(cnt).selectExpr("id as col1").write.mode("overwrite").parquet(path) spark.sql(s"CREATE TABLE table1(col1 bigint) using parquet location '$path'") spark.sql("create view view1 as select col1 from table1 where col1 > -20") // The column name of table2 is inconsistent with the column name of view1. spark.sql("create table table2 (COL1 BIGINT) using parquet") // When querying the view, ensure that the column name of the query matches the column name of the target table. spark.sql("insert overwrite table table2 select COL1 from view1")
The execution plan change track:
===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences===!'Project ['idAS col1#2] Project [id#0LAS col1#2L] +-Range (0, 30, step=1, splits=Some(1)) +-Range (0, 30, step=1, splits=Some(1)) 17:02:55.061WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.CleanupAliases===Project [id#0LAS col1#2L] Project [id#0LAS col1#2L] +-Range (0, 30, step=1, splits=Some(1)) +-Range (0, 30, step=1, splits=Some(1)) 17:02:59.174WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.DataSourceAnalysis===!'CreateTable `table1`, ErrorIfExistsCreateDataSourceTableCommand `table1`, false17:02:59.909WARN org.apache.hadoop.hive.metastore.ObjectStore:Failed to get database global_temp, returning NoSuchObjectException17:03:00.094WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations==='Project ['col1] 'Project ['col1] +-'Filter ('col1>-20) +-'Filter ('col1>-20) !+-'UnresolvedRelation `table1` +-'SubqueryAlias `default`.`table1` !+-'UnresolvedCatalogRelation `default`.`table1`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe17:03:00.254WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.FindDataSourceTable==='Project ['col1] 'Project ['col1] +-'Filter ('col1>-20) +-'Filter ('col1>-20) !+-'SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` !+-'UnresolvedCatalogRelation `default`.`table1`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe+-Relation[col1#5L] parquet 17:03:00.267WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences==='Project ['col1] 'Project ['col1] !+-'Filter ('col1>-20) +-'Filter (col1#5L>-20) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.306WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts==='Project ['col1] 'Project ['col1] !+-'Filter (col1#5L>-20) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.309WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences===!'Project ['col1] Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.314WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.ResolveTimeZone===Project [col1#5L] Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.383WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.DataSourceAnalysis===!'CreateTable `table2`, ErrorIfExistsCreateDataSourceTableCommand `table2`, false17:03:00.729WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations==='Project ['col1] 'Project ['col1] +-'Filter ('col1>-20) +-'Filter ('col1>-20) !+-'UnresolvedRelation `table1` +-'SubqueryAlias `default`.`table1` !+-'UnresolvedCatalogRelation `default`.`table1`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe17:03:00.730WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.FindDataSourceTable==='Project ['col1] 'Project ['col1] +-'Filter ('col1>-20) +-'Filter ('col1>-20) !+-'SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` !+-'UnresolvedCatalogRelation `default`.`table1`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe+-Relation[col1#5L] parquet 17:03:00.731WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences==='Project ['col1] 'Project ['col1] !+-'Filter ('col1>-20) +-'Filter (col1#5L>-20) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.734WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts==='Project ['col1] 'Project ['col1] !+-'Filter (col1#5L>-20) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.735WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences===!'Project ['col1] Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.737WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.ResolveTimeZone===Project [col1#5L] Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.742WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations==='InsertIntoTable'UnresolvedRelation `table2`, true, false'InsertIntoTable'UnresolvedRelation `table2`, true, false+-'Project ['COL1] +-'Project ['COL1] !+-'UnresolvedRelation `view1` +-SubqueryAlias `default`.`view1` !+-View (`default`.`view1`, [col1#6L]) !+-Project [col1#5L] !+-Filter (col1#5L> cast(-20asbigint)) !+-SubqueryAlias `default`.`table1` !+-Relation[col1#5L] parquet 17:03:00.744WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences==='InsertIntoTable'UnresolvedRelation `table2`, true, false'InsertIntoTable'UnresolvedRelation `table2`, true, false!+-'Project ['COL1] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) +-Project [col1#5L] +-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.768WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations===!'InsertIntoTable'UnresolvedRelation `table2`, true, false'InsertIntoTable'UnresolvedCatalogRelation `default`.`table2`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, true, false+-Project [COL1#6L] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) +-Project [col1#5L] +-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:00.852WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.FindDataSourceTable===!'InsertIntoTable'UnresolvedCatalogRelation `default`.`table2`, org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, true, false'InsertIntoTableRelation[COL1#7L] parquet, true, false+-Project [COL1#6L] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) +-Project [col1#5L] +-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet DataSourceStrategy1:COL1#8LDataSourceStrategy2:COL1#6L17:03:00.896WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.execution.datasources.DataSourceAnalysis===!'InsertIntoTableRelation[COL1#7L] parquet, true, falseInsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( !+-Project [COL1#6L] Database: default !+-SubqueryAlias `default`.`view1` Table: table2 !+-View (`default`.`view1`, [col1#6L]) Owner: yumwang !+-Project [col1#5L] CreatedTime:MonAug2017:03:00PDT2018!+-Filter (col1#5L> cast(-20asbigint)) LastAccess:WedDec3116:00:00PST1969!+-SubqueryAlias `default`.`table1` CreatedBy:Spark2.4.0-SNAPSHOT!+-Relation[col1#5L] parquet Type:MANAGED!Provider: parquet !TableProperties: [transient_lastDdlTime=1534809780] !Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 !SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe!InputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat!OutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat!StorageProperties: [serialization.format=1] !Schema: root !|--COL1: long (nullable =true) ! ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] !+-Project [COL1#6L] !+-SubqueryAlias `default`.`view1` !+-View (`default`.`view1`, [col1#6L]) !+-Project [col1#5L] !+-Filter (col1#5L> cast(-20asbigint)) !+-SubqueryAlias `default`.`table1` !+-Relation[col1#5L] parquet 17:03:00.916WARN org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1:===ApplyingRule org.apache.spark.sql.catalyst.analysis.AliasViewChild===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [COL1#6L] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) !+-Project [col1#5L] +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Filter (col1#5L> cast(-20asbigint)) +-Project [col1#5L] !+-SubqueryAlias `default`.`table1` +-Filter (col1#5L> cast(-20asbigint)) !+-Relation[col1#5L] parquet +-SubqueryAlias `default`.`table1` !+-Relation[col1#5L] parquet yumwang123:COL1#6L17:03:00.949WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.analysis.EliminateSubqueryAliases===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [COL1#6L] +-Project [COL1#6L] !+-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) !+-View (`default`.`view1`, [col1#6L]) +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Project [cast(col1#5Lasbigint) AS col1#6L] +-Project [col1#5L] !+-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) !+-Filter (col1#5L> cast(-20asbigint)) +-Relation[col1#5L] parquet !+-SubqueryAlias `default`.`table1` !+-Relation[col1#5L] parquet 17:03:00.959WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.analysis.EliminateView===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [COL1#6L] +-Project [COL1#6L] !+-View (`default`.`view1`, [col1#6L]) +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Project [cast(col1#5Lasbigint) AS col1#6L] +-Project [col1#5L] !+-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) !+-Filter (col1#5L> cast(-20asbigint)) +-Relation[col1#5L] parquet !+-Relation[col1#5L] parquet 17:03:00.975WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.ColumnPruning===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] !+-Project [COL1#6L] +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Project [cast(col1#5Lasbigint) AS col1#6L] +-Filter (col1#5L> cast(-20asbigint)) !+-Project [col1#5L] +-Relation[col1#5L] parquet !+-Filter (col1#5L> cast(-20asbigint)) !+-Relation[col1#5L] parquet 17:03:00.980WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.ConstantFolding===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [cast(col1#5Lasbigint) AS col1#6L] +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L>-20) +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:01.047WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.SimplifyCasts===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] !+-Project [cast(col1#5Lasbigint) AS col1#6L] +-Project [col1#5LAS col1#6L] +-Filter (col1#5L>-20) +-Filter (col1#5L>-20) +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:01.058WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) !), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] !+-Project [col1#5LAS col1#6L] +-Project [col1#5L] +-Filter (col1#5L>-20) +-Filter (col1#5L>-20) +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet 17:03:01.061WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.ColumnPruning===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] !+-Project [col1#5L] +-Filter (col1#5L>-20) !+-Filter (col1#5L>-20) +-Relation[col1#5L] parquet !+-Relation[col1#5L] parquet 17:03:01.116WARN org.apache.spark.sql.internal.BaseSessionStateBuilder$$anon$2:===ApplyingRule org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] !+-Filter (col1#5L>-20) +-Filter (isnotnull(col1#5L) && (col1#5L>-20)) +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet queryExecution:==ParsedLogicalPlan=='InsertIntoTable'UnresolvedRelation `table2`, true, false+-'Project ['COL1] +-'UnresolvedRelation `view1` ==AnalyzedLogicalPlan==InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Table: table2 Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDProvider: parquet TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] Schema: root --COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-Project [cast(col1#5Lasbigint) AS col1#6L] +-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet ==OptimizedLogicalPlan==InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Table: table2 Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDProvider: parquet TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] Schema: root --COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] +-Filter (isnotnull(col1#5L) && (col1#5L>-20)) +-Relation[col1#5L] parquet ==PhysicalPlan==ExecuteInsertIntoHadoopFsRelationCommandInsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Table: table2 Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDProvider: parquet TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] Schema: root --COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] +-*(1) Project [col1#5L] +-*(1) Filter (isnotnull(col1#5L) && (col1#5L>-20)) +-*(1) FileScan parquet default.table1[col1#5L] Batched:true, Format:Parquet, Location:InMemoryFileIndex[file:/tmp/spark/parquet], PartitionFilters: [], PushedFilters: [IsNotNull(col1), GreaterThan(col1,-20)], ReadSchema: struct<col1:bigint>
The main 3 changes are:
===ApplyingRule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences==='InsertIntoTable'UnresolvedRelation `table2`, true, false'InsertIntoTable'UnresolvedRelation `table2`, true, false!+-'Project ['COL1] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) +-Project [col1#5L] +-Project [col1#5L] +-Filter (col1#5L> cast(-20asbigint)) +-Filter (col1#5L> cast(-20asbigint)) +-SubqueryAlias `default`.`table1` +-SubqueryAlias `default`.`table1` +-Relation[col1#5L] parquet +-Relation[col1#5L] parquet
===ApplyingRule org.apache.spark.sql.catalyst.analysis.AliasViewChild===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] +-Project [COL1#6L] +-Project [COL1#6L] +-SubqueryAlias `default`.`view1` +-SubqueryAlias `default`.`view1` +-View (`default`.`view1`, [col1#6L]) +-View (`default`.`view1`, [col1#6L]) !+-Project [col1#5L] +-Project [cast(col1#5Lasbigint) AS col1#6L] !+-Filter (col1#5L> cast(-20asbigint)) +-Project [col1#5L] !+-SubqueryAlias `default`.`table1` +-Filter (col1#5L> cast(-20asbigint)) !+-Relation[col1#5L] parquet +-SubqueryAlias `default`.`table1` !+-Relation[col1#5L] parquet
===ApplyingRule org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases===InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( InsertIntoHadoopFsRelationCommand file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2, false, Parquet, Map(serialization.format ->1, path -> file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2), Overwrite, CatalogTable( Database: default Database: default Table: table2 Table: table2 Owner: yumwang Owner: yumwang CreatedTime:MonAug2017:03:00PDT2018CreatedTime:MonAug2017:03:00PDT2018LastAccess:WedDec3116:00:00PST1969LastAccess:WedDec3116:00:00PST1969CreatedBy:Spark2.4.0-SNAPSHOTCreatedBy:Spark2.4.0-SNAPSHOTType:MANAGEDType:MANAGEDProvider: parquet Provider: parquet TableProperties: [transient_lastDdlTime=1534809780] TableProperties: [transient_lastDdlTime=1534809780] Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 Location: file:/private/var/folders/tg/f5mz46090wg7swzgdc69f8q03965_0/T/warehouse-04d554d2-7ddb-4e13-b065-164afe065972/table2 SerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeSerdeLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDeInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatInputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatOutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormatStorageProperties: [serialization.format=1] StorageProperties: [serialization.format=1] Schema: root Schema: root --COL1: long (nullable =true) |--COL1: long (nullable =true) !), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [COL1#6L] ), org.apache.spark.sql.execution.datasources.InMemoryFileIndex@c613921e, [col1#5L] !+-Project [col1#5LAS col1#6L] +-Project [col1#5L] +-Filter (col1#5L>-20) +-Filter (col1#5L>-20) +-Relation[col1#5L] parquet
We need COL1#6L, but after some optimization, the outputColumns changed to col1#5L.

SparkQA · 2018-08-20T11:36:10Z

Test build #94948 has finished for PR 22124 at commit c5a015c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-08-20T17:06:51Z

Test build #94955 has finished for PR 22124 at commit 419a874.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

# Conflicts: # sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala

cloud-fan · 2018-08-21T08:02:05Z

@wangyum please don't rush into code changes, it's more efficient to come up with a good solution before doing any coding work.

We need COL1#6L, but after some optimization ...

This is the key. We must find out which optimizer rule caused it and how.

SparkQA · 2018-08-21T10:09:09Z

Test build #95006 has finished for PR 22124 at commit 72bde20.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2018-08-30T18:15:55Z

close it. I have create a new PR.

insert datasource table may all null when select from view
276879c

cloud-fan reviewed Aug 17, 2018
View reviewed changes

Add new rule to handle Project contains View
bb93ca0

wangyum commented Aug 17, 2018
View reviewed changes

Fix test error
9b16ff0

Do not remove redundant aliases if plan is a Command
c5a015c

cloud-fan reviewed Aug 20, 2018
View reviewed changes

Don't remove redundant aliases in the top project.
419a874

Merge remote-tracking branch 'upstream/master' intoSPARK-25135
5e4d738
# Conflicts: # sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala

Use has been resolved Attribute instead of create new one.
72bde20

wangyum closed this Aug 30, 2018

gengliangwang mentioned this pull request Sep 1, 2018
[WIP][SPARK-25305][SQL] Respect attribute name in CollapseProject and ColumnPruning #22311
Closed

Uh oh!

Conversation

wangyum commented Aug 16, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

wangyum commented Aug 16, 2018

Uh oh!

SparkQA commented Aug 16, 2018

Uh oh!

cloud-fanAug 17, 2018

Choose a reason for hiding this comment

Uh oh!

wangyumAug 17, 2018

Choose a reason for hiding this comment

Uh oh!

wangyumAug 17, 2018

Choose a reason for hiding this comment

Uh oh!

cloud-fan commented Aug 17, 2018

Uh oh!

wangyum commented Aug 17, 2018

Uh oh!

SparkQA commented Aug 17, 2018

Uh oh!

cloud-fan commented Aug 17, 2018

Uh oh!

wangyum commented Aug 18, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Aug 18, 2018

Uh oh!

cloud-fan commented Aug 18, 2018

Uh oh!

wangyum commented Aug 18, 2018

Uh oh!

gengliangwang commented Aug 19, 2018

Uh oh!

cloud-fan commented Aug 20, 2018

Uh oh!

cloud-fanAug 20, 2018

Choose a reason for hiding this comment

Uh oh!

wangyumAug 20, 2018

Choose a reason for hiding this comment

Uh oh!

wangyumAug 21, 2018

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 20, 2018

Uh oh!

SparkQA commented Aug 20, 2018

Uh oh!

cloud-fan commented Aug 21, 2018

Uh oh!

SparkQA commented Aug 21, 2018

Uh oh!

wangyum commented Aug 30, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wangyum commented Aug 16, 2018•
edited
Loading

wangyum commented Aug 18, 2018•
edited
Loading