[SPARK-14344][SQL] Not creating meta files when summary-metadata property is false in parquet

[kasjain]

What changes were proposed in this pull request?

This change fixes a bug related to the property "parquet.enable.summary-metadata". Currently, the property's value is not considered before writing the metaData (_metadata and _common-metadata files) in the "saveAsParquetFile" code-flow. Hence making this property false still created the meta files.

How was this patch tested?

Integration tests and manual tests

[kasjain]

Can one of the admins verify this patch?

[AmplabJenkins]

Can one of the admins verify this patch?

[HyukjinKwon]

It seems this is fixed in #13455? Could you confirm this please @kasjain ?

[HyukjinKwon]

It seems working fine now. Therefore, it seems not a problem.

test("SPARK-14344 - write metadata")
withSQLConf(ParquetOutputFormat.ENABLE_JOB_SUMMARY->"true") {
withTempPath { dir =>valpath=s"${dir.getCanonicalPath}/part-r-0.parquet"
spark.range(10).write.parquet(path)
valfiles=newFile(path).listFiles()
assert(files.exists(_.getName.endsWith("_common_metadata")))
}
}
withSQLConf(ParquetOutputFormat.ENABLE_JOB_SUMMARY->"false") {
withTempPath { dir =>valpath=s"${dir.getCanonicalPath}/part-r-0.parquet"
spark.range(10).write.parquet(path)
valfiles=newFile(path).listFiles()
assert(!files.exists(_.getName.endsWith("_common_metadata")))
}
}
}