InternalParquetRecordReader Logging is Too Verbose

[asfimport]

A low-level library like Parquet should be pretty quiet. It should just do its work and keep quiet. Most issues should be addressed by throwing Exceptions, and the occasional warning message otherwise it will clutter the logging for the top-level application. If debugging is required, administrator can enable it for the specific workload.

Warning: This is my opinion. No stats to back it up.

Reporter: David Mollitor / @belugabehr
Assignee: David Mollitor / @belugabehr

Related issues:

• Lower Logging Level in ParquetOutputFormat (is related to)

PRs and other links:

• GitHub Pull Request #742

_{Note: This issue was originally created as PARQUET-1758. Please see the migration documentation for further details.}

[asfimport]

Michael Heuer:
+1, excessive logging from Parquet has been a pain for us downstream for many years

[asfimport]

Junjie Chen / @chenjunjiedada:
It might be better to draft a discussion on mail list for this, some user may use these logs for analysis. The debug mode might impact the performance.

[asfimport]

David Mollitor / @belugabehr:
I am certainly open for discussions.  I too have had some logging pain emanating from Parquet with the Apache Hive project.

Debug logging would only help performance since less time would be spent logging.

[asfimport]

David Mollitor / @belugabehr:
I think the general idea is that almost all logging is DEBUG level for such a library.  It may be advantageous to setup YETUS so that the automated builds are with DEBUG log enabled, but my feeling is that most logging shouldn't be enabled by default.