Skip to content
This repository was archived by the owner on Nov 28, 2025. It is now read-only.

Support compression for output in MR v1. - #147

Open
shishaochen wants to merge 1 commit into
tensorflow:masterfrom
shishaochen:output-codec
Open

Support compression for output in MR v1.#147
shishaochen wants to merge 1 commit into
tensorflow:masterfrom
shishaochen:output-codec

Conversation

@shishaochen

@shishaochenshishaochen commented Nov 5, 2019

Copy link
Copy Markdown

Aligning with org.tensorflow.hadoop.io.TFRecordFileOutputFormat, enable compression in org.tensorflow.hadoop.io.TFRecordFileOutputFormatV1 as well.
To activate compression in old MapReduce APIs, simply specify options as below:

-Dmapred.output.compress=true
-Dmapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec

This pull request can be considered as supplementary to #61.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shishaochen@googlebot