[SPARK-16412][SQL][WIP] Generate Java code that gets an array in each column of CachedBatch when DataFrame.cache() is called by kiszk · Pull Request #14091 · apache/spark

kiszk · 2016-07-07T14:12:28Z

What changes were proposed in this pull request?

Waiting #11956 to be merged.

This PR generates Java code to directly get an array of each column from CachedBatch when DataFrame.cache() is called. This is done in whole stage code generation.

When DataFrame.cache()is called, data is stored as column-oriented storage (columnar cache) inCachedBatch`. This PR avoid conversion from column-oriented storage to row-oriented storage. This PR handles an array type that is stored into a column.

This PR generates code both for row-oriented storage and column-oriented storage only if

InMemoryColumnarTableScan exists in a plan sub-tree. A decision is performed by checking an given iterator is ColumnaIterator at runtime
Sort or join does not exist in a plan sub-tree.

This PR generates Java code for columnar cache only if types in all columns, which are accessed in operations, are primitive or an array

I will add benchmark suites into here

Motivating example

valdf = sc.parallelize(Seq(Array(1.0, 2.0), Array(3.0, 4.0)), 1).toDFdf.cache.filter("value[0] > 2.0").show

Generated code
Before applying this PR

/* 001 */publicObjectgenerate(Object[] references) {
/* 002 */returnnewGeneratedIterator(references);
/* 003 */ }
/* 004 *//* 005 *//** * Codegend pipeline for * Filter (value#1[0] > 2.0) * +- InMemoryTableScan [value#1], [(value#1[0] > 2.0)] * : +- InMemoryRelation [value#1], true, 10000, StorageLevel(disk, memory, deserialized, 1 replicas) * : : +- Scan ExistingRDD[value#1] *//* 006 */finalclassGeneratedIteratorextendsorg.apache.spark.sql.execution.BufferedRowIterator {
/* 007 */privateObject[] references;
/* 008 */privatescala.collection.Iteratorinputadapter_input;
/* 009 */privateorg.apache.spark.sql.execution.metric.SQLMetricfilter_numOutputRows;
/* 010 */privateUnsafeRowfilter_result;
/* 011 */privateorg.apache.spark.sql.catalyst.expressions.codegen.BufferHolderfilter_holder;
/* 012 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriterfilter_rowWriter;
/* 013 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriterfilter_arrayWriter;
/* 014 *//* 015 */publicGeneratedIterator(Object[] references) {
/* 016 */this.references = references;
/* 017 */ }
/* 018 *//* 019 */publicvoidinit(intindex, scala.collection.Iteratorinputs[]) {
/* 020 */partitionIndex = index;
/* 021 */inputadapter_input = inputs[0];
/* 022 */this.filter_numOutputRows = (org.apache.spark.sql.execution.metric.SQLMetric) references[0];
/* 023 */filter_result = newUnsafeRow(1);
/* 024 */this.filter_holder = neworg.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(filter_result, 32);
/* 025 */this.filter_rowWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(filter_holder, 1);
/* 026 */this.filter_arrayWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriter();
/* 027 */ }
/* 028 *//* 029 */protectedvoidprocessNext() throwsjava.io.IOException {
/* 030 */// PRODUCE: Filter (value#1[0] > 2.0)/* 031 */// PRODUCE: InputAdapter/* 032 */while (inputadapter_input.hasNext()) {
/* 033 */InternalRowinputadapter_row = (InternalRow) inputadapter_input.next();
/* 034 */// CONSUME: Filter (value#1[0] > 2.0)/* 035 */// input[0, array<double>, true]/* 036 */booleaninputadapter_isNull = inputadapter_row.isNullAt(0);
/* 037 */ArrayDatainputadapter_value = inputadapter_isNull ? null : (inputadapter_row.getArray(0));
/* 038 *//* 039 */// (input[0, array<double>, true][0] > 2.0)/* 040 */booleanfilter_isNull = true;
/* 041 */booleanfilter_value = false;
/* 042 */// input[0, array<double>, true][0]/* 043 */booleanfilter_isNull1 = true;
/* 044 */doublefilter_value1 = -1.0;
/* 045 *//* 046 */if (!inputadapter_isNull) {
/* 047 */filter_isNull1 = false; // resultCode could change nullability./* 048 *//* 049 */finalintfilter_index = (int) 0;
/* 050 */if (filter_index >= inputadapter_value.numElements() || filter_index < 0 || inputadapter_value.isNullAt(filter_index)) {
/* 051 */filter_isNull1 = true;
/* 052 */ } else {
/* 053 */filter_value1 = inputadapter_value.getDouble(filter_index);
/* 054 */ }
/* 055 *//* 056 */ }
/* 057 */if (!filter_isNull1) {
/* 058 */filter_isNull = false; // resultCode could change nullability./* 059 */filter_value = org.apache.spark.util.Utils.nanSafeCompareDoubles(filter_value1, 2.0D) > 0;
/* 060 *//* 061 */ }
/* 062 */if (filter_isNull || !filter_value) continue;
/* 063 *//* 064 */filter_numOutputRows.add(1);
/* 065 *//* 066 */// CONSUME: WholeStageCodegen/* 067 */filter_holder.reset();
/* 068 *//* 069 */filter_rowWriter.zeroOutNullBytes();
/* 070 *//* 071 */if (inputadapter_isNull) {
/* 072 */filter_rowWriter.setNullAt(0);
/* 073 */ } else {
/* 074 */// Remember the current cursor so that we can calculate how many bytes are/* 075 */// written later./* 076 */finalintfilter_tmpCursor = filter_holder.cursor;
/* 077 *//* 078 */if (inputadapter_valueinstanceofUnsafeArrayData) {
/* 079 */finalintfilter_sizeInBytes = ((UnsafeArrayData) inputadapter_value).getSizeInBytes();
/* 080 */// grow the global buffer before writing data./* 081 */filter_holder.grow(filter_sizeInBytes);
/* 082 */ ((UnsafeArrayData) inputadapter_value).writeToMemory(filter_holder.buffer, filter_holder.cursor);
/* 083 */filter_holder.cursor += filter_sizeInBytes;
/* 084 *//* 085 */ } else {
/* 086 */finalintfilter_numElements = inputadapter_value.numElements();
/* 087 */filter_arrayWriter.initialize(filter_holder, filter_numElements, 8);
/* 088 *//* 089 */for (intfilter_index1 = 0; filter_index1 < filter_numElements; filter_index1++) {
/* 090 */if (inputadapter_value.isNullAt(filter_index1)) {
/* 091 */filter_arrayWriter.setNullAt(filter_index1);
/* 092 */ } else {
/* 093 */finaldoublefilter_element = inputadapter_value.getDouble(filter_index1);
/* 094 */filter_arrayWriter.write(filter_index1, filter_element);
/* 095 */ }
/* 096 */ }
/* 097 */ }
/* 098 *//* 099 */filter_rowWriter.setOffsetAndSize(0, filter_tmpCursor, filter_holder.cursor - filter_tmpCursor);
/* 100 */filter_rowWriter.alignToWords(filter_holder.cursor - filter_tmpCursor);
/* 101 */ }
/* 102 */filter_result.setTotalSize(filter_holder.totalSize());
/* 103 */append(filter_result);
/* 104 */if (shouldStop()) return;
/* 105 */ }
/* 106 */ }
/* 107 *//* 108 */ }

After applying this PR

/* 001 */publicObjectgenerate(Object[] references) {
/* 002 */returnnewGeneratedIterator(references);
/* 003 */ }
/* 004 *//* 005 *//** * Codegend pipeline for * Filter (value#1[0] > 2.0) * +- InMemoryTableScan [value#1], [(value#1[0] > 2.0)] * : +- InMemoryRelation [value#1], true, 10000, StorageLevel(disk, memory, deserialized, 1 replicas) * : : +- Scan ExistingRDD[value#1] *//* 006 */finalclassGeneratedIteratorextendsorg.apache.spark.sql.execution.BufferedRowIterator {
/* 007 */privateObject[] references;
/* 008 */privatescala.collection.Iteratorinputadapter_input;
/* 009 */privateorg.apache.spark.sql.execution.metric.SQLMetricfilter_numOutputRows;
/* 010 */privateUnsafeRowfilter_result;
/* 011 */privateorg.apache.spark.sql.catalyst.expressions.codegen.BufferHolderfilter_holder;
/* 012 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriterfilter_rowWriter;
/* 013 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriterfilter_arrayWriter;
/* 014 */privatescala.collection.Iteratorinputadapter_input1;
/* 015 */privateintcolumnar_batchIdx;
/* 016 */privateintcolumnar_numRows;
/* 017 */privateorg.apache.spark.sql.execution.vectorized.ColumnVectorinputadapter_col0;
/* 018 */privateUnsafeRowinputadapter_result;
/* 019 */privateorg.apache.spark.sql.catalyst.expressions.codegen.BufferHolderinputadapter_holder;
/* 020 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriterinputadapter_rowWriter;
/* 021 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriterinputadapter_arrayWriter;
/* 022 */privateorg.apache.spark.sql.execution.metric.SQLMetricfilter_numOutputRows1;
/* 023 */privateUnsafeRowfilter_result1;
/* 024 */privateorg.apache.spark.sql.catalyst.expressions.codegen.BufferHolderfilter_holder1;
/* 025 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriterfilter_rowWriter1;
/* 026 */privateorg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriterfilter_arrayWriter1;
/* 027 */privateorg.apache.spark.sql.execution.columnar.ColumnarIteratorcolumnar_itr;
/* 028 *//* 029 */publicGeneratedIterator(Object[] references) {
/* 030 */this.references = references;
/* 031 */ }
/* 032 *//* 033 */publicvoidinit(intindex, scala.collection.Iteratorinputs[]) {
/* 034 */partitionIndex = index;
/* 035 */inputadapter_input = inputs[0];
/* 036 */this.filter_numOutputRows = (org.apache.spark.sql.execution.metric.SQLMetric) references[0];
/* 037 */filter_result = newUnsafeRow(1);
/* 038 */this.filter_holder = neworg.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(filter_result, 32);
/* 039 */this.filter_rowWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(filter_holder, 1);
/* 040 */this.filter_arrayWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriter();
/* 041 */inputadapter_input1 = inputs[0];
/* 042 */columnar_batchIdx = 0;
/* 043 */columnar_numRows = 0;
/* 044 */inputadapter_col0 = null;
/* 045 */inputadapter_result = newUnsafeRow(1);
/* 046 */this.inputadapter_holder = neworg.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(inputadapter_result, 32);
/* 047 */this.inputadapter_rowWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(inputadapter_holder, 1);
/* 048 */this.inputadapter_arrayWriter = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriter();
/* 049 */this.filter_numOutputRows1 = (org.apache.spark.sql.execution.metric.SQLMetric) references[1];
/* 050 */filter_result1 = newUnsafeRow(1);
/* 051 */this.filter_holder1 = neworg.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(filter_result1, 32);
/* 052 */this.filter_rowWriter1 = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(filter_holder1, 1);
/* 053 */this.filter_arrayWriter1 = neworg.apache.spark.sql.catalyst.expressions.codegen.UnsafeArrayWriter();
/* 054 */columnar_itr = null;
/* 055 */ }
/* 056 *//* 057 */privatevoidprocessBatch() throwsjava.io.IOException {
/* 058 */// PRODUCE: Filter (value#1[0] > 2.0)/* 059 */// PRODUCE: InputAdapter/* 060 */while (true) {
/* 061 */if (columnar_batchIdx == 0) {
/* 062 */columnar_numRows = columnar_itr.initForColumnar();
/* 063 */if (columnar_numRows < 0) {
/* 064 */cleanup();
/* 065 */break;
/* 066 */ }
/* 067 */inputadapter_col0 = columnar_itr.getColumn(0);
/* 068 */ }
/* 069 *//* 070 */while (columnar_batchIdx < columnar_numRows) {
/* 071 */intinputadapter_rowIdx = columnar_batchIdx++;
/* 072 */// CONSUME: Filter (value#1[0] > 2.0)/* 073 */// columnVector[inputadapter_col0, inputadapter_rowIdx, array<double>]/* 074 */booleaninputadapter_isNull1 = inputadapter_col0.isNullAt(inputadapter_rowIdx);
/* 075 */ArrayDatainputadapter_value1 = inputadapter_isNull1 ? null : (inputadapter_col0.getArray(inputadapter_rowIdx));
/* 076 *//* 077 */// (input[0, array<double>, true][0] > 2.0)/* 078 */booleanfilter_isNull6 = true;
/* 079 */booleanfilter_value6 = false;
/* 080 */// input[0, array<double>, true][0]/* 081 */booleanfilter_isNull7 = true;
/* 082 */doublefilter_value7 = -1.0;
/* 083 *//* 084 */if (!inputadapter_isNull1) {
/* 085 */filter_isNull7 = false; // resultCode could change nullability./* 086 *//* 087 */finalintfilter_index2 = (int) 0;
/* 088 */if (filter_index2 >= inputadapter_value1.numElements() || filter_index2 < 0 || inputadapter_value1.isNullAt(filter_index2)) {
/* 089 */filter_isNull7 = true;
/* 090 */ } else {
/* 091 */filter_value7 = inputadapter_value1.getDouble(filter_index2);
/* 092 */ }
/* 093 *//* 094 */ }
/* 095 */if (!filter_isNull7) {
/* 096 */filter_isNull6 = false; // resultCode could change nullability./* 097 */filter_value6 = org.apache.spark.util.Utils.nanSafeCompareDoubles(filter_value7, 2.0D) > 0;
/* 098 *//* 099 */ }
/* 100 */if (filter_isNull6 || !filter_value6) continue;
/* 101 *//* 102 */filter_numOutputRows1.add(1);
/* 103 *//* 104 */// CONSUME: WholeStageCodegen/* 105 */filter_holder1.reset();
/* 106 *//* 107 */filter_rowWriter1.zeroOutNullBytes();
/* 108 *//* 109 */if (inputadapter_isNull1) {
/* 110 */filter_rowWriter1.setNullAt(0);
/* 111 */ } else {
/* 112 */// Remember the current cursor so that we can calculate how many bytes are/* 113 */// written later./* 114 */finalintfilter_tmpCursor1 = filter_holder1.cursor;
/* 115 *//* 116 */if (inputadapter_value1instanceofUnsafeArrayData) {
/* 117 */finalintfilter_sizeInBytes1 = ((UnsafeArrayData) inputadapter_value1).getSizeInBytes();
/* 118 */// grow the global buffer before writing data./* 119 */filter_holder1.grow(filter_sizeInBytes1);
/* 120 */ ((UnsafeArrayData) inputadapter_value1).writeToMemory(filter_holder1.buffer, filter_holder1.cursor);
/* 121 */filter_holder1.cursor += filter_sizeInBytes1;
/* 122 *//* 123 */ } else {
/* 124 */finalintfilter_numElements1 = inputadapter_value1.numElements();
/* 125 */filter_arrayWriter1.initialize(filter_holder1, filter_numElements1, 8);
/* 126 *//* 127 */for (intfilter_index3 = 0; filter_index3 < filter_numElements1; filter_index3++) {
/* 128 */if (inputadapter_value1.isNullAt(filter_index3)) {
/* 129 */filter_arrayWriter1.setNullAt(filter_index3);
/* 130 */ } else {
/* 131 */finaldoublefilter_element1 = inputadapter_value1.getDouble(filter_index3);
/* 132 */filter_arrayWriter1.write(filter_index3, filter_element1);
/* 133 */ }
/* 134 */ }
/* 135 */ }
/* 136 *//* 137 */filter_rowWriter1.setOffsetAndSize(0, filter_tmpCursor1, filter_holder1.cursor - filter_tmpCursor1);
/* 138 */filter_rowWriter1.alignToWords(filter_holder1.cursor - filter_tmpCursor1);
/* 139 */ }
/* 140 */filter_result1.setTotalSize(filter_holder1.totalSize());
/* 141 */append(filter_result1);
/* 142 */if (shouldStop()) return;
/* 143 */ }
/* 144 */columnar_batchIdx = 0;
/* 145 */ }
/* 146 */ }
/* 147 *//* 148 */privatevoidprocessRow() throwsjava.io.IOException {
/* 149 */// PRODUCE: Filter (value#1[0] > 2.0)/* 150 */// PRODUCE: InputAdapter/* 151 */while (inputadapter_input.hasNext()) {
/* 152 */InternalRowinputadapter_row = (InternalRow) inputadapter_input.next();
/* 153 */// CONSUME: Filter (value#1[0] > 2.0)/* 154 */// input[0, array<double>, true]/* 155 */booleaninputadapter_isNull = inputadapter_row.isNullAt(0);
/* 156 */ArrayDatainputadapter_value = inputadapter_isNull ? null : (inputadapter_row.getArray(0));
/* 157 *//* 158 */// (input[0, array<double>, true][0] > 2.0)/* 159 */booleanfilter_isNull = true;
/* 160 */booleanfilter_value = false;
/* 161 */// input[0, array<double>, true][0]/* 162 */booleanfilter_isNull1 = true;
/* 163 */doublefilter_value1 = -1.0;
/* 164 *//* 165 */if (!inputadapter_isNull) {
/* 166 */filter_isNull1 = false; // resultCode could change nullability./* 167 *//* 168 */finalintfilter_index = (int) 0;
/* 169 */if (filter_index >= inputadapter_value.numElements() || filter_index < 0 || inputadapter_value.isNullAt(filter_index)) {
/* 170 */filter_isNull1 = true;
/* 171 */ } else {
/* 172 */filter_value1 = inputadapter_value.getDouble(filter_index);
/* 173 */ }
/* 174 *//* 175 */ }
/* 176 */if (!filter_isNull1) {
/* 177 */filter_isNull = false; // resultCode could change nullability./* 178 */filter_value = org.apache.spark.util.Utils.nanSafeCompareDoubles(filter_value1, 2.0D) > 0;
/* 179 *//* 180 */ }
/* 181 */if (filter_isNull || !filter_value) continue;
/* 182 *//* 183 */filter_numOutputRows.add(1);
/* 184 *//* 185 */// CONSUME: WholeStageCodegen/* 186 */filter_holder.reset();
/* 187 *//* 188 */filter_rowWriter.zeroOutNullBytes();
/* 189 *//* 190 */if (inputadapter_isNull) {
/* 191 */filter_rowWriter.setNullAt(0);
/* 192 */ } else {
/* 193 */// Remember the current cursor so that we can calculate how many bytes are/* 194 */// written later./* 195 */finalintfilter_tmpCursor = filter_holder.cursor;
/* 196 *//* 197 */if (inputadapter_valueinstanceofUnsafeArrayData) {
/* 198 */finalintfilter_sizeInBytes = ((UnsafeArrayData) inputadapter_value).getSizeInBytes();
/* 199 */// grow the global buffer before writing data./* 200 */filter_holder.grow(filter_sizeInBytes);
/* 201 */ ((UnsafeArrayData) inputadapter_value).writeToMemory(filter_holder.buffer, filter_holder.cursor);
/* 202 */filter_holder.cursor += filter_sizeInBytes;
/* 203 *//* 204 */ } else {
/* 205 */finalintfilter_numElements = inputadapter_value.numElements();
/* 206 */filter_arrayWriter.initialize(filter_holder, filter_numElements, 8);
/* 207 *//* 208 */for (intfilter_index1 = 0; filter_index1 < filter_numElements; filter_index1++) {
/* 209 */if (inputadapter_value.isNullAt(filter_index1)) {
/* 210 */filter_arrayWriter.setNullAt(filter_index1);
/* 211 */ } else {
/* 212 */finaldoublefilter_element = inputadapter_value.getDouble(filter_index1);
/* 213 */filter_arrayWriter.write(filter_index1, filter_element);
/* 214 */ }
/* 215 */ }
/* 216 */ }
/* 217 *//* 218 */filter_rowWriter.setOffsetAndSize(0, filter_tmpCursor, filter_holder.cursor - filter_tmpCursor);
/* 219 */filter_rowWriter.alignToWords(filter_holder.cursor - filter_tmpCursor);
/* 220 */ }
/* 221 */filter_result.setTotalSize(filter_holder.totalSize());
/* 222 */append(filter_result);
/* 223 */if (shouldStop()) return;
/* 224 */ }
/* 225 */ }
/* 226 *//* 227 */privatevoidcleanup() {
/* 228 */inputadapter_col0 = null;
/* 229 *//* 230 */columnar_itr = null;
/* 231 */ }
/* 232 *//* 233 */protectedvoidprocessNext() throwsjava.io.IOException {
/* 234 */if ((columnar_batchIdx != 0) ||
/* 235 */ (inputadapter_input1instanceoforg.apache.spark.sql.execution.columnar.ColumnarIterator &&
/* 236 */ (columnar_itr = (org.apache.spark.sql.execution.columnar.ColumnarIterator)inputadapter_input1).isSupportColumnarCodeGen())) {
/* 237 */processBatch();
/* 238 */ } else {
/* 239 */processRow();
/* 240 */ }
/* 241 */ }
/* 242 */ }

How was this patch tested?

Added new tests into DataFrameCacheSuite.scala

SparkQA · 2016-07-07T14:21:49Z

Test build #61913 has finished for PR 14091 at commit 54df41c.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

remove duplicated final attribute at method declaration

SparkQA · 2016-08-08T07:03:48Z

Test build #63350 has finished for PR 14091 at commit 61a4754.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2017-05-11T12:55:12Z

Hi @kiszk, I just wonder if it is still WIP (just for curiosity).

## What changes were proposed in this pull request? This PR proposes to close PRs ... - inactive to the review comments more than a month - WIP and inactive more than a month - with Jenkins build failure but inactive more than a month - suggested to be closed and no comment against that - obviously looking inappropriate (e.g., Branch 0.5) To make sure, I left a comment for each PR about a week ago and I could not have a response back from the author in these PRs below: Closesapache#11129Closesapache#12085Closesapache#12162Closesapache#12419Closesapache#12420Closesapache#12491Closesapache#13762Closesapache#13837Closesapache#13851Closesapache#13881Closesapache#13891Closesapache#13959Closesapache#14091Closesapache#14481Closesapache#14547Closesapache#14557Closesapache#14686Closesapache#15594Closesapache#15652Closesapache#15850Closesapache#15914Closesapache#15918Closesapache#16285Closesapache#16389Closesapache#16652Closesapache#16743Closesapache#16893Closesapache#16975Closesapache#17001Closesapache#17088Closesapache#17119Closesapache#17272Closesapache#17971 Added: Closesapache#17778Closesapache#17303Closesapache#17872 ## How was this patch tested? N/A Author: hyukjinkwon <gurwls223@gmail.com> Closesapache#18017 from HyukjinKwon/close-inactive-prs.

kiszk added 3 commits July 7, 2016 19:46

support codegen for an array in CachedBatch
09af5a5

update test suites
8e218e3

remove debug print
54df41c

kiszk changed the title ~~[SPARK-16412][SQL] Generate Java code that gets an array in each column of CachedBatch when DataFrame.cache() is called~~[SPARK-16412][SQL][WIP] Generate Java code that gets an array in each column of CachedBatch when DataFrame.cache() is calledJul 8, 2016

alleviate performance overhead in constructor
61a4754
remove duplicated final attribute at method declaration

HyukjinKwon mentioned this pull request May 17, 2017
[INFRA] Close stale PRs #18017
Closed

asfgit closed this in 5d2750aMay 18, 2017

Navigation Menu

Uh oh!

[SPARK-16412][SQL][WIP] Generate Java code that gets an array in each column of CachedBatch when DataFrame.cache() is called - #14091

[SPARK-16412][SQL][WIP] Generate Java code that gets an array in each column of CachedBatch when DataFrame.cache() is called#14091
kiszk wants to merge 4 commits into
apache:masterfrom
kiszk:SPARK-16412

kiszk commented Jul 7, 2016•
edited
Loading

Uh oh!

SparkQA commented Jul 7, 2016

Uh oh!

SparkQA commented Aug 8, 2016

Uh oh!

HyukjinKwon commented May 11, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kiszk commented Jul 7, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Jul 7, 2016

Uh oh!

SparkQA commented Aug 8, 2016

Uh oh!

HyukjinKwon commented May 11, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kiszk commented Jul 7, 2016•
edited
Loading