Uh oh!
There was an error while loading. Please reload this page.
ARROW-739: [C++] Target arrow_jemalloc_objlib with EP dependency to avoid race condition in parallel builds - #453
ARROW-739: [C++] Target arrow_jemalloc_objlib with EP dependency to avoid race condition in parallel builds#453wesm wants to merge 2 commits into
Conversation
…on in parallel builds Change-Id: Ieda1dd130055e0c04e209f652cfbd4fedcdbaae8
wesm
commented
Mar 30, 2017
robertnishihara
commented
Mar 30, 2017
Unfortunately, I think this did not fix the problem. With a modified script I still see the error. Doing Output copied below. |
robertnishihara
commented
Mar 30, 2017
Although it seemed to take me more tries to get it to fail. |
robertnishihara
commented
Mar 30, 2017
I tried some more and the problems seems to still happen on this branch. That said, it may not be high priority since there are workarounds. |
wesm
commented
Mar 30, 2017
I've spent the last 40 minutes googling around trying to find out a solution to the parallel build issue with ExternalProject. What CMake version are you using? Seems there were some bugs that were fixed around this |
robertnishihara
commented
Mar 30, 2017
I was using cmake |
xhochy
commented
Mar 30, 2017
Looking at the log output, the jemalloc install is only started once. I would rather suspect that somehow |
wesm
commented
Mar 30, 2017
Good call. Let me see if changing the build command fixes it |
wesm
commented
Mar 30, 2017
Change-Id: I051b05194cf9387b77e0936bcf73559e521367d3
wesm
commented
Mar 30, 2017
@robertnishihara@xhochy let me know if that works and looks right |
wesm
commented
Mar 30, 2017
closing in favor of #456. @robertnishihara give that one a try when you have a chance |
The error was reported here: https://issues.apache.org/jira/browse/ARROW-1938. Because dictionary types are not supported in writing yet, the code converts the dictionary column to the actual values first before writing. However, the existing code was accidentally using zero as the offset and the length of the column as the size. This resulted in writing all of the column values for each chunk of the column that was supposed to be written. The fix is to pass the offset and size when recursively calling through to WriteColumnChunk with the "flattened" data. Author: Joshua Storck <joshua.storck@twosigma.com> Closesapache#453 from joshuastorck/ARROW_1938 and squashes the following commits: c2af50f [Joshua Storck] Remove extraneous semicolon in unit test 23f5722 [Joshua Storck] Ran clang-format on arrow-reader-writer-test.cc 314b159 [Joshua Storck] Removing print statements from AssertTableEqual f0bc71a [Joshua Storck] Fixing bug reported in https://issues.apache.org/jira/browse/ARROW-1938, namely preventing all of the values in a dictionary column from being written to parquet for each chunk created as a result of specifying row_group_size Change-Id: Ie9fe39fe3e61c6349abab961d7a350c31955be9b
No description provided.