Skip to content

ARROW-63: [C++] Enable ctest to work on systems with Python 3 as the default Python - #42

Closed
wesm wants to merge 1 commit into
apache:masterfrom
wesm:ARROW-63
Closed

ARROW-63: [C++] Enable ctest to work on systems with Python 3 as the default Python#42
wesm wants to merge 1 commit into
apache:masterfrom
wesm:ARROW-63

Conversation

@wesm

@wesmwesm commented Mar 27, 2016

Copy link
Copy Markdown
Member

No description provided.

wesm
wesm reviewed Mar 27, 2016
View reviewed changes
Comment threadcpp/build-support/asan_symbolize.py
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
else:
# Unbuffered output is not supported in Python 3
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w')

@wesmwesmMar 27, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xhochy are you aware of problems that might be introduced by turning off unbuffered output? I did not write this code initially, so there must have been some reason for disabling buffering (a la python -u). Too bad there was no code comment.

@xhochyxhochyMar 27, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code I see no benefit except that it may limit the memory usage for the buffers in case of large stacktraces that can not be processed fast enough by the process consuming sys.stdout.

@wesmwesmMar 27, 2016

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thank you

@wesm
wesm deleted the ARROW-63 branch March 27, 2016 19:28
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Aug 30, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
c1046cb
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Aug 30, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
1f620d3
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Aug 30, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
9a38b88
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Aug 30, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
5a39298
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Sep 4, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
d3cc004
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
wesm added a commit to wesm/arrow that referenced this pull request Sep 8, 2018
@wesm@julienledem
PARQUET-513: Fail build if valgrind finds error during ctest, fix a c…
b47f67e
…ore dump
Also adds an option to use ccache with a custom gcc toolchain.
Fixes an improper memory access stemming from a unit test that had a wrong parameter.
Author: Wes McKinney <wes@cloudera.com>
Closesapache#42 from wesm/PARQUET-513 and squashes the following commits:
43df9e7 [Wes McKinney] Fix core dump from unit test bug
b46c617 [Wes McKinney] Fail build if valgrind finds error during ctest
Change-Id: I4e04fee7db2ed7a0af21ab25b9d7566314d07ac4
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Sep 10, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
aa316c1
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
praveenbingo added a commit to praveenbingo/arrow that referenced this pull request Sep 10, 2018
@praveenbingo
GDV-71:[C++]Made Gandiva JNI a packagable library. (apache#42)
ece64fa
Modified the build to package the gandiva jni as a stand alone library that
can be packaged in the Gandiva JAR.
Also producing two versions of gandiva core - a static and a shared one.
Fixed LLVM dependencies to be target based.
kou pushed a commit that referenced this pull request May 10, 2020
@kiszk@kou
ARROW-8754: [C++][CI] Enable additional tests on s390x
392cb9e
This PR enables tests for `ARROW_COMPUTE`, `ARROW_DATASET`, `ARROW_FILESYSTEM`, `ARROW_HDFS`, `ARROW_ORC`, and `ARROW_IPC` (default on). #7131 enabled a minimal set of tests as a starting point.
I confirmed that these tests pass locally with the current master. In the current TravisCI environment, we cannot see this result due to a lot of error messages in `arrow-utility-test`.
```
$ git log | head -1
commit ed5f534
% ctest
...
Start 1: arrow-array-test
1/51 Test #1: arrow-array-test ..................... Passed 4.62 sec
Start 2: arrow-buffer-test
2/51 Test #2: arrow-buffer-test .................... Passed 0.14 sec
Start 3: arrow-extension-type-test
3/51 Test #3: arrow-extension-type-test ............ Passed 0.12 sec
Start 4: arrow-misc-test
4/51 Test #4: arrow-misc-test ...................... Passed 0.14 sec
Start 5: arrow-public-api-test
5/51 Test #5: arrow-public-api-test ................ Passed 0.12 sec
Start 6: arrow-scalar-test
6/51 Test #6: arrow-scalar-test .................... Passed 0.13 sec
Start 7: arrow-type-test
7/51 Test #7: arrow-type-test ...................... Passed 0.14 sec
Start 8: arrow-table-test
8/51 Test #8: arrow-table-test ..................... Passed 0.13 sec
Start 9: arrow-tensor-test
9/51 Test #9: arrow-tensor-test .................... Passed 0.13 sec
Start 10: arrow-sparse-tensor-test
10/51 Test #10: arrow-sparse-tensor-test ............. Passed 0.16 sec
Start 11: arrow-stl-test
11/51 Test #11: arrow-stl-test ....................... Passed 0.12 sec
Start 12: arrow-concatenate-test
12/51 Test #12: arrow-concatenate-test ............... Passed 0.53 sec
Start 13: arrow-diff-test
13/51 Test #13: arrow-diff-test ...................... Passed 1.45 sec
Start 14: arrow-c-bridge-test
14/51 Test #14: arrow-c-bridge-test .................. Passed 0.18 sec
Start 15: arrow-io-buffered-test
15/51 Test #15: arrow-io-buffered-test ............... Passed 0.20 sec
Start 16: arrow-io-compressed-test
16/51 Test #16: arrow-io-compressed-test ............. Passed 3.48 sec
Start 17: arrow-io-file-test
17/51 Test #17: arrow-io-file-test ................... Passed 0.74 sec
Start 18: arrow-io-hdfs-test
18/51 Test #18: arrow-io-hdfs-test ................... Passed 0.12 sec
Start 19: arrow-io-memory-test
19/51 Test #19: arrow-io-memory-test ................. Passed 2.77 sec
Start 20: arrow-utility-test
20/51 Test #20: arrow-utility-test ...................***Failed 5.65 sec
Start 21: arrow-threading-utility-test
21/51 Test #21: arrow-threading-utility-test ......... Passed 1.34 sec
Start 22: arrow-compute-compute-test
22/51 Test #22: arrow-compute-compute-test ........... Passed 0.13 sec
Start 23: arrow-compute-boolean-test
23/51 Test #23: arrow-compute-boolean-test ........... Passed 0.15 sec
Start 24: arrow-compute-cast-test
24/51 Test #24: arrow-compute-cast-test .............. Passed 0.22 sec
Start 25: arrow-compute-hash-test
25/51 Test #25: arrow-compute-hash-test .............. Passed 2.61 sec
Start 26: arrow-compute-isin-test
26/51 Test #26: arrow-compute-isin-test .............. Passed 0.81 sec
Start 27: arrow-compute-match-test
27/51 Test #27: arrow-compute-match-test ............. Passed 0.40 sec
Start 28: arrow-compute-sort-to-indices-test
28/51 Test #28: arrow-compute-sort-to-indices-test ... Passed 3.33 sec
Start 29: arrow-compute-nth-to-indices-test
29/51 Test #29: arrow-compute-nth-to-indices-test .... Passed 1.51 sec
Start 30: arrow-compute-util-internal-test
30/51 Test #30: arrow-compute-util-internal-test ..... Passed 0.13 sec
Start 31: arrow-compute-add-test
31/51 Test #31: arrow-compute-add-test ............... Passed 0.12 sec
Start 32: arrow-compute-aggregate-test
32/51 Test #32: arrow-compute-aggregate-test ......... Passed 14.70 sec
Start 33: arrow-compute-compare-test
33/51 Test #33: arrow-compute-compare-test ........... Passed 7.96 sec
Start 34: arrow-compute-take-test
34/51 Test #34: arrow-compute-take-test .............. Passed 4.80 sec
Start 35: arrow-compute-filter-test
35/51 Test #35: arrow-compute-filter-test ............ Passed 8.23 sec
Start 36: arrow-dataset-dataset-test
36/51 Test #36: arrow-dataset-dataset-test ........... Passed 0.25 sec
Start 37: arrow-dataset-discovery-test
37/51 Test #37: arrow-dataset-discovery-test ......... Passed 0.13 sec
Start 38: arrow-dataset-file-ipc-test
38/51 Test #38: arrow-dataset-file-ipc-test .......... Passed 0.21 sec
Start 39: arrow-dataset-file-test
39/51 Test #39: arrow-dataset-file-test .............. Passed 0.12 sec
Start 40: arrow-dataset-filter-test
40/51 Test #40: arrow-dataset-filter-test ............ Passed 0.16 sec
Start 41: arrow-dataset-partition-test
41/51 Test #41: arrow-dataset-partition-test ......... Passed 0.13 sec
Start 42: arrow-dataset-scanner-test
42/51 Test #42: arrow-dataset-scanner-test ........... Passed 0.20 sec
Start 43: arrow-filesystem-test
43/51 Test #43: arrow-filesystem-test ................ Passed 1.62 sec
Start 44: arrow-hdfs-test
44/51 Test #44: arrow-hdfs-test ...................... Passed 0.13 sec
Start 45: arrow-feather-test
45/51 Test #45: arrow-feather-test ................... Passed 0.91 sec
Start 46: arrow-ipc-read-write-test
46/51 Test #46: arrow-ipc-read-write-test ............ Passed 5.77 sec
Start 47: arrow-ipc-json-simple-test
47/51 Test #47: arrow-ipc-json-simple-test ........... Passed 0.16 sec
Start 48: arrow-ipc-json-test
48/51 Test #48: arrow-ipc-json-test .................. Passed 0.27 sec
Start 49: arrow-json-integration-test
49/51 Test #49: arrow-json-integration-test .......... Passed 0.13 sec
Start 50: arrow-json-test
50/51 Test #50: arrow-json-test ...................... Passed 0.26 sec
Start 51: arrow-orc-adapter-test
51/51 Test #51: arrow-orc-adapter-test ............... Passed 1.92 sec
98% tests passed, 1 tests failed out of 51
Label Time Summary:
arrow-tests = 27.38 sec (27 tests)
arrow_compute = 45.11 sec (14 tests)
arrow_dataset = 1.21 sec (7 tests)
arrow_ipc = 6.20 sec (3 tests)
unittest = 79.91 sec (51 tests)
Total Test time (real) = 79.99 sec
The following tests FAILED:
20 - arrow-utility-test (Failed)
Errors while running CTest
```
Closes#7142 from kiszk/ARROW-8754
Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jikunshang pushed a commit to jikunshang/arrow that referenced this pull request May 29, 2020
offthewall123
[Detect before initialize] POAE7-288: Detect configuration before ini…
d19d83b
…tialize (apache#42)
* detect InitailPath
* add loaderr when loading persistent-memory.xml
* modify according to comments
* modify according to comments
* bug fix code style
* bug fix
* bug fix
* fix bug & code style
* bug fix & rename
* change to read properties from .properties file
* modify according to comments
* fix according to comments
* if no persistent-memory.properties found, use default setttings
* add persisitent-memory-sample.properties
* delete log
* modify according to comments
* modify according to comments
* modify according to comments
* modify according to comments
* code refact
* bug fix
* bug fix, code style
* code style
* avoid magic num
* code refactor
* modify unit tests
* code style
* revert according to comments
zhztheplayer pushed a commit to zhztheplayer/arrow-1 that referenced this pull request Nov 9, 2021
@zhixingheyi-tian
Add ORC Dataset JNI Support (apache#42)
16e554b
@paleolimbotpaleolimbot mentioned this pull request Jan 28, 2023
Closed
@prniiiprniii mentioned this pull request Jan 12, 2024
Open
@jonkeanejonkeane mentioned this pull request May 11, 2025
Closed
incrypto32 pushed a commit to incrypto32/arrow that referenced this pull request Oct 27, 2025
@alinaliBQ
Enable mock test (apache#42)
fd6c8a4
* Add todo for noauth validation
* mock server with token auth
Add tests
* run same test with both modes
* Enable ODBC tests in workflow
* Switch current test cases to use FlightSQLODBCTestBase
So the tests can be skipped when `TEST_CONNECT_STR` is not set.
* Change tests to run on both mock and remote modes
Wrap usage of TEST_CONNECT_STR where possible
* Rename test fixtures and make connection string functions virtual
* Fix lint issue
* Attempt to enable ODBC build on Windows platforms
* Attempt to fix clang64 and MinGW errors
* Attempt to register ODBC
* Address James' comments
Use constant string for token
* use ServerMiddleware to validate token
cbb330 added a commit to cbb330/arrow that referenced this pull request Feb 20, 2026
@cbb330@claude
Task#16: Integrate TryCountRows into CountRows (apache#42)
a6417b0
- Modified OrcFileFormat::CountRows to use TryCountRows optimization
- If metadata loaded: call TryCountRows immediately (synchronous fast path)
- If not loaded: load metadata async, then call TryCountRows
- TryCountRows returns count if computable from statistics, nullopt otherwise
- nullopt triggers fallback to full scan (default FileFragment behavior)
Flow:
1. CountRows called with predicate
2. Check if metadata cached
- Yes: TryCountRows(predicate) → count or nullopt
- No: async { load metadata, TryCountRows(predicate) }
3. If nullopt: FileFragment base class does full scan
Mirrors Parquet's CountRows (file_parquet.cc:669-684) exactly.
Result: COUNT(*) queries with compatible predicates avoid reading data.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
cbb330 added a commit to cbb330/arrow that referenced this pull request Feb 24, 2026
@cbb330@claude
Task#16: Integrate TryCountRows into CountRows (apache#42)
bd8e680
- Modified OrcFileFormat::CountRows to use TryCountRows optimization
- If metadata loaded: call TryCountRows immediately (synchronous fast path)
- If not loaded: load metadata async, then call TryCountRows
- TryCountRows returns count if computable from statistics, nullopt otherwise
- nullopt triggers fallback to full scan (default FileFragment behavior)
Flow:
1. CountRows called with predicate
2. Check if metadata cached
- Yes: TryCountRows(predicate) → count or nullopt
- No: async { load metadata, TryCountRows(predicate) }
3. If nullopt: FileFragment base class does full scan
Mirrors Parquet's CountRows (file_parquet.cc:669-684) exactly.
Result: COUNT(*) queries with compatible predicates avoid reading data.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wesm@xhochy