Uh oh!
There was an error while loading. Please reload this page.
ARROW-55: [Python] Fix unit tests in 2.7 - #25
Closed
danrobinson wants to merge 2 commits into
Closed
Conversation
wesm
commented
Mar 16, 2016
Member
thanks dan! Can you add Python 2.7 to the Travis CI build? https://github.com/apache/arrow/blob/master/ci/travis_script_python.sh The miniconda I used in the script is Python 2.7 by default, so you can use the default Python environment for the tests. You may want to turn this bit into a reusable bash function: python --version
which python
# faster builds, please
conda install -y nomkl
# Expensive dependencies install from Continuum package repo
conda install -y pip numpy pandas cython
# Other stuff pip install
pip install -r requirements.txt
export ARROW_HOME=$ARROW_CPP_INSTALL
python setup.py build_ext --inplace
py.test -vv -r sxX pyarrow |
wesm
commented
Mar 16, 2016
Member
+1, thank you! |
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Aug 30, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Aug 30, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Aug 30, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Aug 30, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Sep 4, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
wesm added a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
See PARQUET-470 for more details on why. PARQUET-468 will make these auto-generated. Author: Wes McKinney <wes@cloudera.com> Closesapache#25 from wesm/PARQUET-469 and squashes the following commits: 91a4daa [Wes McKinney] Roll back thirdparty and thrift sources to 0.9.0 Change-Id: Icbc25f3eff30aac813da1005a7878ffe2b1fbc49
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Sep 10, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
praveenbingo added a commit
to praveenbingo/arrow
that referenced
this pull request
Sep 10, 2018
* GDV-58: [CPP] Fix order of includes. Fixing the order of includes to follow style guideline. The order to follow is documented here : https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes Also enabled the check in lint.
kou pushed a commit
that referenced
this pull request
May 10, 2020
This PR enables tests for `ARROW_COMPUTE`, `ARROW_DATASET`, `ARROW_FILESYSTEM`, `ARROW_HDFS`, `ARROW_ORC`, and `ARROW_IPC` (default on). #7131 enabled a minimal set of tests as a starting point. I confirmed that these tests pass locally with the current master. In the current TravisCI environment, we cannot see this result due to a lot of error messages in `arrow-utility-test`. ``` $ git log | head -1 commit ed5f534 % ctest ... Start 1: arrow-array-test 1/51 Test #1: arrow-array-test ..................... Passed 4.62 sec Start 2: arrow-buffer-test 2/51 Test #2: arrow-buffer-test .................... Passed 0.14 sec Start 3: arrow-extension-type-test 3/51 Test #3: arrow-extension-type-test ............ Passed 0.12 sec Start 4: arrow-misc-test 4/51 Test #4: arrow-misc-test ...................... Passed 0.14 sec Start 5: arrow-public-api-test 5/51 Test #5: arrow-public-api-test ................ Passed 0.12 sec Start 6: arrow-scalar-test 6/51 Test #6: arrow-scalar-test .................... Passed 0.13 sec Start 7: arrow-type-test 7/51 Test #7: arrow-type-test ...................... Passed 0.14 sec Start 8: arrow-table-test 8/51 Test #8: arrow-table-test ..................... Passed 0.13 sec Start 9: arrow-tensor-test 9/51 Test #9: arrow-tensor-test .................... Passed 0.13 sec Start 10: arrow-sparse-tensor-test 10/51 Test #10: arrow-sparse-tensor-test ............. Passed 0.16 sec Start 11: arrow-stl-test 11/51 Test #11: arrow-stl-test ....................... Passed 0.12 sec Start 12: arrow-concatenate-test 12/51 Test #12: arrow-concatenate-test ............... Passed 0.53 sec Start 13: arrow-diff-test 13/51 Test #13: arrow-diff-test ...................... Passed 1.45 sec Start 14: arrow-c-bridge-test 14/51 Test #14: arrow-c-bridge-test .................. Passed 0.18 sec Start 15: arrow-io-buffered-test 15/51 Test #15: arrow-io-buffered-test ............... Passed 0.20 sec Start 16: arrow-io-compressed-test 16/51 Test #16: arrow-io-compressed-test ............. Passed 3.48 sec Start 17: arrow-io-file-test 17/51 Test #17: arrow-io-file-test ................... Passed 0.74 sec Start 18: arrow-io-hdfs-test 18/51 Test #18: arrow-io-hdfs-test ................... Passed 0.12 sec Start 19: arrow-io-memory-test 19/51 Test #19: arrow-io-memory-test ................. Passed 2.77 sec Start 20: arrow-utility-test 20/51 Test #20: arrow-utility-test ...................***Failed 5.65 sec Start 21: arrow-threading-utility-test 21/51 Test #21: arrow-threading-utility-test ......... Passed 1.34 sec Start 22: arrow-compute-compute-test 22/51 Test #22: arrow-compute-compute-test ........... Passed 0.13 sec Start 23: arrow-compute-boolean-test 23/51 Test #23: arrow-compute-boolean-test ........... Passed 0.15 sec Start 24: arrow-compute-cast-test 24/51 Test #24: arrow-compute-cast-test .............. Passed 0.22 sec Start 25: arrow-compute-hash-test 25/51 Test #25: arrow-compute-hash-test .............. Passed 2.61 sec Start 26: arrow-compute-isin-test 26/51 Test #26: arrow-compute-isin-test .............. Passed 0.81 sec Start 27: arrow-compute-match-test 27/51 Test #27: arrow-compute-match-test ............. Passed 0.40 sec Start 28: arrow-compute-sort-to-indices-test 28/51 Test #28: arrow-compute-sort-to-indices-test ... Passed 3.33 sec Start 29: arrow-compute-nth-to-indices-test 29/51 Test #29: arrow-compute-nth-to-indices-test .... Passed 1.51 sec Start 30: arrow-compute-util-internal-test 30/51 Test #30: arrow-compute-util-internal-test ..... Passed 0.13 sec Start 31: arrow-compute-add-test 31/51 Test #31: arrow-compute-add-test ............... Passed 0.12 sec Start 32: arrow-compute-aggregate-test 32/51 Test #32: arrow-compute-aggregate-test ......... Passed 14.70 sec Start 33: arrow-compute-compare-test 33/51 Test #33: arrow-compute-compare-test ........... Passed 7.96 sec Start 34: arrow-compute-take-test 34/51 Test #34: arrow-compute-take-test .............. Passed 4.80 sec Start 35: arrow-compute-filter-test 35/51 Test #35: arrow-compute-filter-test ............ Passed 8.23 sec Start 36: arrow-dataset-dataset-test 36/51 Test #36: arrow-dataset-dataset-test ........... Passed 0.25 sec Start 37: arrow-dataset-discovery-test 37/51 Test #37: arrow-dataset-discovery-test ......... Passed 0.13 sec Start 38: arrow-dataset-file-ipc-test 38/51 Test #38: arrow-dataset-file-ipc-test .......... Passed 0.21 sec Start 39: arrow-dataset-file-test 39/51 Test #39: arrow-dataset-file-test .............. Passed 0.12 sec Start 40: arrow-dataset-filter-test 40/51 Test #40: arrow-dataset-filter-test ............ Passed 0.16 sec Start 41: arrow-dataset-partition-test 41/51 Test #41: arrow-dataset-partition-test ......... Passed 0.13 sec Start 42: arrow-dataset-scanner-test 42/51 Test #42: arrow-dataset-scanner-test ........... Passed 0.20 sec Start 43: arrow-filesystem-test 43/51 Test #43: arrow-filesystem-test ................ Passed 1.62 sec Start 44: arrow-hdfs-test 44/51 Test #44: arrow-hdfs-test ...................... Passed 0.13 sec Start 45: arrow-feather-test 45/51 Test #45: arrow-feather-test ................... Passed 0.91 sec Start 46: arrow-ipc-read-write-test 46/51 Test #46: arrow-ipc-read-write-test ............ Passed 5.77 sec Start 47: arrow-ipc-json-simple-test 47/51 Test #47: arrow-ipc-json-simple-test ........... Passed 0.16 sec Start 48: arrow-ipc-json-test 48/51 Test #48: arrow-ipc-json-test .................. Passed 0.27 sec Start 49: arrow-json-integration-test 49/51 Test #49: arrow-json-integration-test .......... Passed 0.13 sec Start 50: arrow-json-test 50/51 Test #50: arrow-json-test ...................... Passed 0.26 sec Start 51: arrow-orc-adapter-test 51/51 Test #51: arrow-orc-adapter-test ............... Passed 1.92 sec 98% tests passed, 1 tests failed out of 51 Label Time Summary: arrow-tests = 27.38 sec (27 tests) arrow_compute = 45.11 sec (14 tests) arrow_dataset = 1.21 sec (7 tests) arrow_ipc = 6.20 sec (3 tests) unittest = 79.91 sec (51 tests) Total Test time (real) = 79.99 sec The following tests FAILED: 20 - arrow-utility-test (Failed) Errors while running CTest ``` Closes#7142 from kiszk/ARROW-8754 Authored-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
paddyroddy referenced
this pull request
in rok/arrow
Jul 19, 2025
pribor pushed a commit
to GlobalWebIndex/arrow
that referenced
this pull request
Oct 24, 2025
cbb330 referenced
this pull request
in cbb330/arrow
Feb 20, 2026
- Converts ORC column statistics into guarantee expressions - Handles all-null case (num_values=0): returns is_null(field) - Handles single-value case (min==max): returns field==value with null handling - Handles range case (min<max): returns field in [min, max] with null handling - Validates statistics: ignores deprecated or corrupted stats (min>max) - Returns nullopt for incomplete or missing statistics (conservative fallback) Verified: - Checks is_statistics_deprecated flag before using stats - Properly handles has_null flag for null value inclusion - Validates min <= max to detect corrupted statistics - Uses std::optional for safe failure handling
cbb330 referenced
this pull request
in cbb330/arrow
Feb 20, 2026
Implements comprehensive edge case testing for statistics-based predicate pushdown. Tests critical scenarios where statistics need special handling. Test coverage: **All-Null Stripes:** - Filters with >= should skip null stripes - IS NULL finds rows only in null stripe - IS VALID skips null stripe - Range filters correctly ignore null stripes **Single-Value Stripes (min = max):** - Equality matches single-value stripe exactly - < operator correctly skips when min=max=value - > operator correctly skips when min=max=value - <= and >= handle boundary correctly - != handles single-value conservatively (reads all stripes) **Partial Overlaps:** - Range filters that partially overlap single-value stripes - Exclusive ranges (>) correctly exclude boundary values **Complex Predicates:** - OR with null checks (value OR IS NULL) - AND with validity checks (value AND IS VALID) Uses OrcTestFileGenerator::MakeFileWithNullStripe and MakeFileWithSingleValueStripe to create controlled test files with 3 stripes each. All tests verify correct row counts after stripe filtering to ensure: 1. No stripes are incorrectly skipped (correctness) 2. Appropriate stripes are skipped when safe (optimization) 3. Edge cases don't cause crashes or data corruption Critical for validating conservative behavior in ambiguous cases. Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
cbb330 referenced
this pull request
in cbb330/arrow
Feb 20, 2026
cbb330 referenced
this pull request
in cbb330/arrow
Feb 24, 2026
- Converts ORC column statistics into guarantee expressions - Handles all-null case (num_values=0): returns is_null(field) - Handles single-value case (min==max): returns field==value with null handling - Handles range case (min<max): returns field in [min, max] with null handling - Validates statistics: ignores deprecated or corrupted stats (min>max) - Returns nullopt for incomplete or missing statistics (conservative fallback) Verified: - Checks is_statistics_deprecated flag before using stats - Properly handles has_null flag for null value inclusion - Validates min <= max to detect corrupted statistics - Uses std::optional for safe failure handling
cbb330 referenced
this pull request
in cbb330/arrow
Feb 24, 2026
Implements comprehensive edge case testing for statistics-based predicate pushdown. Tests critical scenarios where statistics need special handling. Test coverage: **All-Null Stripes:** - Filters with >= should skip null stripes - IS NULL finds rows only in null stripe - IS VALID skips null stripe - Range filters correctly ignore null stripes **Single-Value Stripes (min = max):** - Equality matches single-value stripe exactly - < operator correctly skips when min=max=value - > operator correctly skips when min=max=value - <= and >= handle boundary correctly - != handles single-value conservatively (reads all stripes) **Partial Overlaps:** - Range filters that partially overlap single-value stripes - Exclusive ranges (>) correctly exclude boundary values **Complex Predicates:** - OR with null checks (value OR IS NULL) - AND with validity checks (value AND IS VALID) Uses OrcTestFileGenerator::MakeFileWithNullStripe and MakeFileWithSingleValueStripe to create controlled test files with 3 stripes each. All tests verify correct row counts after stripe filtering to ensure: 1. No stripes are incorrectly skipped (correctness) 2. Appropriate stripes are skipped when safe (optimization) 3. Edge cases don't cause crashes or data corruption Critical for validating conservative behavior in ambiguous cases. Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
cbb330 referenced
this pull request
in cbb330/arrow
Feb 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixing the #define check for Python 2 makes all unit tests pass in Python 2.7.