ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473

Closed
ghost wants to merge 9 commits into
masterfrom
unknown repository
Closed

ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom
unknown repository

Conversation

@ghost

@ghostghost commented Jun 4, 2019

Copy link
Copy Markdown

No description provided.

@pitroupitrou changed the title [ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou

pitrou commented Jun 5, 2019

Copy link
Copy Markdown
Member

@kszucs

@pitrou

pitrou commented Jun 6, 2019

Copy link
Copy Markdown
Member

Does it work? I get the following error here:

$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost

ghost commented Jun 6, 2019

Copy link
Copy Markdown
Author

It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/

@pitrou

pitrou commented Jun 6, 2019

Copy link
Copy Markdown
Member

Now I get another error:

$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost

ghost commented Jun 6, 2019•
edited by ghost
Loading

Copy link
Copy Markdown
Author

Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml (of course use linux_python3.7.yaml or your desired config)

@pitrou

pitrou commented Jun 6, 2019

Copy link
Copy Markdown
Member

Thanks :-) It finally worked.

@kszucs

kszucs commented Jun 11, 2019

Copy link
Copy Markdown
Member

Looks good to me, but let me check via crossbow.

@pitrou

pitrou commented Jun 24, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow build -g conda

@ursabot

ursabot commented Jun 24, 2019

Copy link
Copy Markdown 
No such command "build".

@pitrou

pitrou commented Jun 24, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 24, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.

Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4

Crossbow: 'python crossbow.py ...' step's stderr:

Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou

pitrou commented Jun 24, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 24, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.

Revision: 584ba8404c95810b4cf4e59209a4e438036989df

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58

TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status

@pitrou

pitrou commented Jun 24, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 24, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.

Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59

TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status

@pitrou

pitrou commented Jun 24, 2019

Copy link
Copy Markdown
Member

Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs

@kszucs

kszucs commented Jun 24, 2019

Copy link
Copy Markdown
Member

I'm working on it #4649

@pitrou

pitrou commented Jun 26, 2019

Copy link
Copy Markdown
Member

@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)

@ghost

ghost commented Jun 26, 2019

Copy link
Copy Markdown
Author

I just rebased this.

@kszucs

kszucs commented Jun 26, 2019•
edited
Loading

Copy link
Copy Markdown
Member

I've just fetched it :)
Let's fire the packaging tasks.

@kszucs

kszucs commented Jun 26, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 26, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.

Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77

TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor

@pitrou

pitrou commented Jun 26, 2019

Copy link
Copy Markdown
Member

@lihalite Can you add a test that pyarrow.flight imports?

@ghost

ghost commented Jun 26, 2019

Copy link
Copy Markdown
Author

@pitrou done.

@pitrou

pitrou commented Jun 26, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 26, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.

Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79

TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor

@pitrou

pitrou commented Jun 26, 2019

Copy link
Copy Markdown
Member

Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs

@kszucs

kszucs commented Jun 26, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-linux-gcc-py37

@apacheapache deleted a comment from ursabotJun 26, 2019
@kszucs

kszucs commented Jun 26, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-linux-gcc-py37

@ursabot

ursabot commented Jun 26, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.

Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84

TaskStatus
conda-linux-gcc-py37Azure

@kszucs

kszucs commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-linux-gcc-py37

@ursabot

ursabot commented Jun 27, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.

Revision: f612508

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90

TaskStatus
conda-linux-gcc-py37Azure

xhochy
xhochy reviewed Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
-DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
-DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
-DARROW_DEPENDENCY_SOURCE=SYSTEM \
-Duriparser_SOURCE=BUNDLED \

@xhochyxhochyJun 27, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.

@kszucskszucsJun 27, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.

@pitroupitrouJun 27, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?

@wesmwesmJun 27, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor

@kszucs

kszucs commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 27, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.

Revision: e764a59

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91

TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor

@kszucs

kszucs commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36

@ursabot

ursabot commented Jun 27, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.

Revision: c0b95ac

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93

TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure

@kszucs

kszucs commented Jun 27, 2019

Copy link
Copy Markdown
Member

Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.

@kszucs

kszucs commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package -g conda

@ursabot

ursabot commented Jun 27, 2019

Copy link
Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.

Revision: c1baa4c

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94

TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor

@codecov-io

codecov-io commented Jun 27, 2019

Copy link
Copy Markdown

Codecov Report

Merging #4473 into master will decrease coverage by 21.45%.
The diff coverage is 100%.

Impacted file tree graph

@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.

@wesm

wesm commented Jun 27, 2019

Copy link
Copy Markdown
Member

Ugly mess of conda-build warnings in the Windows build

https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532

I'm looking at the Windows build script

@wesm

wesm commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37

@wesm

wesm commented Jun 27, 2019

Copy link
Copy Markdown
Member

@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37

wesm
wesm approved these changes Jun 28, 2019
View reviewed changes

@wesmwesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now

@wesm

wesm commented Jun 28, 2019

Copy link
Copy Markdown
Member

Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27

@wesmwesm closed this in a0c1c04Jun 28, 2019
@kszucs

kszucs commented Jun 28, 2019

Copy link
Copy Markdown
Member

@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.

@asfimportasfimport mentioned this pull request Jun 28, 2019
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
ARROW-5511: [Packaging] Enable Flight in Conda packages - #4473
Closed
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Closed
ARROW-5511: [Packaging] Enable Flight in Conda packages#4473
ghost wants to merge 9 commits into
masterfrom 
unknown repository
Conversation
@ghost
@ghostghost
commented
Jun 4, 2019

Copy link


Copy Markdown

 No description provided. 
@pitroupitrou
changed the title
[ARROW-5511] Enable Flight in Conda packagesARROW-5511: [Packaging] Enable Flight in Conda packagesJun 5, 2019
@pitrou
pitrou
commented
Jun 5, 2019

Copy link


Copy Markdown

Member
@kszucs
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Does it work? I get the following error here:
$ conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1321, in _increment
last_version = str(int(version) + 1) + suffix
ValueError: invalid literal for int() with base 10: 'none'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 13, in top-level template code
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 324, in pin_subpackage
exact, permit_undefined_jinja)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 295, in pin_subpackage_against_outputs
max_pin))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1351, in apply_pin_expressions
v = _increment(v, alpha_ver)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 1323, in _increment
last_version = chr(ord(version) + 1)
TypeError: ord() expected a character, but string of length 4 found

@ghost
ghost
commented
Jun 6, 2019

Copy link


Copy Markdown

Author
It's because ARROW_VERSION needs to be set (the CI scripts do so) env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Now I get another error:
$ ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for arrow-cpp
INFO:conda_build.metadata:Attempting to finalize metadata for arrow-cpp
Collecting package metadata: ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/home/antoine/miniconda3/conda-bld/work Test:
/home/antoine/miniconda3/conda-bld/test_tmp Leaving build/test environments:
Test:
source activate /home/antoine/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ Build:
source activate /home/antoine/miniconda3/conda-bld/_build_env Traceback (most recent call last):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 753, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 473, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 107, in solve_for_transaction
force_remove, force_reinstall)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 145, in solve_for_diff
force_remove)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 242, in solve_final_state
ssc = self._run_sat(ssc)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 475, in _run_sat
conflicting_specs = ssc.r.get_conflicting_specs(tuple(final_environment_specs))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 852, in get_conflicting_specs
reduced_index = self.get_reduced_index(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 356, in get_reduced_index
specs, features = self.verify_specs(specs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 244, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: - re2
- clangdev=7
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/antoine/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
execute(sys.argv[1:])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
verify=args.verify, variants=args.variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
notest=notest,
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
bypass_env_check=bypass_env_check)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 715, in finalize_outputs_pass
output_d = om.get_rendered_output(metadata.name()) or {'name': metadata.name()}
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2095, in get_rendered_output
permit_undefined_jinja=permit_undefined_jinja):
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 2084, in get_rendered_outputs_section
allow_no_other_outputs=permit_undefined_jinja)) or
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/metadata.py", line 1525, in _get_contents
rendered = template.render(environment=env)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/home/antoine/arrow/dev/dev/tasks/conda-recipes/arrow-cpp/meta.yaml", line 46, in top-level template code
- glog
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/jinja_context.py", line 230, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/render.py", line 143, in get_env_dependencies
channel_urls=tuple(m.config.channel_urls))
File "/home/antoine/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 755, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'clangdev=7', 're2'}

@ghost
ghost
commented
Jun 6, 2019•

edited by ghost

Loading

Copy link


Copy Markdown

Author
Ah sorry, forgot conda-forge is needed, and there appears to be a config file to pin certain packages: env ARROW_VERSION=0.14.0 conda build dev/tasks/conda-recipes/arrow-cpp/ -c conda-forge -m dev/tasks/conda-recipes/variants/osx_python3.7.yaml  (of course use linux_python3.7.yaml or your desired config)
@pitrou
pitrou
commented
Jun 6, 2019

Copy link


Copy Markdown

Member
Thanks :-) It finally worked.
@kszucs
kszucs
commented
Jun 11, 2019

Copy link


Copy Markdown

Member
Looks good to me, but let me check via crossbow.
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow build -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

No such command "build".

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24815) builder failed.
Revision: 5685d10c4765b1bf5b34722339825d7f233b51e4
Crossbow: 'python crossbow.py ...' step's stderr:
Traceback (most recent call last):
File "crossbow.py", line 937, in <module>
crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "crossbow.py", line 761, in submit
tasks[name] = Task(artifacts=artifacts, **task)
TypeError: __init__() missing 1 required positional argument: 'ci'

@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24824) builder has been succeeded.
Revision: 584ba8404c95810b4cf4e59209a4e438036989df
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-58
TaskStatus
conda-win-vs2015-py37Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-linux-gcc-py36TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-linux-gcc-py37TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 24, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#24851) builder has been succeeded.
Revision: eda0f67ff60f6d0e5fd51d2e0042a2b34b973cf5
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-59
TaskStatus
conda-linux-gcc-py36TravisCI Status
conda-osx-clang-py27TravisCI Status
conda-linux-gcc-py27TravisCI Status
conda-osx-clang-py37TravisCI Status
conda-win-vs2015-py36Appveyor Status
conda-osx-clang-py36TravisCI Status
conda-win-vs2015-py37Appveyor Status
conda-linux-gcc-py37TravisCI Status
@pitrou
pitrou
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
Unfortunately the crossbow conda builds seem broken on git master. I don't think we can merge this until they're fixed :-/ @wesm@kszucs
@kszucs
kszucs
commented
Jun 24, 2019

Copy link


Copy Markdown

Member
I'm working on it #4649
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@kszucs Could you help rebasing this and fixing conflicts? Thanks :-)
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
I just rebased this.
@kszucs
kszucs
commented
Jun 26, 2019•

edited

Loading

Copy link


Copy Markdown

Member
I've just fetched it :)

Let's fire the packaging tasks.
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26191) builder has been succeeded.
Revision: f1e8b5db3fc27ef8ba6673df5fb0ae911647e3aa
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-77
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@lihalite Can you add a test that pyarrow.flight imports?
@ghost
ghost
commented
Jun 26, 2019

Copy link


Copy Markdown

Author
@pitrou done.
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26209) builder has been succeeded.
Revision: 689d61de2bcf2859cb44cd08f58e9072e0c73214
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-79
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@pitrou
pitrou
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
Not sure what happened but this build fails on uriparser: https://dev.azure.com/ursa-labs/crossbow/_build/results?buildId=123
@kszucs
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@apacheapache
deleted a comment from ursabotJun 26, 2019
@kszucs
kszucs
commented
Jun 26, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 26, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#26379) builder has been succeeded.
Revision: 7ca809be7c46a1f0bf299c4f143ab5dc950e8c61
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-84
TaskStatus
conda-linux-gcc-py37Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-linux-gcc-py37
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27029) builder has been succeeded.
Revision: f612508
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-90
TaskStatus
conda-linux-gcc-py37Azure
xhochy
xhochy
reviewed
Jun 27, 2019
View reviewed changes
Comment threaddev/tasks/conda-recipes/arrow-cpp/build.sh
 -DCMAKE_INSTALL_LIBDIR=$PREFIX/lib \
 -DLLVM_TOOLS_BINARY_DIR=$PREFIX/bin \
 -DARROW_DEPENDENCY_SOURCE=SYSTEM \
 -Duriparser_SOURCE=BUNDLED \
@xhochyxhochyJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we need to package uriparser before we can release this on conda-forge; there bundling of thirdparties is not really accepted.
@kszucskszucsJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, didn't know that.
@pitroupitrouJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for small private dependencies?
@wesmwesmJun 27, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can temporarily statically-link uriparser, we had a bunch of stuff statically linked on conda-forge for a long time before the CMake refactor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27040) builder has been succeeded.
Revision: e764a59
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-91
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-osx-clang-py27 conda-osx-clang-py36
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27173) builder has been succeeded.
Revision: c0b95ac
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-93
TaskStatus
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Conda probably doesn't move the submodules to the testing directory, skipping the tests instead.
@kszucs
kszucs
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package -g conda
@ursabot
ursabot
commented
Jun 27, 2019

Copy link


Copy Markdown

AMD64 Conda Crossbow (#27225) builder has been succeeded.
Revision: c1baa4c
Submitted crossbow builds: ursa-labs/crossbow @ ursabot-94
TaskStatus
conda-linux-gcc-py27Azure
conda-linux-gcc-py36Azure
conda-linux-gcc-py37Azure
conda-osx-clang-py27Azure
conda-osx-clang-py36Azure
conda-osx-clang-py37Azure
conda-win-vs2015-py36Appveyor
conda-win-vs2015-py37Appveyor
@codecov-io
codecov-io
commented
Jun 27, 2019

Copy link


Copy Markdown

Codecov Report
Merging #4473 into master will decrease coverage by 21.45%.

The diff coverage is 100%.
Impacted file tree graph
@@ Coverage Diff @@## master #4473 +/- ##
===========================================
- Coverage 86.42% 64.96% -21.46% 
===========================================
Files 989 482 -507 Lines 137498 63194 -74304 Branches 1418 0 -1418 ===========================================
- Hits 118834 41057 -77777 - Misses 18302 22137 +3835 + Partials 362 0 -362
Impacted FilesCoverage Δ
python/pyarrow/tests/test_flight.py70.58% <100%> (+0.06%)⬆️
cpp/src/arrow/util/memory.h0% <0%> (-100%)⬇️
cpp/src/gandiva/date_utils.h0% <0%> (-100%)⬇️
cpp/src/arrow/util/memory.cc0% <0%> (-100%)⬇️
cpp/src/arrow/filesystem/util-internal.cc0% <0%> (-100%)⬇️
cpp/src/arrow/util/sse-util.h0% <0%> (-100%)⬇️
cpp/src/gandiva/decimal_type_util.h0% <0%> (-100%)⬇️
cpp/src/arrow/compute/logical_type.h0% <0%> (-100%)⬇️
cpp/src/parquet/hasher.h0% <0%> (-100%)⬇️
cpp/src/gandiva/basic_decimal_scalar.h0% <0%> (-100%)⬇️
... and 745 more
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 2264791...c1baa4c. Read the comment docs.
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
Ugly mess of conda-build warnings in the Windows build
https://gist.github.com/wesm/fdbdcfb0882ecba6d27d563566c90532
I'm looking at the Windows build script
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
@wesm
wesm
commented
Jun 27, 2019

Copy link


Copy Markdown

Member
@ursabot crossbow package conda-win-vs2015-py36 conda-win-vs2015-py37
wesm
wesm
approved these changes
Jun 28, 2019
View reviewed changes
@wesmwesm
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The Windows builds are passing now
@wesm
wesm
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
Crossbow builds https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93&query=build-27
@wesmwesm
closed this
in
a0c1c04Jun 28, 2019
@kszucs
kszucs
commented
Jun 28, 2019

Copy link


Copy Markdown

Member
@wesm thanks for fixing the windows builds. When use use [skip ci] pattern in the commit message, then ursabot will skip executing that particular commit. It is because we proxy the request to the standard PR building mechanism of buildbot.
@asfimportasfimport
mentioned this pull request
Jun 28, 2019

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@pitrou@kszucs@ursabot@codecov-io@wesm@xhochy@lidavidm