arrow-cpp: 0.9.0 -> 0.10.0 #47832

veprbl · 2018-10-03T23:32:41Z

pythonPackages.pyarrow: 0.9.0 -> 0.10.0
parquet: 1.4.0 -> 1.5.0

Motivation for this change

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

GrahamcOfBorg · 2018-10-03T23:48:38Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

-- Installing: /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/pkgconfig/arrow-python.pc
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0
shrinking /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/libarrow.so.10.0.0
shrinking /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/libarrow_python.so.10.0.0
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib
patching script interpreter paths in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0
checking for references to /build in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0...
/nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0

GrahamcOfBorg · 2018-10-03T23:49:21Z

Success on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

-- Installing: /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0/lib/pkgconfig/arrow-python.pc
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0
shrinking /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0/lib/libarrow_python.so.10.0.0
shrinking /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0/lib/libarrow.so.10.0.0
strip is /nix/store/428gs2z4b8f9byvghzlpbjwjb3a7jwxx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0/lib
patching script interpreter paths in /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0
checking for references to /build in /nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0...
/nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0

GrahamcOfBorg · 2018-10-03T23:50:15Z

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

-- Installing: /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/include/arrow/python/python_to_arrow.h
-- Installing: /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/include/arrow/python/platform.h
-- Installing: /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/include/arrow/python/pyarrow.h
-- Installing: /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/include/arrow/python/type_traits.h
-- Installing: /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/lib/pkgconfig/arrow-python.pc
post-installation fixup
strip is /nix/store/9xjkb4xz0b5lmizij9ppxy7lkxdxhx6b-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0/lib
patching script interpreter paths in /nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0
/nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0

veprbl · 2018-10-04T00:16:39Z

@GrahamcOfBorg build arrow-cpp parquet-cpp python2Packages.pyarrow python3Packages.pyarrow

GrahamcOfBorg · 2018-10-04T00:26:52Z

Failure on x86_64-darwin (full log)

Attempted: arrow-cpp, parquet-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/nix/store/rpyns4fjgvdwc2v81xjkyv6d0mz3q6ay-python3-3.6.6/lib/python3.6/unittest/loader.py", line 219, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/nix/store/rpyns4fjgvdwc2v81xjkyv6d0mz3q6ay-python3-3.6.6/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
  File "/private/tmp/nix-build-python3.6-pyarrow-0.10.0.drv-0/apache-arrow-0.10.0/python/pyarrow/__init__.py", line 50, in <module>
    import pyarrow.compat as compat
AttributeError: module 'pyarrow' has no attribute 'compat'
builder for '/nix/store/b5jyxmqadhkg07r750nwqvam4caac026-python3.6-pyarrow-0.10.0.drv' failed with exit code 1
error: build of '/nix/store/7fzs8059v77pwkrn996vbmqp6c75i87g-python2.7-pyarrow-0.10.0.drv', '/nix/store/b5jyxmqadhkg07r750nwqvam4caac026-python3.6-pyarrow-0.10.0.drv' failed

GrahamcOfBorg · 2018-10-04T00:27:36Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp, parquet-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

('Moving built C-extension', 'release/_parquet.so', 'to build path', '/build/apache-arrow-0.10.0/python/pyarrow/_parquet.so')
release/_orc.so
Cython module _orc failure permitted
release/_plasma.so
Cython module _plasma failure permitted
pytestcachePhase
/nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0
/nix/store/mscc8a8xrmvmn596hnfgddkzcb586j7x-parquet-cpp-1.5.0
/nix/store/g8c36lpyksrl73zqxmw58kr9d0rs8rn9-python2.7-pyarrow-0.10.0
/nix/store/yawpixc3wa6492jzg3hkw0jd1p15dq00-python3.6-pyarrow-0.10.0

GrahamcOfBorg · 2018-10-04T00:41:49Z

Failure on aarch64-linux (full log)

Attempted: arrow-cpp, parquet-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

  /nix/store/3pwdm1xqvyqj2y9hb0rvszj4jqjpdr2f-python3.6-pandas-0.23.4/lib/python3.6/site-packages/pandas/core/internals.py:312: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
    return self.values[slicer]
  /nix/store/3pwdm1xqvyqj2y9hb0rvszj4jqjpdr2f-python3.6-pandas-0.23.4/lib/python3.6/site-packages/pandas/core/indexes/datetimelike.py:428: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
    result = getitem(key)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
 5 failed, 16897 passed, 1027 skipped, 11896 deselected, 37 xfailed, 2 xpassed, 10 warnings in 590.84 seconds
builder for '/nix/store/62v3rqb77cqxjrx6r7ln3fl3ii8yz885-python3.6-pandas-0.23.4.drv' failed with exit code 1
cannot build derivation '/nix/store/iaf499nwxn18hshaz09c4f6667d1ps97-python3.6-pyarrow-0.10.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/iaf499nwxn18hshaz09c4f6667d1ps97-python3.6-pyarrow-0.10.0.drv', '/nix/store/zfd2syn1hvazx6j13yr2c2q6rhxkbz92-python2.7-pyarrow-0.10.0.drv' failed

veprbl · 2018-10-04T00:43:38Z

Failure on darwin is an old problem. I was never able to reproduce it on my machine, so for now I won't try to fix it.

bhipple · 2018-10-04T01:41:45Z

pkgs/development/libraries/arrow-cpp/default.nix

@@ -1,16 +1,19 @@
-{ stdenv, symlinkJoin, fetchurl, boost, brotli, cmake, flatbuffers, gtest, gflags, lz4, python, rapidjson, snappy, zlib, zstd }:
+{ stdenv, fetchpatch, symlinkJoin, fetchurl, boost, brotli, cmake, flatbuffers, gtest, gflags, lz4, python, rapidjson, snappy, zlib, zstd }:


It looks like the patch is inlined and not actually retrieved via fetchpatch; leftover from development?

Good catch! I had a private overlay with a bugfix that I never made a PR for. Now fixed.

bhipple · 2018-10-04T01:42:40Z

pkgs/development/python-modules/pyarrow/default.nix

@@ -1,4 +1,4 @@
-{ lib, buildPythonPackage, python, isPy3k, fetchurl, arrow-cpp, cmake, cython, futures, numpy, pandas, pytest, pytestrunner, parquet-cpp, pkgconfig, setuptools_scm, six }:
+{ lib, fetchpatch, buildPythonPackage, python, isPy3k, fetchurl, arrow-cpp, cmake, cython, futures, JPype1, numpy, pandas, pytest, pytestrunner, parquet-cpp, pkgconfig, setuptools_scm, six }:


Ditto here.

pythonPackages.pyarrow: 0.9.0 -> 0.10.0 parquet: 1.4.0 -> 1.5.0

GrahamcOfBorg · 2018-10-04T01:57:29Z

Success on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

/nix/store/rmfwmnww0cf4lj1h4ax2r0wr50qpw7i9-arrow-cpp-0.10.0

GrahamcOfBorg · 2018-10-04T01:57:52Z

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

/nix/store/i0cg7wi8bakvxnv1ll25y4gxlcsfbr2q-arrow-cpp-0.10.0

GrahamcOfBorg · 2018-10-04T02:12:07Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

-- Installing: /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/pkgconfig/arrow-python.pc
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0
shrinking /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/libarrow.so.10.0.0
shrinking /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib/libarrow_python.so.10.0.0
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0/lib
patching script interpreter paths in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0
checking for references to /build in /nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0...
/nix/store/wycy9msx7v14lfmmbn7s2vkar58230b0-arrow-cpp-0.10.0

veprbl requested a review from FRidh as a code owner October 3, 2018 23:32

GrahamcOfBorg added 6.topic: python 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 3, 2018

bhipple requested changes Oct 4, 2018

View reviewed changes

arrow-cpp: 0.9.0 -> 0.10.0

3942226

pythonPackages.pyarrow: 0.9.0 -> 0.10.0 parquet: 1.4.0 -> 1.5.0

veprbl force-pushed the pr/arrow_0.10.0 branch from c447a00 to 3942226 Compare October 4, 2018 01:43

Mic92 merged commit a363ed1 into NixOS:master Oct 4, 2018

veprbl deleted the pr/arrow_0.10.0 branch December 1, 2020 17:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

arrow-cpp: 0.9.0 -> 0.10.0 #47832

arrow-cpp: 0.9.0 -> 0.10.0 #47832

veprbl commented Oct 3, 2018

GrahamcOfBorg commented Oct 3, 2018

GrahamcOfBorg commented Oct 3, 2018

GrahamcOfBorg commented Oct 3, 2018

veprbl commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

veprbl commented Oct 4, 2018

bhipple Oct 4, 2018

veprbl Oct 4, 2018

bhipple Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

		@@ -1,16 +1,19 @@
		{ stdenv, symlinkJoin, fetchurl, boost, brotli, cmake, flatbuffers, gtest, gflags, lz4, python, rapidjson, snappy, zlib, zstd }:
		{ stdenv, fetchpatch, symlinkJoin, fetchurl, boost, brotli, cmake, flatbuffers, gtest, gflags, lz4, python, rapidjson, snappy, zlib, zstd }:

		@@ -1,4 +1,4 @@
		{ lib, buildPythonPackage, python, isPy3k, fetchurl, arrow-cpp, cmake, cython, futures, numpy, pandas, pytest, pytestrunner, parquet-cpp, pkgconfig, setuptools_scm, six }:
		{ lib, fetchpatch, buildPythonPackage, python, isPy3k, fetchurl, arrow-cpp, cmake, cython, futures, JPype1, numpy, pandas, pytest, pytestrunner, parquet-cpp, pkgconfig, setuptools_scm, six }:

arrow-cpp: 0.9.0 -> 0.10.0 #47832

arrow-cpp: 0.9.0 -> 0.10.0 #47832

Conversation

veprbl commented Oct 3, 2018

Motivation for this change

Things done

GrahamcOfBorg commented Oct 3, 2018

GrahamcOfBorg commented Oct 3, 2018

GrahamcOfBorg commented Oct 3, 2018

veprbl commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

veprbl commented Oct 4, 2018

bhipple Oct 4, 2018

Choose a reason for hiding this comment

veprbl Oct 4, 2018

Choose a reason for hiding this comment

bhipple Oct 4, 2018

Choose a reason for hiding this comment

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018

GrahamcOfBorg commented Oct 4, 2018