arrow-cpp: 0.10.0 -> 0.11.0 #48066

veprbl · 2018-10-08T18:36:05Z

Motivation for this change

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

veprbl · 2018-10-08T18:36:50Z

@GrahamcOfBorg build arrow-cpp python2Packages.pyarrow python3Packages.pyarrow

GrahamcOfBorg · 2018-10-08T18:48:25Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

Moving generated C++ source _parquet.cpp to build path /build/apache-arrow-0.11.0/python/pyarrow/_parquet.cpp
Moving built C-extension release/_parquet.cpython-36m-x86_64-linux-gnu.so to build path /build/apache-arrow-0.11.0/python/pyarrow/_parquet.cpython-36m-x86_64-linux-gnu.so
release/_orc.cpython-36m-x86_64-linux-gnu.so
Cython module _orc failure permitted
release/_plasma.cpython-36m-x86_64-linux-gnu.so
Cython module _plasma failure permitted
pytestcachePhase
/nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0
/nix/store/00vyqililra74cbaav0ry7fmpcgl94wr-python2.7-pyarrow-0.11.0
/nix/store/mz68s69can7ii677rrnaxpmq3n7si7bx-python3.6-pyarrow-0.11.0

GrahamcOfBorg · 2018-10-08T19:07:08Z

Success on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

/nix/store/z9wx4ndc0aza98j54k7hxhynwmjs84if-arrow-cpp-0.11.0

GrahamcOfBorg · 2018-10-08T19:10:38Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0
shrinking /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0/lib/libarrow.so.11.0.0
shrinking /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0/lib/libarrow_python.so.11.0.0
shrinking /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0/lib/libparquet.so.11.0.0
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0/lib
patching script interpreter paths in /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0
checking for references to /build in /nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0...
/nix/store/p59817193q0shn9mkmwrnayxwx79dchh-arrow-cpp-0.11.0

GrahamcOfBorg · 2018-10-08T19:18:18Z

Failure on x86_64-darwin (full log)

Attempted: arrow-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/nix/store/rpyns4fjgvdwc2v81xjkyv6d0mz3q6ay-python3-3.6.6/lib/python3.6/unittest/loader.py", line 219, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/nix/store/rpyns4fjgvdwc2v81xjkyv6d0mz3q6ay-python3-3.6.6/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
  File "/private/tmp/nix-build-python3.6-pyarrow-0.11.0.drv-0/apache-arrow-0.11.0/python/pyarrow/__init__.py", line 45, in <module>
    import pyarrow.compat as compat
AttributeError: module 'pyarrow' has no attribute 'compat'
builder for '/nix/store/l958k2bk53aifi32smrlcmwrgfh4jywp-python3.6-pyarrow-0.11.0.drv' failed with exit code 1
error: build of '/nix/store/9f75qnljivfb985hpjwryarik1f1kp2l-python2.7-pyarrow-0.11.0.drv', '/nix/store/l958k2bk53aifi32smrlcmwrgfh4jywp-python3.6-pyarrow-0.11.0.drv' failed

GrahamcOfBorg · 2018-10-08T19:26:28Z

Failure on aarch64-linux (full log)

Attempted: arrow-cpp, python2Packages.pyarrow, python3Packages.pyarrow

Partial log (click to expand)

  /nix/store/dsl1zn5j39iwc1dgmkndazn48mcyps0y-python2.7-pandas-0.23.4/lib/python2.7/site-packages/pandas/core/internals.py:312: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
    return self.values[slicer]
  /nix/store/dsl1zn5j39iwc1dgmkndazn48mcyps0y-python2.7-pandas-0.23.4/lib/python2.7/site-packages/pandas/core/indexes/datetimelike.py:428: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
    result = getitem(key)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
 5 failed, 16896 passed, 1030 skipped, 11887 deselected, 37 xfailed, 2 xpassed, 6 warnings in 587.23 seconds
builder for '/nix/store/khzgx4hmpgl197sbxbygwmn9kxpnmyf0-python2.7-pandas-0.23.4.drv' failed with exit code 1
cannot build derivation '/nix/store/4dvn6x08nrfy94ym1cbz5b9kvmcdmfdw-python2.7-pyarrow-0.11.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/4dvn6x08nrfy94ym1cbz5b9kvmcdmfdw-python2.7-pyarrow-0.11.0.drv', '/nix/store/g18lpqixfgfn367mszagb9fndx08j4nj-python3.6-pyarrow-0.11.0.drv' failed

GrahamcOfBorg · 2018-10-08T19:42:21Z

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

/nix/store/c62ib4lmwkmqjfsg4a16v5qwz6rrxgls-arrow-cpp-0.11.0

veprbl · 2018-10-08T23:20:56Z

pkgs/top-level/aliases.nix

@@ -202,6 +202,7 @@ mapAliases ({
  openssh_with_kerberos = openssh; # added 2018-01-28
  owncloudclient = owncloud-client;  # added 2016-08
  p11_kit = p11-kit; # added 2018-02-25
+  parquet-cpp = arrow-cpp; # added 2018-09-08


This alias doesn't seem to be too useful. Say I have an override like

arrow-cpp = self.arrow-cpp.override { xxx=yyy; }; parquet-cpp = self.parquet-cpp.override { inherit (self) arrow-cpp; };

I will now get an error:

anonymous function at arrow-cpp/default.nix:1:1 called with unexpected argument 'arrow-cpp', at nixpkgs/lib/customisation.nix:74:12

GrahamcOfBorg · 2018-10-09T20:36:50Z

Failure on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

         51 - parquet-types-test (BAD_COMMAND)
         52 - parquet-reader-test (BAD_COMMAND)
         53 - parquet-schema-test (BAD_COMMAND)
         54 - parquet-arrow-schema-test (BAD_COMMAND)
         55 - parquet-arrow-reader-writer-test (BAD_COMMAND)
         56 - parquet-comparison-test (BAD_COMMAND)
         57 - parquet-memory-test (BAD_COMMAND)
Errors while running CTest
builder for '/nix/store/qz60nyn7sz5b9108jld2pfww2h6iqv2v-arrow-cpp-0.11.0.drv' failed with exit code 8
error: build of '/nix/store/qz60nyn7sz5b9108jld2pfww2h6iqv2v-arrow-cpp-0.11.0.drv' failed

GrahamcOfBorg · 2018-10-09T20:36:57Z

Failure on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

         51 - parquet-types-test (BAD_COMMAND)
         52 - parquet-reader-test (BAD_COMMAND)
         53 - parquet-schema-test (BAD_COMMAND)
         54 - parquet-arrow-schema-test (BAD_COMMAND)
         55 - parquet-arrow-reader-writer-test (BAD_COMMAND)
         56 - parquet-comparison-test (BAD_COMMAND)
         57 - parquet-memory-test (BAD_COMMAND)
Errors while running CTest
builder for '/nix/store/nwnpikv0sn9768h6bayl48zdjfd8rdkc-arrow-cpp-0.11.0.drv' failed with exit code 8
error: build of '/nix/store/nwnpikv0sn9768h6bayl48zdjfd8rdkc-arrow-cpp-0.11.0.drv' failed

GrahamcOfBorg · 2018-10-09T20:54:46Z

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

57: /private/tmp/nix-build-arrow-cpp-0.11.0.drv-0/apache-arrow-0.11.0/cpp/build/src/parquet/util
56/56 Test #57: parquet-memory-test ................   Passed    0.15 sec

100% tests passed, 0 tests failed out of 56

Label Time Summary:
parquet     =  16.75 sec*proc (18 tests)
unittest    = 119.49 sec*proc (56 tests)

Total Test time (real) = 119.57 sec

GrahamcOfBorg · 2018-10-10T17:52:11Z

Success on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

57: /build/apache-arrow-0.11.0/cpp/build/src/parquet/util
56/56 Test #57: parquet-memory-test ................   Passed    0.08 sec

100% tests passed, 0 tests failed out of 56

Label Time Summary:
parquet     =  13.99 sec*proc (18 tests)
unittest    = 111.41 sec*proc (56 tests)

Total Test time (real) = 111.85 sec

GrahamcOfBorg · 2018-10-10T17:55:57Z

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

57: /private/tmp/nix-build-arrow-cpp-0.11.0.drv-0/apache-arrow-0.11.0/cpp/build/src/parquet/util
56/56 Test #57: parquet-memory-test ................   Passed    0.13 sec

100% tests passed, 0 tests failed out of 56

Label Time Summary:
parquet     =  15.49 sec*proc (18 tests)
unittest    = 121.95 sec*proc (56 tests)

Total Test time (real) = 121.99 sec

GrahamcOfBorg · 2018-10-10T17:56:54Z

Success on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

57: /build/apache-arrow-0.11.0/cpp/build/src/parquet/util
56/56 Test #57: parquet-memory-test ................   Passed    0.06 sec

100% tests passed, 0 tests failed out of 56

Label Time Summary:
parquet     =  15.67 sec*proc (18 tests)
unittest    = 111.05 sec*proc (56 tests)

Total Test time (real) = 111.08 sec

c0bw3b · 2018-10-12T17:43:07Z

Well this is ready for merging IMHO.
Let's keep the alias for potential out-of-tree usage. At the very least it will be a pointer to the new arrow-cpp lib for someone using parquet-cpp out of the nixpkgs tree and encountering build errors.

arrow-cpp: 0.10.0 -> 0.11.0

956e3bf

veprbl requested a review from FRidh as a code owner October 8, 2018 18:36

GrahamcOfBorg added the 6.topic: python label Oct 8, 2018

GrahamcOfBorg added 8.has: clean-up 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 8, 2018

veprbl commented Oct 8, 2018

View reviewed changes

arrow-cpp: enable testing

7b7ee57

veprbl force-pushed the pr/arrow_0.11.0 branch from 6d36f66 to 7b7ee57 Compare October 10, 2018 17:29

c0bw3b mentioned this pull request Oct 12, 2018

flatbuffers: 1.8.0 -> 1.9.0 #38731

Closed

FRidh merged commit 4ed9933 into NixOS:master Oct 13, 2018

c0bw3b mentioned this pull request Oct 16, 2018

flatbuffers: 1.8.0 -> 1.10.0 #48521

Merged

veprbl deleted the pr/arrow_0.11.0 branch December 1, 2020 17:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

arrow-cpp: 0.10.0 -> 0.11.0 #48066

arrow-cpp: 0.10.0 -> 0.11.0 #48066

veprbl commented Oct 8, 2018

veprbl commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

veprbl Oct 8, 2018 •

edited

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 10, 2018

GrahamcOfBorg commented Oct 10, 2018

GrahamcOfBorg commented Oct 10, 2018

c0bw3b commented Oct 12, 2018

arrow-cpp: 0.10.0 -> 0.11.0 #48066

arrow-cpp: 0.10.0 -> 0.11.0 #48066

Conversation

veprbl commented Oct 8, 2018

Motivation for this change

Things done

veprbl commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

GrahamcOfBorg commented Oct 8, 2018

veprbl Oct 8, 2018 • edited

Choose a reason for hiding this comment

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 9, 2018

GrahamcOfBorg commented Oct 10, 2018

GrahamcOfBorg commented Oct 10, 2018

GrahamcOfBorg commented Oct 10, 2018

c0bw3b commented Oct 12, 2018

veprbl Oct 8, 2018 •

edited