Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

arrow-cpp: disable zstd #48833

Closed
wants to merge 1 commit into from
Closed

arrow-cpp: disable zstd #48833

wants to merge 1 commit into from

Conversation

veprbl
Copy link
Member

@veprbl veprbl commented Oct 22, 2018

"compression-test" fails when built against zstd 1.3.6

Fixes: c8f7188 ('zstd: 1.3.5 -> 1.3.6 (#47965)')
Fixes: #47965

Motivation for this change

arrow-cpp was broken by zstd upgrade from 1.3.5 to 1.3.6. I don't have time to fix this myself. I also have impression that upstream would not care about this.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@veprbl
arrow-cpp: disable zstd
63d123f 
"compression-test" fails when built against zstd 1.3.6

Fixes: c8f7188 ('zstd: 1.3.5 -> 1.3.6 (NixOS#47965)')
Fixes: NixOS#47965
@Mic92
Copy link
Member

Mic92 commented Oct 22, 2018

@GrahamcOfBorg build arrow-cpp

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

  See also

    /build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-*.log


make[2]: *** [CMakeFiles/zstd_ep.dir/build.make:90: zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: CMakeFiles/zstd_ep.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
builder for '/nix/store/95f4466vbpxr4gv0p0pc8kkzxkj6qd2m-arrow-cpp-0.11.0.drv' failed with exit code 2
error: build of '/nix/store/95f4466vbpxr4gv0p0pc8kkzxkj6qd2m-arrow-cpp-0.11.0.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

  See also

    /build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-*.log


make[2]: *** [CMakeFiles/zstd_ep.dir/build.make:90: zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: CMakeFiles/zstd_ep.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
builder for '/nix/store/9gaafv5k8f4ji2f0s91sa2sz5hxhv9v9-arrow-cpp-0.11.0.drv' failed with exit code 2
error: build of '/nix/store/9gaafv5k8f4ji2f0s91sa2sz5hxhv9v9-arrow-cpp-0.11.0.drv' failed

@GrahamcOfBorg
Copy link

Unexpected error: command failed with exit code 1 on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

copying path '/nix/store/1ng4k8k9wf742798k3rlwg9wcaiwbzbg-thrift-0.11.0.tar.gz' from 'https://cache.nixos.org'...
/nix/store/29n15888jcw0siy31krv5cqm7s477c7g-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
/nix/store/29n15888jcw0siy31krv5cqm7s477c7g-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
copying path '/nix/store/pqhhjzi5hj2a7g43vaz8i73l6x5dnkqv-snappy-1.1.7-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/izffg5d0g0fmfj786f47fbjyrhac1s23-zope.event-4.0.3.tar.gz' from 'https://cache.nixos.org'...
copying path '/nix/store/6cil9vpj5d08dndz7yhl8gfw7gpy94ah-zope.interface-4.5.0.tar.gz' from 'https://cache.nixos.org'...
building '/nix/store/pv31mddhhcjvvxn7wq8a8sv7mzj4s2wf-double-conversion-3.1.1.drv'...
/nix/store/8qai18dv0nqyw6s61g9p91n8684rnlg8-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
error (ignored): unable to fork: Resource temporarily unavailable
error: unable to fork: Resource temporarily unavailable

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

  See also

    /build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-*.log


make[2]: *** [CMakeFiles/zstd_ep.dir/build.make:90: zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: CMakeFiles/zstd_ep.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
builder for '/nix/store/95f4466vbpxr4gv0p0pc8kkzxkj6qd2m-arrow-cpp-0.11.0.drv' failed with exit code 2
error: build of '/nix/store/95f4466vbpxr4gv0p0pc8kkzxkj6qd2m-arrow-cpp-0.11.0.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: arrow-cpp

Partial log (click to expand)

  See also

    /build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-*.log


make[2]: *** [CMakeFiles/zstd_ep.dir/build.make:90: zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: CMakeFiles/zstd_ep.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
builder for '/nix/store/9gaafv5k8f4ji2f0s91sa2sz5hxhv9v9-arrow-cpp-0.11.0.drv' failed with exit code 2
error: build of '/nix/store/9gaafv5k8f4ji2f0s91sa2sz5hxhv9v9-arrow-cpp-0.11.0.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: arrow-cpp

Partial log (click to expand)

57: /private/tmp/nix-build-arrow-cpp-0.11.0.drv-0/apache-arrow-0.11.0/cpp/build/src/parquet/util
56/56 Test #57: parquet-memory-test ................   Passed    0.14 sec

100% tests passed, 0 tests failed out of 56

Label Time Summary:
parquet     =  14.83 sec*proc (18 tests)
unittest    = 119.58 sec*proc (56 tests)

Total Test time (real) = 120.25 sec

@Mic92
Copy link
Member

Mic92 commented Oct 22, 2018

I seems one has to patch/configure arrow-cpp to not download zstd itself:

[  1%] Performing download step (download, verify and extract) for 'zstd_ep'
CMake Error at zstd_ep-stamp/zstd_ep-download-RELEASE.cmake:16 (message):
  Command failed: 1
   '/nix/store/sxxy01wn82n7wjl5a384344m2i4xszx6-cmake-3.12.1/bin/cmake' '-Dmake=' '-Dconfig=' '-P' '/build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-RELEASE-impl.cmake'
  See also
    /build/apache-arrow-0.11.0/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-download-*.log

@veprbl
Copy link
Member Author

veprbl commented Oct 23, 2018

@Mic92 I'm sorry for the noise and wasting your time. It doesn't seem like I can really disable zstd and keep the tests for arrow-cpp. I've opened the issue with upstream at facebook/zstd#1385 .

@veprbl veprbl closed this Oct 23, 2018
@Mic92
Copy link
Member

Mic92 commented Oct 23, 2018

No worries.

@veprbl veprbl deleted the pr/arrow-cpp_zstd_fix branch December 1, 2020 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@veprbl @Mic92 @GrahamcOfBorg