Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bazel: fix darwin build #41911

Merged
merged 1 commit into from Jun 17, 2018
Merged

bazel: fix darwin build #41911

merged 1 commit into from Jun 17, 2018

Conversation

uri-canva
Copy link
Contributor

@uri-canva uri-canva commented Jun 13, 2018 •
edited

Motivation for this change

#40990 was merged only partially and doesn't work on darwin without this.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@GrahamcOfBorg GrahamcOfBorg added the 6.topic: darwin Running or building packages on Darwin label Jun 13, 2018
@uri-canva uri-canva mentioned this pull request Jun 13, 2018
Closed
8 tasks
@uri-canva
Copy link
Contributor Author

cc @matthewbauer

This fixes this build failures on darwin when building the bazel derivation:

When running ./compile.sh in the build phase without the changes to scripts/bootstrap/compile.sh:

ERROR: /private/tmp/nix-build-bazel-0.12.0.drv-0/third_party/protobuf/3.4.0/BUILD:399:1: Linking of rule '//third_party/protobuf/3.4.0:js_embed' failed (Exit 1): cc_wrapper.sh failed: error executing command
  (cd /tmp/bazel_OOUl0vJV/out/execroot/io_bazel && \
  exec env - \
    APPLE_SDK_PLATFORM='' \
    APPLE_SDK_VERSION_OVERRIDE='' \
    PATH=/nix/store/sj35fiqw5s0ia0q93fmkpqczjmq8fpng-zip-3.0/bin:/nix/store/yac2j5l15gh3ph0wghpdf7gb7fir4s41-python-2.7.15/bin:/nix/store/1bazvm78y23j6fas5av8why7p2z3xmmq-unzip-6.0/bin:/nix/store/10wijab7qhsn76nyq6ixx743k6zkfxkp-which-2.21/bin:/nix/store/gabngkfw4gnpmjfbrsw5s9rc4r28hn9x-bash/bin:/nix/store/1raxjnnz5fzjl216r6w9bffprqqxw3jr-clang-wrapper-5.0.2/bin:/nix/store/cjqwnzxl7wzgrf51ai6sr3mv82jk1gz9-clang-5.0.2/bin:/nix/store/yrlzm8nkyzs5bwy7dhb1vhfzx936p6zn-coreutils-8.29/bin:/nix/store/6zknvic2wi25qg7ha39mm8wb3x0ppwn9-cctools-binutils-darwin-wrapper/bin:/nix/store/yyak5sjv68n9vdgnkd2cb5djk1l9sqj5-cctools-binutils-darwin/bin:/nix/store/yrlzm8nkyzs5bwy7dhb1vhfzx936p6zn-coreutils-8.29/bin:/nix/store/fgx4nvl9v1b7mvhvyszvjh1mhw94hj9r-zulu1.8.0_121-8.20.0.5/bin:/nix/store/yrlzm8nkyzs5bwy7dhb1vhfzx936p6zn-coreutils-8.29/bin:/nix/store/4yrg510n4w2ccgakfcsj0b076yjdxkvr-findutils-4.6.0/bin:/nix/store/99g59wd4hbdzd31ls6xdgcnvf7c3zknx-diffutils-3.6/bin:/nix/store/18aw3q65npjazk30hw9zymkxpnkvlraq-gnused-4.5/bin:/nix/store/kzqw5l2ad0c3p0c2isfh4p1rlffymlqc-gnugrep-3.1/bin:/nix/store/7slp7i2xikqn3igkii9vwbrgj2zacmpd-gawk-4.2.1/bin:/nix/store/3wjxhr0q0a68zvb0kvf8ars7valvpcq3-gnutar-1.30/bin:/nix/store/1fsfsv9pldj95s4xyl4l4mq0h3n31xs8-gzip-1.9/bin:/nix/store/p8sn7shhdva92hdfbz6zs6slyjd7rnib-bzip2-1.0.6.0.1-bin/bin:/nix/store/kj54d2pnnaq9dmx88icx0cb136nbgqnb-gnumake-4.2.1/bin:/nix/store/9gghkkqhn6v7dvpf2319qvrxkkw03vwr-bash-4.4-p19/bin:/nix/store/wwpsf6lp73i2fpzv6cdics69zas7rx05-patch-2.7.6/bin:/nix/store/dq1p53lc0s4kjkvyrwc5prj13wyxy4vi-xz-5.2.4-bin/bin:/nix/store/yrlzm8nkyzs5bwy7dhb1vhfzx936p6zn-coreutils-8.29/bin \
    XCODE_VERSION_OVERRIDE=9.4.0 \
  external/local_config_cc/cc_wrapper.sh -fobjc-link-runtime -Wl,-S -o bazel-out/host/bin/third_party/protobuf/3.4.0/js_embed bazel-out/host/bin/third_party/protobuf/3.4.0/_objs/js_embed/third_party/protobuf/3.4.0/src/google/protobuf/compiler/js/embed.o -headerpad_max_install_names -lc++ -no-canonical-prefixes)

When running ./output/bazel --output_user_root=/tmp/.bazel build //scripts:bash_completion ... in the build phase without the changes to .bazelrc:

ERROR: /private/tmp/.bazel/6d71e96597567e0ddffbed07de5f5263/external/io_bazel/third_party/protobuf/3.4.0/BUILD:399:1: Linking of rule '@io_bazel//third_party/protobuf/3.4.0:js_embed' failed (Exit 1)
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Target //scripts:bash_completion failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 165.495s, Critical Path: 17.21s
FAILED: Build did NOT complete successfully
builder for '/nix/store/jq8b4lmj3hli29h44yxww9iggr1h6n8k-bazel-0.12.0.drv' failed with exit code 1

Both are needed because ./compile.sh bootstraps bazel and ignores .bazelrc, but we still use the built bazel to build the bash completions and run the tests in the check phase, and that one respects .bazelrc. Both will strip the environment, so the nix cc and ld wrappers will not be able to find the flags themselves.

The reason why it's not needed when building on linux is because:

  1. The build on linux doesn't need any library beyond the standard library.
  2. default_cxx_stdlib_compile is set (see

    nixpkgs/pkgs/build-support/cc-wrapper/default.nix

    Line 45 in 51af2c3

    default_cxx_stdlib_compile = optionalString (targetPlatform.isLinux && !(cc.isGNU or false) && !nativeTools)
    ) for linux, but not for darwin, opened stdenv.cc: use libc++ as default c++ standard library on darwin #41589 for that but it seems that's already not considered a good thing for linux to begin with, so we don't want to extend it to darwin.
  3. ld-wrapper adds -L flags automatically (see

    nixpkgs/pkgs/build-support/bintools-wrapper/setup-hook.sh

    Line 23 in 51af2c3

    export NIX_${role_pre}LDFLAGS+=" -L$1/lib"
    ), but doesn't add -F flags, even though cc-wrapper does (see

    nixpkgs/pkgs/build-support/cc-wrapper/setup-hook.sh

    Line 78 in 51af2c3

    export NIX_${role_pre}CFLAGS_COMPILE+=" -F$1/Library/Frameworks"
    ).

So the ideal scenario is that clang would pick up libc++ by default on darwin, like gcc picks up libstdc++ on linux, or have both require listing libc++ or libstdc++ explicitly, and -F flags would be picked up through the hooks.

I've sent out the PR for the former, and will send out a PR for the latter, but since they affect the way nixpkgs builds derivations I suspect it will take a long time to figure out all the ramifications of the changes and what the best way of doing them is, so I want to get this working for this use case in particular.

@uri-canva
Copy link
Contributor Author

To reiterate, the bazel derivation doesn't build at all right now on darwin.

@uri-canva
Copy link
Contributor Author

@LnL7 @matthewbauer ping

@matthewbauer matthewbauer merged commit 9370818 into NixOS:master Jun 17, 2018
@uri-canva uri-canva deleted the bazelrc-stuff branch June 17, 2018 05:17
@xeji xeji mentioned this pull request Jun 24, 2018
Merged
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@uri-canva @matthewbauer @GrahamcOfBorg