Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Revert "bazel: drop absolute Python path" #65137

Closed

Conversation

kalbasit
Copy link
Member

This reverts commit bd98f35.

Motivation for this change

I'm currently using rules_docker, and #64903 has broken it. It cannot find the Python binary anymore. I'm going to attempt to add a failing test for the regression on Bazel, until then please consider this a draft.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@abbradar
Copy link
Member

Sad news. We'll also need Python 2 test, and just reverting this commit would break it, so we need to find another way.

Do you use Nix hacks (i.e. do you build with buildBazelPackage)? If not, do you have Python in PATH?

@kalbasit
Copy link
Member Author

We do not use buildBazelPackage, instead, we load Bazel within a nix-shell development environment.

Yes I do make python2 available in the PATH through the tools/bazel file, and build --host_force_python=PY2 in .bazelrc.

#!/nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin/bash

set -euo pipefail

# Tell Bazel which C compiler to use, and put it in the path
export AR=$'/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/ar'
export CC=$'/nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/gcc'
export CXX=$'/nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/g++'
export LD=$'/nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/ld'

# XXX: hack for macosX, this flags disable bazel usage of xcode
# See: https://github.com/bazelbuild/bazel/issues/4231
export BAZEL_USE_CPP_ONLY_TOOLCHAIN=1

# Hack around a known issue with Nix and Python:
# See: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/python.section.md#python-setuppy-bdist_wheel-cannot-create-whl
export SOURCE_DATE_EPOCH=315532800

# PLATF-431: We require python3 for our development, but Bazel then fails to
# download docker images or even invoke the puller as it fails to run with
# Python 3. The purpose of this wrapper is to force Bazel to see Python 2 in
# the PATH before any other Python that might exist in the PATH.
export PATH='/nix/store/md0smqcx2ll4hr234jr1abjsifgcc5ac-python-2.7.16/bin'${PATH:+':'}$PATH

# Bazel cannot find the javac binary on Darwin if JAVA_HOME is not set
export JAVA_HOME="/nix/store/lhfii4yzkq9qar1ki11bh2avhh9zjcm3-openjdk-8u212-ga/lib/openjdk"

# pass the execution to bazel-real now
exec "$BAZEL_REAL" "$@"

@abbradar
Copy link
Member

abbradar commented Jul 20, 2019

It'd be very helpful to have a regression test to debug this because I'm not sure how exactly to reproduce; do you think you'll be able to implement it?

EDIT: even a script to reproduce would be helpful, not necessarily a Nix expression.

@kalbasit
Copy link
Member Author

Enabling the nix hacks makes it possible to use Bazel without reverting #64903. I'm wondering if we should enable the hacks by default.

(bazel.override { enableNixHacks = true; })

@abbradar
Copy link
Member

This would break promise that Bazel gives about reproducibility. When in buildBazelPackage Nix ensures it for us but otherwise we'd be better not touching it...

@kalbasit
Copy link
Member Author

This would break promise that Bazel gives about reproducibility. When in buildBazelPackage Nix ensures it for us but otherwise we'd be better not touching it...

I see. I'll get you a sample project sometime next week.

@kalbasit kalbasit force-pushed the nixpkgs_bazel-fix-work-environment branch from a10743d to 4f0089f Compare July 25, 2019 16:03
@kalbasit
Copy link
Member Author

Now that rules_docker has moved on from Python2, I'm going to close this PR. I'll re-open it as an issue if it's still a problem.

@kalbasit kalbasit closed this Oct 25, 2019
@kalbasit kalbasit deleted the nixpkgs_bazel-fix-work-environment branch October 25, 2019 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants