Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildRustCrate: Replace hyphen with underscore in env variables #88054

Merged
merged 2 commits into from May 26, 2020
Merged

buildRustCrate: Replace hyphen with underscore in env variables #88054

merged 2 commits into from May 26, 2020

Conversation

notriddle
Copy link
Contributor

This fixes a bug that prevents encoding_c from building.

Motivation for this change

I tried to use crate2nix to build a Rust package that depends on mozjs. This depends on encoding_c, which passes this output from build.rs:

cargo:include-dir=$PWD/includes

Which gets rewritten to this env file:

export DEPS_encoding_c_INCLUDE-DIR=$PWD/includes

This is invalid; nix doesn't allow environment variables to contain hyphens. In any case, it also doesn't match up with how mozjs is trying to reference it here. The mozjs build script expects it to be spelled DEPS_ENCODING_C_INCLUDE_DIR, with all of it in SCREAMING_CASE.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@notriddle notriddle requested a review from andir as a code owner May 18, 2020 09:13
@andir
Copy link
Member

andir commented May 20, 2020

@ofborg build buildRustCrateTests

@andir andir self-assigned this May 20, 2020
andir
andir reviewed May 20, 2020
View reviewed changes
Copy link
Member

@andir andir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! The change looks good. Do you mind adding a test for this edge case? It will prevent us from regressing on this issue with the next refactoring.

@Mic92 Mic92 changed the title Replace hyphen with underscore in env variables buildRustCrate: Replace hyphen with underscore in env variables May 20, 2020
@notriddle
Copy link
Contributor Author

notriddle commented May 24, 2020 •
edited

I couldn't find any good docs on this, so just as a note, I actually ran that test suite by executing nix-build -A buildRustCrateTests.test from the root of the nixpkgs repository. Is there a better way?

Running it against master, with the test case cherry-picked 
$ nix-build -A buildRustCrateTests.test
these derivations will be built:
  /nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv
  /nix/store/dnjnc55hzx1a9qqnnxx4j8h6kqsw16qc-run-buildRustCrate-foo-test.drv
  /nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv
building '/nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/ccsm2m5ihgngmdprix638dnvyk7njmqb-build-script-and-include-dir-foo
source root is build-script-and-include-dir-foo
patching sources
configuring
Running cd .
/nix/store/969yral7dpsanjracak90xy0nbny7jpb-rust_bar-0.1.0-lib/env: line 1: export: `DEP_bar_INCLUDE-DIR=/build/build-script-and-include-dir-bar/src': not a valid identifier
builder for '/nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv' failed with exit code 1
cannot build derivation '/nix/store/dnjnc55hzx1a9qqnnxx4j8h6kqsw16qc-run-buildRustCrate-foo-test.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv': 1 dependencies couldn't be built
error: build of '/nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv' failed
Running against my patch branch 
$ nix-build -A buildRustCrateTests.test
these derivations will be built:
  /nix/store/fkc5r6ff6h610swk8psrvab21ycvqk8d-src.drv
  /nix/store/vigvvmh0iqfmrbv7dz3b4xp6d7sjjg5g-build-script-and-include-dir-bar.drv
  /nix/store/587pnaryla3d990drm0r4wwsxjv1drnd-rust_bar-0.1.0.drv
  /nix/store/m6657w9hp7xq7k8288y45ry62knkqn1p-rust_foo-0.1.0.drv
  /nix/store/hkc6h7rl07zz2sagvlk1kinhp5x2k4j7-run-buildRustCrate-foo-test.drv
  /nix/store/f8x6khaaqgqf75kjzb5r3qzdcjwnf5b6-buildRustCrate-tests.drv
building '/nix/store/fkc5r6ff6h610swk8psrvab21ycvqk8d-src.drv'...
building '/nix/store/vigvvmh0iqfmrbv7dz3b4xp6d7sjjg5g-build-script-and-include-dir-bar.drv'...
building '/nix/store/587pnaryla3d990drm0r4wwsxjv1drnd-rust_bar-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/iqq9iwprbg5dx6fxcw1j405v792vgzr5-build-script-and-include-dir-bar
source root is build-script-and-include-dir-bar
patching sources
configuring
Running cd .
Building build.rs (bar)
Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --out-dir target/build/bar --emit=dep-info,link -L dependency=target/buildDeps --cap-lints allow --color always
building
Building src/lib.rs (bar)
Running rustc --crate-name bar src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ -C metadata=558120c4eb -C extra-filename=-558120c4eb --crate-type lib -L /build/build-script-and-include-dir-bar/target/build/bar.out --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
patching script interpreter paths in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0
checking for references to /build/ in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0...
shrinking RPATHs of ELF executables and libraries in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib
shrinking /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/bar/build_script_build
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib 
patching script interpreter paths in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib
checking for references to /build/ in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib...
building '/nix/store/m6657w9hp7xq7k8288y45ry62knkqn1p-rust_foo-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/ccsm2m5ihgngmdprix638dnvyk7njmqb-build-script-and-include-dir-foo
source root is build-script-and-include-dir-foo
patching sources
configuring
Running cd .
Building build.rs (foo)
Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --out-dir target/build/foo --emit=dep-info,link -L dependency=target/buildDeps --extern bar=/nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/libbar-558120c4eb.rlib --cap-lints allow --color always
building
Building foo (src/main.rs)
Running rustc --crate-name foo src/main.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern bar=/nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/libbar-558120c4eb.rlib --out-dir target/bin -L dependency=target/deps --cap-lints allow -L /build/build-script-and-include-dir-foo/target/build/foo.out --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0
shrinking /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0/bin/foo
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0/bin 
patching script interpreter paths in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0
checking for references to /build/ in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0...
shrinking RPATHs of ELF executables and libraries in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib
shrinking /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib/lib/foo/build_script_build
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib/lib 
patching script interpreter paths in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib
checking for references to /build/ in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib...
building '/nix/store/hkc6h7rl07zz2sagvlk1kinhp5x2k4j7-run-buildRustCrate-foo-test.drv'...
building '/nix/store/f8x6khaaqgqf75kjzb5r3qzdcjwnf5b6-buildRustCrate-tests.drv'...
/nix/store/1rqydxhx6lmscia5vslmi6h299c778ax-buildRustCrate-tests

In any case, @andir, I've added a regression test.

@ofborg ofborg bot requested a review from andir May 24, 2020 18:43
@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

@ofborg build buildRustCrateTests

@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

Looks like all the tests are passing, and there is now a regression test… let's merge :)

@Ekleog Ekleog merged commit c21cbf2 into NixOS:master May 26, 2020
@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

@andir Do you think this should be backported? I'm a bit cautious about it, seeing as it's infrastructure, but on the other hand as it's a quite significant bug…?

@andir
Copy link
Member

andir commented May 26, 2020 via email

Ekleog pushed a commit that referenced this pull request May 30, 2020
@notriddle @Ekleog
buildRustCrate: Replace hyphen with underscore in env variables (#88054)
5a46292 
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.

(cherry picked from commit c21cbf2)
@Ekleog
Copy link
Member

Ekleog commented May 30, 2020

Done :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andir andir Awaiting requested review from andir

Assignees

@andir andir

Labels
6.topic: rust 10.rebuild-darwin: 0 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@notriddle @andir @Ekleog