Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildRustCrate: only link build deps into build script #82404

Merged
merged 2 commits into from Mar 20, 2020
Merged

buildRustCrate: only link build deps into build script #82404

merged 2 commits into from Mar 20, 2020

Conversation

danieldk
Copy link
Contributor

Motivation for this change

According to the Cargo documentation:

The build script does not have access to the dependencies listed in
the dependencies or dev-dependencies section (they’re not built
yet!). Also, build dependencies are not available to the package
itself unless also explicitly added in the [dependencies] table.

https://doc.rust-lang.org/cargo/reference/build-scripts.html

This change separates linkage of regular dependencies and build
dependencies.

Tested with buildRustCrateTests and some personal crates.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@danieldk danieldk requested a review from andir as a code owner March 12, 2020 12:30
@danieldk
Copy link
Contributor Author

@GrahamcOfBorg build buildRustCrateTests

andir
andir requested changes Mar 12, 2020
View reviewed changes
Copy link
Member

@andir andir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good and makes sense (if the cargo docs are correct).

Could you add a test the buildRustCrates that tests this?

Example setup:

Have a dependency A that is used in both the build script and in the actual build (e.g. a simple binary). Then have a version A1 and A2 that differ in some constant they export. Assert for the constant of A1 in build build.rs file (during build time) and assert for the constant of A2 in the main.rs (during run-time).

I also have some recollection of an issue where those dependencies were not exactly separated as described in the docs. In any case we should follow the upstream docs and fix downstream users if those fail after this change - if this is in fact correct behavior.

I will try to run this through some tests on a personal collection of crates to see where my recollection of issues comes from. Other then that (and the missing test ;)) this looks really good. Thank you! 👍

@danieldk
Copy link
Contributor Author

Thanks for the feedback! I have added a test. Did you means something like this? Or really a single dependent create with feature gating for the constant.

@ofborg ofborg bot requested a review from andir March 12, 2020 14:48
@andir
Copy link
Member

andir commented Mar 12, 2020

@GrahamcOfBorg build buildRustCrateTests

andir
andir reviewed Mar 12, 2020
View reviewed changes
Copy link
Member

@andir andir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works on some of my personal projects.

Do you by chance have motivation to add a few more comments in the code paths you touched? It wasn't great so far but I am trying to gradually increase the coverage of comments. At least some notes about what is happening there. The term link is used very often and to people new to the code base this is probably a bit confusing.

@danieldk
Copy link
Contributor Author

Looks good and works on some of my personal projects.

Do you by chance have motivation to add a few more comments in the code paths you touched? It wasn't great so far but I am trying to gradually increase the coverage of comments. At least some notes about what is happening there. The term link is used very often and to people new to the code base this is probably a bit confusing.

No problem, I'll try to add some comments tonight!

@danieldk
Copy link
Contributor Author

I have added some comments to the affected parts. I hope they are clear and helpful.

@ofborg ofborg bot requested a review from andir March 13, 2020 07:50
andir
andir reviewed Mar 13, 2020
View reviewed changes
pkgs/build-support/rust/build-rust-crate/configure-crate.nix Outdated
if [[ -e target/link.build ]]; then
sort -u target/link.build > target/link.build.sorted
mv target/link.build.sorted target/link.build
tr '\n' ' ' < target/link.build > target/link.build_
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I don't like about this is that we are using the underscored (e.g. build_) file name more often now. IMO that should only be used temporarily because of the invocation of sort.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, it's pretty pointless here, even more because link.build is only use in this phase. I have removed tr completely here and replaced

EXTRA_BUILD_FLAGS="$EXTRA_BUILD_FLAGS $(cat target/link.build)"

by

EXTRA_BUILD_FLAGS="$EXTRA_BUILD_FLAGS $(tr '\n' ' ' < target/link.build)"

below.

Copy link
Contributor Author

@danieldk danieldk Mar 13, 2020 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll tack on another commit, because the temporary files are not necessary at all. It seems that GNU sort support in-place sorts. From the info page:

Write output to output-file instead of standard output. Normally, sort reads all input before opening output-file, so you can sort a file in place by using commands like sort -o F F and cat F | sort -o F.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added another commit which sort the linking options in-place. I'll remove it if you don't like it ;).

@andir
Copy link
Member

andir commented Mar 13, 2020

Thanks! Comments look fine. I had two more minor remarks.

@danieldk
buildRustCrate: only link build deps into build script
ea6e048 
According to the Cargo documentation:

> The build script does not have access to the dependencies listed in
> the dependencies or dev-dependencies section (they’re not built
> yet!). Also, build dependencies are not available to the package
> itself unless also explicitly added in the [dependencies] table.

https://doc.rust-lang.org/cargo/reference/build-scripts.html

This change separates linkage of regular dependencies and build
dependencies.
@danieldk
Copy link
Contributor Author

@GrahamcOfBorg build buildRustCrateTests

@ofborg ofborg bot requested a review from andir March 13, 2020 10:37
@andir andir merged commit 4e8d099 into NixOS:master Mar 20, 2020
@danieldk danieldk deleted the build-script-link-fix branch March 20, 2020 18:39
@andir andir mentioned this pull request Mar 28, 2020
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andir andir Awaiting requested review from andir

Assignees
No one assigned
Labels
6.topic: rust 10.rebuild-darwin: 0 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@danieldk @andir