Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

glibc: fix darwin -> linux cross compilation #103517

Closed
wants to merge 2 commits into from
Closed

glibc: fix darwin -> linux cross compilation #103517

wants to merge 2 commits into from

Conversation

Gaelan
Copy link
Contributor

@Gaelan Gaelan commented Nov 12, 2020

Motivation for this change

darwin -> linux cross-compilation of glibc was broken for two reasons. This fixes both of them:

  • A recent glibc update removed some code we were patching. I fixed this by removing the relevant part of the patch.
  • When NIX_COREFOUNDATION_RPATH is set, we create a ld.so binary with an RPATH of [], but ld.so expects not to have an RPATH set at all. I fix this by unsetting the variable in the glibc derivation, as suggested by @parthy, but I'm not sure this is the best solution (in particular, I'm not convinced we should be setting NIX_COREFOUNDATION_RPATH at all when host != darwin). However, this does get cross-compilation working for now.

Tested by doing nix-build . --arg crossSystem '(import ./. {}).lib.systems.examples.gnu64' -A hello on Darwin, copying the resulting closure onto a Linux box, and running hello.

This should only change code on the cross-compilation path, so it shouldn't cause any mass rebuilds.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@Gaelan
glibc: fix darwin-cross-build.patch with new glibc
15e7f82 
A recent-ish glibc update removed some sunrpc code, causing a patch
against it to fail to apply. Since it's removed, fixing it is now
unnecessary, so we just remove that part of the patch.
@Gaelan
glibc: unset NIX_COREFOUNDATION_RPATH
8f9ac60 
When cross-compiling glibc from darwin to Linux, if
NIX_COREFOUNDATION_RPATH is set, the resulting ld.so ends up with an
RPATH of [], instead of no RPATH at all; this causes ld.so to fail with
an assertion.

I'm not sure if this is the best place to fix this, but it gets
cross-compilation working for now.
@Gaelan Gaelan changed the title <!-- To help with the large amounts of pull requests, we would appreciate your reviews of other pull requests, especially simple package updates. Just leave a comment describing what you have tested in the relevant package/service. Reviewing helps to reduce the average time-to-merge for everyone. Thanks a lot if you do! List of open PRs: https://github.com/NixOS/nixpkgs/pulls Reviewing guidelines: https://hydra.nixos.org/job/nixpkgs/trunk/manual/latest/download/1/nixpkgs/manual.html#chap-reviewing-contributions --> glib: fix darwin -> linux cross compilation Nov 12, 2020
@Gaelan Gaelan changed the title glib: fix darwin -> linux cross compilation glibc: fix darwin -> linux cross compilation Nov 12, 2020
@veprbl veprbl linked an issue Nov 12, 2020 that may be closed by this pull request
glibc can't be built on darwin #18048
Closed
@Gaelan Gaelan mentioned this pull request Nov 12, 2020
Closed
@vcunat vcunat added 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on 6.topic: darwin Running or building packages on Darwin labels Nov 12, 2020
@veprbl veprbl linked an issue Nov 12, 2020 that may be closed by this pull request
glibc fails to build on macOS #103356
Closed
LnL7
LnL7 reviewed Nov 22, 2020
View reviewed changes
pkgs/development/libraries/glibc/common.nix
@@ -206,6 +206,8 @@ stdenv.mkDerivation ({


'' + lib.optionalString (stdenv.hostPlatform != stdenv.buildPlatform) ''
unset NIX_COREFOUNDATION_RPATH
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to cause problems in multiple places, we should not set this at all if the target platform is linux. This might also be wrong for darwin->darwin cross.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's fair. It looks like it's getting set here:

nixpkgs/pkgs/stdenv/darwin/default.nix

Lines 464 to 467 in dbedb52

preHook = commonPreHook + ''
export NIX_COREFOUNDATION_RPATH=${pkgs.darwin.CF}/Library/Frameworks
export PATH_LOCALE=${pkgs.darwin.locale}/share/locale
'';

But that gets deep enough into the weeds of stdenv and cross infrastructure that I'm not sure how best to fix it. If possible, it would probably be better to do the target check in nix (as opposed to in bash) so we don't cause rebuilds for non-cross packages.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way that's currently added isn't great and was also done before most of the cross work. Most things, including the counterpart that overrides this when frameworks are included, use addEnvHooks which I think would improve the situation.

Ideally this would also just be another setup-hook, just for the pure CF. The reason I added it here was to ensure that the hooks from frameworks run last. There's probably a better way to do that.

For testing it might be possible to use an if (stdenv.hostPlatform != stdenv.buildPlatform) then addEnvHook else export in the preHook but that might be too late. Either way the proper thing should probably go to staging.

@siraben
Copy link
Member

siraben commented Nov 23, 2020 •
edited

I was able to cross-compile from darwin to gnu64 using this PR.

$ helloCross="$(nix-build . --arg crossSystem '(import ./. {}).lib.systems.examples.gnu64' -A hello)"
$ nix-copy-closure --to nix-linuxkit "$helloCross"
$ ssh nix-linuxkit "$helloCross/bin/hello"
Hello, world!

@Gaelan Gaelan mentioned this pull request Dec 27, 2020
Merged
10 tasks
@veprbl veprbl mentioned this pull request Jan 31, 2021
Closed
10 tasks
@stale
Copy link

stale bot commented Jul 21, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 21, 2021
@siraben
Copy link
Member

siraben commented Jul 21, 2021

Still important.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 21, 2021
@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Jul 21, 2021 •
edited

Still important.

and still stale. Maybe just open a new PR if you want it to be shipped.

@stale
Copy link

stale bot commented Apr 19, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 19, 2022
@SuperSandro2000
Copy link
Member

Closing because author did not react in almost 2 years.

@tartavull
Copy link

tartavull commented Oct 29, 2022 •
edited

This seems important to better support Darwin. How can someone help?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LnL7 LnL7 LnL7 left review comments

Assignees
No one assigned
Labels
2.status: merge conflict 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 11-100 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

glibc fails to build on macOS glibc can't be built on darwin
6 participants
@Gaelan @siraben @SuperSandro2000 @tartavull @LnL7 @vcunat