Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llvmPackages_10: cross-compilation support #100574

Closed
wants to merge 2 commits into from
Closed

llvmPackages_10: cross-compilation support #100574

wants to merge 2 commits into from

Conversation

thefloweringash
Copy link
Member

@thefloweringash thefloweringash commented Oct 15, 2020 •
edited

Motivation for this change

Cross compilation of clang 10. Ultimately to make bootstrap tools for platforms that use clang in their stdenv.

Tested building on x86_64-linux with

nix-build \
  --keep-going \
  --argstr crossSystem aarch64-linux \
  -A llvmPackages_10.llvm \
  -A llvmPackages_10.lld \
  -A llvmPackages_10.compiler-rt \
  -A llvmPackages_10.libcxx \
  -A llvmPackages_10.libcxxabi \
  -A llvmPackages_10.clang-unwrapped \
  -A pkgsCross.wasi32.hello

Note that presently I can't test the following due to a build cycle in gcc:

cycle detected in the references of '/nix/store/dkc8jicyd1pibg3hfgasbnvl6schw9da-gcc-debug-9.3.0-aarch64-unknown-linux-gnu-lib' from '/nix/store/nv8nb53m5ypywdgjh6b38pxi0klncgls-gcc-debug-9.3.0-aarch64-unknown-linux-gnu'

nix-build \
  --argstr crossSystem aarch64-linux \
  -A llvmPackages_10.clang \
  -A llvmPackages_10.libcxxClang

Note that while it's possible to build a wrapper, the wrapper uses the build platform's shell so I was unable to test it.

Some interesting points:

  • clang-tblgen doesn't seem to have an install rule within clang, which suggests it's never meant to be installed. To export it without altering the interface I added a tablegen output.
  • llvm-config is installed by llvm to work a bit like pkg-config and help locate components of llvm. Installing the native version to a different output didn't work, since it assumes it's installed alongside llvm. I made up the name llvm-config-native to avoid a conflict, but I'm hoping there's a nicer way to do this. The two versions here are:
    • the installed version, which runs on the host platform, installed by cmake as llvm-config. This tool is probably never used when building with nixpkgs.
    • the "NATIVE" version, which runs on the build platform, installed in postInstall as llvm-config-native. This is required for the lld and clang builds.
  • I don't see why lld would be required to build clang so I removed it. The build didn't fail, but I don't have a strong argument as for why it should or should not be included.
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@thefloweringash thefloweringash mentioned this pull request Nov 26, 2020
Merged
21 tasks
@SuperSandro2000
Copy link
Member

@thefloweringash Can you fix the merge conflict?

matthewbauer
matthewbauer approved these changes Dec 8, 2020
View reviewed changes
Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useLLVM support still works:

$ nix build github:thefloweringash/nixpkgs?ref=clang-10-cross-compile#pkgsCross.wasi32.hello

@thefloweringash
Copy link
Member Author

Rebased to fix the merge conflict, and updated to include current state of testing.

@ck3d ck3d mentioned this pull request Jan 23, 2021
Merged
9 tasks
@Ericson2314
llvm-10: Fix definition of nativeCC
a9f3aa8 
`stdenv` takes tools from the previous stage, so:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
 
while:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
@Ericson2314
Copy link
Member

N.B. I started fixing conflicts and then got carried away; I'll try to open the patch to staging soon.

@Ericson2314 Ericson2314 mentioned this pull request Jan 31, 2021
Merged
12 tasks
@Hoverbear Hoverbear mentioned this pull request Feb 23, 2021
Merged
9 tasks
@thefloweringash
Copy link
Member Author

Going to close this in favor of #111487. There's a similar version of this in #105026 for llvm 11 for making bootstrap tools.

@Ericson2314
Copy link
Member

@thefloweringash well I just edited that one' descriptions to point out I didn't really end up fixing all the cross things :)

@thefloweringash
Copy link
Member Author

@thefloweringash well I just edited that one' descriptions to point out I didn't really end up fixing all the cross things :)

Testing the one use case I'm focused on right now (cross compiling unwrapped llvmPackages_11), it looks like it works as long as I remove lld from clang-unwrapped's buildInputs.

@Ericson2314
Copy link
Member

@thefloweringash do you know why it's deemed needed at all?

@Ericson2314
Copy link
Member

Also would you like to review #111487 ? We have quite little time before the 21.05 staging freeze.

@Ericson2314
Copy link
Member

Hmm it was db29857. Fishy... I'll leave a comment there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ericson2314 Ericson2314 Ericson2314 left review comments

@matthewbauer matthewbauer matthewbauer approved these changes

Assignees
No one assigned
Labels
2.status: merge conflict 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@thefloweringash @SuperSandro2000 @Ericson2314 @matthewbauer