Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brotli: ensure CMAKE_SYSTEM_NAME=Windows on mingw cross builds #46773

Merged
merged 1 commit into from Sep 22, 2018

Conversation

mcmtroffaes
Copy link
Contributor

Motivation for this change

CMAKE passes the flag -rdynamic otherwise (which is not recognized by mingw). Setting the appropriate flag avoids this problem and allows the build to succeed.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions (Fedora)
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

I've also tested the resulting executable using wine on fedora (compressing & decompressing a file, checked the difference).

@vcunat @Ericson2314

CMAKE passes the flag -rdynamic otherwise (which is not recognized by mingw).
Setting the appropriate flag avoids this problem and allows the build
to succeed.
@mcmtroffaes
Copy link
Contributor Author

For reference, here's the shell.nix file I used for testing:

with import <nixpkgs> {};
{
  my-env = pkgs.stdenv.mkDerivation {
    name = "my-env";
    buildInputs = [
      pkgs.pkgsCross.mingwW64.brotli
    ];
  };
}

@matthewbauer
Copy link
Member

Thanks! I'm glad someone is also working with Mingw.

I think we can actually set this up for all CMake builds. Reading the documentation, it looks like by default CMAKE_SYSTEM_NAME is set to CMAKE_HOST_SYSTEM_NAME (which awkwardly is a different kind of "host" compared to the one in Nixpkgs): https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_NAME.html

The only thing is keeping track of uname -s output for each system. @Ericson2314 is the something we can do? (otherwise we can probably just map "kernel" to system name).

# We should set the proper `CMAKE_SYSTEM_NAME`.
# http://www.cmake.org/Wiki/CMake_Cross_Compiling

@matthewbauer
Copy link
Member

@GrahamcOfBorg build pkgs.pkgsCross.mingwW64.brotli

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: pkgs.pkgsCross.mingwW64.brotli

Partial log (click to expand)

patching script interpreter paths in /nix/store/ddc8fkjnsgisyam7yz5zqswzxbpfcpyg-brotli-1.0.5-x86_64-pc-mingw32
shrinking RPATHs of ELF executables and libraries in /nix/store/kyai2bvc0g323dfajh3xr42yil6rdprl-brotli-1.0.5-x86_64-pc-mingw32-dev
x86_64-pc-mingw32-strip is /nix/store/83sfv5l525j0jygfri303pcdzl98r844-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/kyai2bvc0g323dfajh3xr42yil6rdprl-brotli-1.0.5-x86_64-pc-mingw32-dev/lib
patching script interpreter paths in /nix/store/kyai2bvc0g323dfajh3xr42yil6rdprl-brotli-1.0.5-x86_64-pc-mingw32-dev
shrinking RPATHs of ELF executables and libraries in /nix/store/p4bpyk1fwicyq5lwzkza75wvnmb6255g-brotli-1.0.5-x86_64-pc-mingw32-lib
x86_64-pc-mingw32-strip is /nix/store/83sfv5l525j0jygfri303pcdzl98r844-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/p4bpyk1fwicyq5lwzkza75wvnmb6255g-brotli-1.0.5-x86_64-pc-mingw32-lib/lib
patching script interpreter paths in /nix/store/p4bpyk1fwicyq5lwzkza75wvnmb6255g-brotli-1.0.5-x86_64-pc-mingw32-lib
/nix/store/ddc8fkjnsgisyam7yz5zqswzxbpfcpyg-brotli-1.0.5-x86_64-pc-mingw32

@Ericson2314
Copy link
Member

Yeah CMAKE_SYSTEM_* looks like Autoconf's and our host platform, and CMAKE_HOST_SYSTEM_* looks like Autoconf's and our build platform. If I understand https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling correctly, those two might have the same structure, though they only spell it out explicitly for *_NAME.

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: pkgs.pkgsCross.mingwW64.brotli

Partial log (click to expand)

patching script interpreter paths in /nix/store/6gnmlv02hka0hdfkyiim1809j92sk0rr-brotli-1.0.5-x86_64-pc-mingw32
shrinking RPATHs of ELF executables and libraries in /nix/store/dwwhixasf2cnvrfl245k745ygysq9y7s-brotli-1.0.5-x86_64-pc-mingw32-dev
x86_64-pc-mingw32-strip is /nix/store/9k60x6qpcid97j9f0lj6g66ikq0j7h1y-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/dwwhixasf2cnvrfl245k745ygysq9y7s-brotli-1.0.5-x86_64-pc-mingw32-dev/lib
patching script interpreter paths in /nix/store/dwwhixasf2cnvrfl245k745ygysq9y7s-brotli-1.0.5-x86_64-pc-mingw32-dev
shrinking RPATHs of ELF executables and libraries in /nix/store/gngp1rcw34jd488hapiss83vvq4bsm6z-brotli-1.0.5-x86_64-pc-mingw32-lib
x86_64-pc-mingw32-strip is /nix/store/9k60x6qpcid97j9f0lj6g66ikq0j7h1y-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/gngp1rcw34jd488hapiss83vvq4bsm6z-brotli-1.0.5-x86_64-pc-mingw32-lib/lib
patching script interpreter paths in /nix/store/gngp1rcw34jd488hapiss83vvq4bsm6z-brotli-1.0.5-x86_64-pc-mingw32-lib
/nix/store/6gnmlv02hka0hdfkyiim1809j92sk0rr-brotli-1.0.5-x86_64-pc-mingw32

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: pkgs.pkgsCross.mingwW64.brotli

Partial log (click to expand)

x86_64-pc-mingw32-strip is /nix/store/hhz6hd6ykdbaxlpdbd227sqn3z44bcvf-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/kn22gady26r5fpdz35965dixsxxwqk0n-brotli-1.0.5-x86_64-pc-mingw32/bin
patching script interpreter paths in /nix/store/kn22gady26r5fpdz35965dixsxxwqk0n-brotli-1.0.5-x86_64-pc-mingw32
x86_64-pc-mingw32-strip is /nix/store/hhz6hd6ykdbaxlpdbd227sqn3z44bcvf-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/bz6w6lwkw32qmzjhz4p9d2k23x2pgwil-brotli-1.0.5-x86_64-pc-mingw32-dev/lib
patching script interpreter paths in /nix/store/bz6w6lwkw32qmzjhz4p9d2k23x2pgwil-brotli-1.0.5-x86_64-pc-mingw32-dev
x86_64-pc-mingw32-strip is /nix/store/hhz6hd6ykdbaxlpdbd227sqn3z44bcvf-x86_64-pc-mingw32-binutils-2.30/bin/x86_64-pc-mingw32-strip
stripping (with command x86_64-pc-mingw32-strip and flags -S) in /nix/store/2n2dgnwf6ynaprcpx9lwrpfma5sagjwz-brotli-1.0.5-x86_64-pc-mingw32-lib/lib
patching script interpreter paths in /nix/store/2n2dgnwf6ynaprcpx9lwrpfma5sagjwz-brotli-1.0.5-x86_64-pc-mingw32-lib
/nix/store/kn22gady26r5fpdz35965dixsxxwqk0n-brotli-1.0.5-x86_64-pc-mingw32

@mcmtroffaes
Copy link
Contributor Author

Many thanks for the review @matthewbauer @Ericson2314 - I agree it would be ideal if this were to be fixed for all packages at once via the cmake setup hook. I would be happy to try to contribute a patch to that effect. However, I'm not that experienced with setup hooks, so please forgive me for I cannot see immediately how to easily detect within a setup hook that we are running a mingw build (or more generally, that we are running a cross build and what the target then is). I've grep'd all current setup hooks in the repository for inspiration, to no avail. Suggestions?

Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay to merge right now (not sure if it should be master or staging though). Discussing with @Ericson2314 how we can do this everywhere - it gets a little complicated because now we need some kind of a wrapper to hold what the targeted system is, previously we reused the same thing every time. Anyway, it's used in other places already so should be okay.

@Ericson2314
Copy link
Member

Maybe I would add this version to 18.09, and do the wrapper on maste/staging.

@matthewbauer
Copy link
Member

matthewbauer commented Sep 19, 2018

I am still wanting to explore ways to avoid the wrapping of cmake. The setup hook should have enough information to get the target from the c compiler without a wrapper. Somehow parsing dumpmachine output? Or some other variable

@Ericson2314
Copy link
Member

Ericson2314 commented Sep 20, 2018

I just don't see how computing the same information every time dynamically is superior to doing it once statically. The wrapper would be a lot simpler than cc-wrapper. It can just be the setup hook and a propagated dependency, in fact.

@matthewbauer matthewbauer merged commit 3bc7ad9 into NixOS:master Sep 22, 2018
@Ericson2314
Copy link
Member

I agree. Merge this now and generalize after.

@vcunat
Copy link
Member

vcunat commented Sep 23, 2018

It's possible to create the setup hook once, based on the platform, but I don't see a simple nice way immediately.

@mcmtroffaes mcmtroffaes deleted the feature/fix-brotli-mingw branch October 1, 2018 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants