Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pythonPackages.kmsxx: fix build #38147

Merged
merged 1 commit into from Apr 8, 2018
Merged

pythonPackages.kmsxx: fix build #38147

merged 1 commit into from Apr 8, 2018

Conversation

xeji
Copy link
Contributor

@xeji xeji commented Mar 30, 2018

Motivation for this change

gcc7 build fails with "internal compiler error", use gcc6 instead.

(#37653 fixed this for the regular kmsxx package but not for the python packages which are built from the same expression. Not sure why we have both, looks kinda redundant, but let's fix it first.)

/cc ZHF #36453

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@xeji
Copy link
Contributor Author

xeji commented Mar 30, 2018

@GrahamcOfBorg build kmsxx python27Packages.kmsxx python36Packages.kmsxx

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: kmsxx, python27Packages.kmsxx, python36Packages.kmsxx

Partial log (click to expand)

checking for references to /build in /nix/store/fsd34vg01nzk646ajv565szdliikfdp8-kmsxx-2017-10-10...
shrinking /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10/bin/kmstest
shrinking /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10/lib/python3.6/site-packages/pykms/pykms.so
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10/lib  /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10
checking for references to /build in /nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10...
/nix/store/jl60fb888fzy3czs935i656ibi2iz4pb-kmsxx-2017-10-10
/nix/store/fsd34vg01nzk646ajv565szdliikfdp8-kmsxx-2017-10-10
/nix/store/7fjjkp411xvwlzg8dzmbggvbv7hxb99y-kmsxx-2017-10-10

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: kmsxx, python27Packages.kmsxx, python36Packages.kmsxx

Partial log (click to expand)

shrinking /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10/bin/kmstest
shrinking /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10/bin/fbtest
shrinking /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10/lib/python2.7/site-packages/pykms/pykms.so
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10/lib  /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10
checking for references to /build in /nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10...
/nix/store/sabn95xd54yin0l4dxcxf8wpx9p9gw8s-kmsxx-2017-10-10
/nix/store/s1zq8zx90la2x9hf076l08kzp5ss5alm-kmsxx-2017-10-10
/nix/store/3fz2rgw93vlx4nki05ql7kpcsg7h5f0a-kmsxx-2017-10-10

@fpletz
Copy link
Member

fpletz commented Apr 1, 2018

Please override stdenv in pkgs/top-level/all-packages.nix instead: stdenv = overrideCC stdenv gcc6;

@xeji
Copy link
Contributor Author

xeji commented Apr 1, 2018

I did that overrideCC for the non-python kmsxx package previously and tried to do the same for the python packages built from pkgs/top-level/python-packages.nix but that did not work. Is there another way?
Edit: This is a weird expression because it is referenced from both all-packages.nix and python-packages.nix

@xeji
Copy link
Contributor Author

xeji commented Apr 2, 2018

Turns out you can override gcc from top-level/python-packages.nix, although overrideCC isn't defined in that context.

@xeji
Copy link
Contributor Author

xeji commented Apr 2, 2018

@GrahamcOfBorg build kmsxx python27Packages.kmsxx python36Packages.kmsxx

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: kmsxx, python27Packages.kmsxx, python36Packages.kmsxx

Partial log (click to expand)

strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/k1afk5xr2mph7rfvnramr8k4a5hl2klw-kmsxx-2017-10-10/lib  /nix/store/k1afk5xr2mph7rfvnramr8k4a5hl2klw-kmsxx-2017-10-10/bin
stripping (with command strip and flags -S) in /nix/store/jzi8hmcl9hrr9hzg3mqf2n3m59ffx86n-kmsxx-2017-10-10/lib  /nix/store/jzi8hmcl9hrr9hzg3mqf2n3m59ffx86n-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/k1afk5xr2mph7rfvnramr8k4a5hl2klw-kmsxx-2017-10-10
patching script interpreter paths in /nix/store/jzi8hmcl9hrr9hzg3mqf2n3m59ffx86n-kmsxx-2017-10-10
checking for references to /build in /nix/store/jzi8hmcl9hrr9hzg3mqf2n3m59ffx86n-kmsxx-2017-10-10...
checking for references to /build in /nix/store/k1afk5xr2mph7rfvnramr8k4a5hl2klw-kmsxx-2017-10-10...
/nix/store/pbsb5p7qixycv6v5ddi9faxsr4ikb4lj-kmsxx-2017-10-10
/nix/store/jzi8hmcl9hrr9hzg3mqf2n3m59ffx86n-kmsxx-2017-10-10
/nix/store/k1afk5xr2mph7rfvnramr8k4a5hl2klw-kmsxx-2017-10-10

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: kmsxx, python27Packages.kmsxx, python36Packages.kmsxx

Partial log (click to expand)

shrinking /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10/bin/kmsprint
shrinking /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10/bin/kmstest
shrinking /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10/lib/python2.7/site-packages/pykms/pykms.so
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10/lib  /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10
checking for references to /build in /nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10...
/nix/store/ck8rh6zhylv34bk3wcx99irjrq0dd0xy-kmsxx-2017-10-10
/nix/store/n2bf4zilzshilh4v81qjmvwa72jafdwx-kmsxx-2017-10-10
/nix/store/bccgflscx3qlrhfbj2iza4jg3fg0axkx-kmsxx-2017-10-10

kmsxx = callPackage ../development/libraries/kmsxx { };
kmsxx = callPackage ../development/libraries/kmsxx {
# overrideCC isn't defined here, let's do without it
stdenv = stdenv.override { allowedRequisites = null; cc = pkgs.gcc6; };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use

stdenv = pkgs.overrideCC stdenv pkgs.gcc6;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I think doing it similar to dlib expression would be a better approach:

  dlib = buildPythonPackage rec {
    inherit (pkgs.dlib) name src nativeBuildInputs meta;

    buildInputs = pkgs.dlib.buildInputs ++ [ self.boost ];

    checkInputs = with self; [ pytest ];
  };

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I tried your second suggestion but did not get it to work (the package is built using cmake, not setuptools), so let's go with your first suggestion instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I came up with an even better solution: inherit (pkgs.kmsxx) stdenv;.
And you should maybe add name = "${python.executable}-${pkgs.kmsxx.name}".
So we have

kmsxx = (callPackage ../development/libraries/kmsxx {
  inherit (pkgs.kmsxx) stdenv;
}).overrideAttrs (oldAttrs: {
  name = "${python.executable}-${pkgs.kmsxx.name}";
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Works fine, but I think we should use name = "${python.libPrefix}-${pkgs.kmsxx.name}", otherwise the python3 package will be called python3.6m-kmsxx...

gcc7 build fails with "internal compiler error", use gcc6 instead
by inheriting stdenv from pkgs.kmsxx
@xeji
Copy link
Contributor Author

xeji commented Apr 8, 2018

@GrahamcOfBorg build python.pkgs.kmsxx python3.pkgs.kmsxx

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python.pkgs.kmsxx, python3.pkgs.kmsxx

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10
shrinking /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/lib/python3.6/site-packages/pykms/pykms.so
shrinking /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/bin/kmstest
shrinking /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/bin/kmsprint
shrinking /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/bin/fbtest
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/lib  /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10
checking for references to /build in /nix/store/cribficzz9q3nj2jid2dmlbcshm9dmvy-python3.6-kmsxx-2017-10-10...

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python.pkgs.kmsxx, python3.pkgs.kmsxx

Partial log (click to expand)

shrinking /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/bin/fbtest
shrinking /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/bin/kmsprint
shrinking /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/bin/kmstest
shrinking /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/lib/python3.6/site-packages/pykms/pykms.so
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/lib  /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10/bin
patching script interpreter paths in /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10
checking for references to /build in /nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10...
/nix/store/cp21r7i9r05cyjdfxbqyh4c54l8s2xxd-python2.7-kmsxx-2017-10-10
/nix/store/gj9pa2gcvnccbvi14znhxsfvabb0dysi-python3.6-kmsxx-2017-10-10

@dotlambda dotlambda merged commit 17bfcb5 into NixOS:master Apr 8, 2018
@dotlambda
Copy link
Member

Thanks a lot!

@dotlambda
Copy link
Member

Backported in df3a740.

@xeji xeji deleted the python-kmsxx branch April 8, 2018 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants