Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openblas: 0.3.4 -> 0.3.5, rework a bit #53972

Merged
merged 4 commits into from Jan 15, 2019

Conversation

dtzWill
Copy link
Member

@dtzWill dtzWill commented Jan 14, 2019

  • prefer attributes
  • let nix string-ify for us
  • bools become the same 1/0, FWIW, in case that matters
Motivation for this change

https://github.com/xianyi/OpenBLAS/releases/tag/v0.3.5

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member

FRidh commented Jan 15, 2019

This needs a rebase after my CC change.

* doCheck should be 'true' in (at least) the non-cross case,
  this looks like an inverted check that's largely benign
* doCheck will be set to 'false' in the cross case anyway,
  makeDerivation does this IIRC
* targetPrefix can be used without checking, probably by design

Derivation hash does change but no "real" functionality change intended.
(but set appropriately for cross and non-cross cases both)

* I'm not sure what NO_BINARY_MODE does,
  this change now sets explicitly false in the non-cross scenario
  (previously unset unless cross).
* Drop musl NO_AFFINITY case, will be removed in upgrade shortly
@dtzWill
Copy link
Member Author

dtzWill commented Jan 15, 2019

rebased! Basically same changes but across a few commits to make easier for review, squash as preferred (or LMK and i'll do so).

@FRidh FRidh merged commit 1f04670 into NixOS:staging Jan 15, 2019
@FRidh
Copy link
Member

FRidh commented Jan 15, 2019

Thanks for the improvements and the doCheck fix.

@dtzWill
Copy link
Member Author

dtzWill commented Jan 31, 2019

FWIW #54886 conflicts with this, I resolved it in my tree with this result: https://github.com/dtzWill/nixpkgs/blob/1c5bfc30a7fcf22138adb8f8441d2c40dda201e1/pkgs/development/libraries/science/math/openblas/default.nix

Quick notes about what was done and why: dtzWill@1c5bfc3

Thought I'd share for whoever/whenever master and staging get merged.

@vcunat
Copy link
Member

vcunat commented Feb 2, 2019

This commit (1f046700) broke openblas build on aarch64-linux. (I re-tried the commit itself and its parent.) Error logs e.g. on Hydra: https://hydra.nixos.org/build/87629037

@ttuegel
Copy link
Member

ttuegel commented Feb 2, 2019

Is it related to the DYNAMIC_ARCH and USE_OPENMP changes?

@dtzWill
Copy link
Member Author

dtzWill commented Feb 3, 2019

Hmm, two things I noticed:

  1. While builtins.toString true is "1", apparently builtins.toString false is "". So the values that used to be "0" are now empty, which likely has a different meaning to openblas's build system. Sorry, I really thought i checked that :/.

  2. Why is fortran a buildPackages like HOSTCC? Is that correct? Whatever the right answer is, we may need to fix/change the prefix used to match its role.

  3. I don't know if this is the trouble but FWIW this enables tests in the non-cross case (as usual), we were apparently previously disabling them unless it was cross... which got overridden to disabled by mkDerivation anyway >.<.

Very sorry re:1, although it's a shame because I like this much better. But broken is worse than ugly! :P

@vcunat
Copy link
Member

vcunat commented Feb 10, 2019

Note: I just merged this into master via c40f211. I didn't feel like attempting to solve this as well, and I guess blas on aarch64 isn't such a big deal, so we might afford to break it temporarily (I hope).

@vcunat
Copy link
Member

vcunat commented Feb 10, 2019

dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request Feb 12, 2019
My earlier change mistakenly expected `toString false` to produce '0'
instead of the empty string, leading to unexpected config changes.

Intended to address issue mentioned here and in following discussion:

NixOS#53972 (comment)

Sorry, folks!

(special-case handling of bools here makes this "cleanup" a bit
 less of an obvious win but hopefully still preferable overall :))

-----------

makeFlags in resulting derivation, according to this one-liner:

$ nix show-derivation -f . openblas|jq ".[].env.makeFlags"

before:
"BINARY=64 CC=cc CROSS= DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE= NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

after:
"BINARY=64 CC=cc CROSS=0 DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE=0 NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

Without knowing how `placeholder` works, it seems interesting if
entirely unrelated that the `PREFIX` is same for both! :). TIL.
dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request Feb 12, 2019
My earlier change mistakenly expected `toString false` to produce '0'
instead of the empty string, leading to unexpected config changes.

Intended to address issue mentioned here and in following discussion:

NixOS#53972 (comment)

Sorry, folks!

(special-case handling of bools here makes this "cleanup" a bit
 less of an obvious win but hopefully still preferable overall :))

-----------

makeFlags in resulting derivation, according to this one-liner:

$ nix show-derivation -f . openblas|jq ".[].env.makeFlags"

before:
"BINARY=64 CC=cc CROSS= DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE= NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

after:
"BINARY=64 CC=cc CROSS=0 DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE=0 NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

Without knowing how `placeholder` works, it seems interesting if
entirely unrelated that the `PREFIX` is same for both! :). TIL.
FRidh pushed a commit that referenced this pull request Feb 17, 2019
My earlier change mistakenly expected `toString false` to produce '0'
instead of the empty string, leading to unexpected config changes.

Intended to address issue mentioned here and in following discussion:

#53972 (comment)

Sorry, folks!

(special-case handling of bools here makes this "cleanup" a bit
 less of an obvious win but hopefully still preferable overall :))

-----------

makeFlags in resulting derivation, according to this one-liner:

$ nix show-derivation -f . openblas|jq ".[].env.makeFlags"

before:
"BINARY=64 CC=cc CROSS= DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE= NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

after:
"BINARY=64 CC=cc CROSS=0 DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE=0 NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"

Without knowing how `placeholder` works, it seems interesting if
entirely unrelated that the `PREFIX` is same for both! :). TIL.
@averelld averelld mentioned this pull request Feb 22, 2019
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants