Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

amd-libflame: init at 2.2 #94991

Merged
merged 1 commit into from Aug 9, 2020
Merged

amd-libflame: init at 2.2 #94991

merged 1 commit into from Aug 9, 2020

Conversation

danieldk
Copy link
Contributor

@danieldk danieldk commented Aug 9, 2020 •
edited

Motivation for this change

libflame is a protable library for dense matrix computations, providing a complete LAPACK implementation. The AMD fork of libflame is optimized for AMD CPUs.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@danieldk
amd-libflame: init at 2.2
a49a59b 
libflame is a protable library for dense matrix computations,
providing a complete LAPACK implementation. The AMD fork of libflame
is optimized for AMD CPUs.
bhipple
bhipple requested changes Aug 9, 2020
View reviewed changes
pkgs/development/libraries/science/math/amd-libflame/default.nix
# up to the application to explicitly link to a BLAS. This is
# problematic for us, since then the BLAS library becomes an
# implicit dependency. Moreover, since the point of the AMD forks
# is to optimized for recent AMD CPUs, link against AMD BLIS.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we can hook into @matthewbauer's nice alternatives implementation to provide AMD BLIS as a blas provider, the way we do for Intel MKL and openblas?

This gives you one overlay variable that you can update to slice all of NixPkgs consistently between openblas / mkl / etc. The purpose here is to allow someone to slice a numpy / pandas / scipy / tensorflow / pytorch / etc. stack between MKL vs. openblas vs. whatever in O(1) package modifications instead of O(n) with the risk of missing some and ending in an incoherent build.

nixpkgs/doc/using/overlays.xml

Lines 140 to 262 in 21be830

<section xml:id="sec-overlays-alternatives">
<title>Using overlays to configure alternatives</title>
<para>
Certain software packages have different implementations of the
same interface. Other distributions have functionality to switch
between these. For example, Debian provides <link
xlink:href="https://wiki.debian.org/DebianAlternatives">DebianAlternatives</link>.
Nixpkgs has what we call <literal>alternatives</literal>, which
are configured through overlays.
</para>
<section xml:id="sec-overlays-alternatives-blas-lapack">
<title>BLAS/LAPACK</title>
<para>
In Nixpkgs, we have multiple implementations of the BLAS/LAPACK
numerical linear algebra interfaces. They are:
</para>
<itemizedlist>
<listitem>
<para>
<link xlink:href="https://www.openblas.net/">OpenBLAS</link>
</para>
<para>
The Nixpkgs attribute is <literal>openblas</literal> for
ILP64 (integer width = 64 bits) and
<literal>openblasCompat</literal> for LP64 (integer width =
32 bits). <literal>openblasCompat</literal> is the default.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="http://www.netlib.org/lapack/">LAPACK
reference</link> (also provides BLAS)
</para>
<para>
The Nixpkgs attribute is <literal>lapack-reference</literal>.
</para>
</listitem>
<listitem>
<para>
<link
xlink:href="https://software.intel.com/en-us/mkl">Intel
MKL</link> (only works on x86 architecture, unfree)
</para>
<para>
The Nixpkgs attribute is <literal>mkl</literal>.
</para>
</listitem>
</itemizedlist>
<para>
Introduced in <link
xlink:href="https://github.com/NixOS/nixpkgs/pull/83888">PR
#83888</link>, we are able to override the ‘blas’ and ‘lapack’
packages to use different implementations, through the
‘blasProvider’ and ‘lapackProvider’ argument. This can be used
to select a different provider. BLAS providers will have
symlinks in <literal>$out/lib/libblas.so.3</literal> and
<literal>$out/lib/libcblas.so.3</literal> to their respective
BLAS libraries. Likewise, LAPACK providers will have symlinks
in <literal>$out/lib/liblapack.so.3</literal> and
<literal>$out/lib/liblapacke.so.3</literal> to their respective
LAPCK libraries. For example, Intel MKL is both a BLAS and
LAPACK provider. An overlay can be created to use Intel MKL
that looks like:
</para>
<programlisting>
self: super:
{
blas = super.blas.override {
blasProvider = self.mkl;
}
lapack = super.lapack.override {
lapackProvider = self.mkl;
}
}
</programlisting>
<para>
This overlay uses Intel’s MKL library for both BLAS and LAPACK
interfaces. Note that the same can be accomplished at runtime
using <literal>LD_LIBRARY_PATH</literal> of libblas.so.3 and
liblapack.so.3. For instance:
</para>
<programlisting>
$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib:$LD_LIBRARY_PATH nix-shell -p octave --run octave
</programlisting>
<para>
Intel MKL requires an <literal>openmp</literal> implementation
when running with multiple processors. By default,
<literal>mkl</literal> will use Intel’s <literal>iomp</literal>
implementation if no other is specified, but this is a
runtime-only dependency and binary compatible with the LLVM
implementation. To use that one instead, Intel recommends users
set it with <literal>LD_PRELOAD</literal>. Note that
<literal>mkl</literal> is only available on
<literal>x86_64-linux</literal> and
<literal>x86_64-darwin</literal>. Moreover, Hydra is not
building and distributing pre-compiled binaries using it.
</para>
<para>
For BLAS/LAPACK switching to work correctly, all packages must
depend on <literal>blas</literal> or <literal>lapack</literal>.
This ensures that only one BLAS/LAPACK library is used at one
time. There are two versions versions of BLAS/LAPACK currently
in the wild, <literal>LP64</literal> (integer size = 32 bits)
and <literal>ILP64</literal> (integer size = 64 bits). Some
software needs special flags or patches to work with
<literal>ILP64</literal>. You can check if
<literal>ILP64</literal> is used in Nixpkgs with
<varname>blas.isILP64</varname> and
<varname>lapack.isILP64</varname>. Some software does NOT work
with <literal>ILP64</literal>, and derivations need to specify
an assertion to prevent this. You can prevent
<literal>ILP64</literal> from being used with the following:
</para>
<programlisting>
{ stdenv, blas, lapack, ... }:
assert (!blas.isILP64) &amp;&amp; (!lapack.isILP64);
stdenv.mkDerivation {
...
}
</programlisting>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we can hook into @matthewbauer's nice alternatives implementation to provide AMD BLIS as a blas provider, the way we do for Intel MKL and openblas?

This should already work. E.g.:

❯ nix-build -E 'with import ./default.nix { overlays = [(self: super: { blas = (super.blas.override { blasProvider = self.amd-blis; }); lapack = (super.lapack.override { lapackProvider = self.amd-libflame; } ); } )]; }; finalfusion-utils'
/nix/store/3lljihln5hph2qxl4d5lhnpg0h53w05x-finalfusion-utils-0.11.2
❯ nix-store -qR result | egrep '(blis|flame)'
/nix/store/d9cssd4xp6d2jy5gazwfpksmpl1wc62v-amd-blis-2.2
/nix/store/66gx4lk596va3kvs97gbyyshyfhb42g8-amd-libflame-2.2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fantastic! Can you update the manual to advertise this nice feature?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try to do that tomorrow!

@danieldk danieldk merged commit 593ec80 into NixOS:master Aug 9, 2020
@danieldk danieldk deleted the amd-libflame branch August 10, 2020 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bhipple bhipple bhipple approved these changes

@matthewbauer matthewbauer Awaiting requested review from matthewbauer

Assignees
No one assigned
Labels
8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 10.rebuild-linux: 1 11.by: package-maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@danieldk @bhipple