Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

hdf4: fix build on aarch64-linux #47209

Merged
merged 1 commit into from Sep 23, 2018
Merged

Conversation

andrew-d
Copy link
Contributor

Motivation for this change

Pull in the Debian patches for AArch64 support, and a bonus patch for reproducible builds while we're at it.

I don't have an AArch64 host to test on, so I'd appreciate someone poking @GrahamcOfBorg for me 馃槂 I think this should work, though, according to the failures in the build log.

/cc ZHF #45960

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

cc @knedlsepp

@knedlsepp
Copy link
Member

@GrahamcOfBorg build hdf4

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

patching script interpreter paths in /nix/store/d6xx9yhiql3bbvnwis3fl2xyzsbm4063-hdf-4.2.14-dev
checking for references to /build in /nix/store/d6xx9yhiql3bbvnwis3fl2xyzsbm4063-hdf-4.2.14-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14
shrinking /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14/lib/libhdf.so.4.2.14
shrinking /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14/lib/libmfhdf.so.4.2.14
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14/lib
patching script interpreter paths in /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14
checking for references to /build in /nix/store/l2znbc5q7c8b881vmmpcrrnr39yqxn67-hdf-4.2.14...
/nix/store/1kdqz51m1j2v7qn12vzhk3n7ph436wkn-hdf-4.2.14-bin

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

patching script interpreter paths in /nix/store/zhbynn6mi5jxlp61jpklycayyy5kgwa8-hdf-4.2.14-dev
checking for references to /build in /nix/store/zhbynn6mi5jxlp61jpklycayyy5kgwa8-hdf-4.2.14-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14
shrinking /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14/lib/libmfhdf.so.4.2.14
shrinking /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14/lib/libhdf.so.4.2.14
strip is /nix/store/y4ymnvgxygpq05h03kyzbj572zmh6zla-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14/lib
patching script interpreter paths in /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14
checking for references to /build in /nix/store/vggxk22ckcmynvy62l49dm6q8asf8hmg-hdf-4.2.14...
/nix/store/0zrrs4cmiwwpffj03k6qicnyh83h863i-hdf-4.2.14-bin

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: hdf4

Partial log (click to expand)

rmdir: failed to remove '/nix/store/mbb4npppp0csk29qmdky8k4n22y308qp-hdf-4.2.14': Directory not empty
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/87zpazlzpzhg70nns8dnf3vn0b2jknp7-hdf-4.2.14-bin/bin
patching script interpreter paths in /nix/store/87zpazlzpzhg70nns8dnf3vn0b2jknp7-hdf-4.2.14-bin
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/fdmc3n39l6202xzirq781zyrjgvggw9w-hdf-4.2.14-dev
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/mbb4npppp0csk29qmdky8k4n22y308qp-hdf-4.2.14/lib
patching script interpreter paths in /nix/store/mbb4npppp0csk29qmdky8k4n22y308qp-hdf-4.2.14
/nix/store/87zpazlzpzhg70nns8dnf3vn0b2jknp7-hdf-4.2.14-bin

let
debianPatches = fetchurl {
# NOTE: debian hasn't packaged 4.2.14 just yet, but these patches apply cleanly.
url = "mirror://debian/pool/main/libh/libhdf4/libhdf4_4.2.13-4.debian.tar.xz";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These debian source URLs are not stable. They're not under version control and may be removed when Debian releases a new patch level (for instance, 4.2.13-3 doesn't exist anymore, only 4.2.13-4), leading to 404s in our builds ...

Please fetchpatch the patches from the Debian gitlab repo https://salsa.debian.org/debian-gis-team/hdf4/tree/4e4e3f543e325911533b8617e2eb4082987f9fe2/debian/patches instead. These are pinned to a git revision and don't change or disappear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, good to know! I switched to individual fetchpatches for those patches, since I didn't want to clone the whole git repo in order to fetch 4 files 馃槂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, LGTM now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and yes , fetchpatch ing each patch individually is the standard way to do it 馃憤 ).

Pull in the Debian patches for AArch64 support, and a bonus patch for
reproducible builds while we're at it.
@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

@GrahamcOfBorg build hdf4

@GrahamcOfBorg
Copy link

Unexpected error: command failed with exit code 1 on x86_64-darwin (full log)

Attempted: hdf4

Partial log (click to expand)

copying path '/nix/store/1ikjdn4r0kmz995cvwnks4sdywi9qp7i-libjpeg-turbo-1.5.3-bin' from 'https://cache.nixos.org'...
building '/nix/store/p85hvxclzhha43z7f21p2glvkh4sd3w9-64bit.drv'...
building '/nix/store/ikvi6vj3wzlx84cwb6fwsq082i3d3gi8-hdf-4.2.10-aarch64.patch.drv'...
/nix/store/cz8xwi6lmz86lbhj7a1nzlhz2n8j6i9f-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
/nix/store/cz8xwi6lmz86lbhj7a1nzlhz2n8j6i9f-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
building '/nix/store/03rk70xbr3v171l5gh8407m0w8zqqdpf-hdfi.h.drv'...
/nix/store/cz8xwi6lmz86lbhj7a1nzlhz2n8j6i9f-stdenv-darwin/setup: fork: retry: Resource temporarily unavailable
copying path '/nix/store/896hr8l72qhyhp57xz44856m5ab273w2-libjpeg-turbo-1.5.3-dev' from 'https://cache.nixos.org'...
error (ignored): unable to fork: Resource temporarily unavailable
error: unable to fork: Resource temporarily unavailable

@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

darwin: looks like a transient builder issue.

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

patching script interpreter paths in /nix/store/h87ndq9vza6myjk1zlhwi2kcmhsq5ipw-hdf-4.2.14-dev
checking for references to /build in /nix/store/h87ndq9vza6myjk1zlhwi2kcmhsq5ipw-hdf-4.2.14-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14
shrinking /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14/lib/libhdf.so.4.2.14
shrinking /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14/lib/libmfhdf.so.4.2.14
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14/lib
patching script interpreter paths in /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14
checking for references to /build in /nix/store/1xy070lnjndvyr43i1hcvdbhrc52d8qn-hdf-4.2.14...
/nix/store/829183d08c6dkkl35sqv8wybdrjqflpk-hdf-4.2.14-bin

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

patching script interpreter paths in /nix/store/3cjrvdi8rc445v2nydz2knc4mkh456ml-hdf-4.2.14-dev
checking for references to /build in /nix/store/3cjrvdi8rc445v2nydz2knc4mkh456ml-hdf-4.2.14-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14
shrinking /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14/lib/libmfhdf.so.4.2.14
shrinking /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14/lib/libhdf.so.4.2.14
strip is /nix/store/y4ymnvgxygpq05h03kyzbj572zmh6zla-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14/lib
patching script interpreter paths in /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14
checking for references to /build in /nix/store/0axqgc6d9phb0wz3lk2z1hpkn5al2k6c-hdf-4.2.14...
/nix/store/syv5d0imq30j4qqjrqb9zwkcrbbwql36-hdf-4.2.14-bin

@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

Try again for darwin:
@GrahamcOfBorg build hdf4

@GrahamcOfBorg
Copy link

Unexpected error: command failed with exit code 1 on x86_64-darwin (full log)

Attempted: hdf4

Partial log (click to expand)

  /nix/store/03rk70xbr3v171l5gh8407m0w8zqqdpf-hdfi.h.drv
  /nix/store/b2kza6br518afsx39sqn1gz4r5f6aq3j-reproducible-builds.patch.drv
  /nix/store/ikvi6vj3wzlx84cwb6fwsq082i3d3gi8-hdf-4.2.10-aarch64.patch.drv
  /nix/store/p85hvxclzhha43z7f21p2glvkh4sd3w9-64bit.drv
  /nix/store/599ra6d97kmx5p05lkglr8c14w2y53r1-hdf-4.2.14.drv
building '/nix/store/p85hvxclzhha43z7f21p2glvkh4sd3w9-64bit.drv'...
building '/nix/store/ikvi6vj3wzlx84cwb6fwsq082i3d3gi8-hdf-4.2.10-aarch64.patch.drv'...
building '/nix/store/03rk70xbr3v171l5gh8407m0w8zqqdpf-hdfi.h.drv'...
error (ignored): unable to fork: Resource temporarily unavailable
error: unable to fork: Resource temporarily unavailable

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

/nix/store/829183d08c6dkkl35sqv8wybdrjqflpk-hdf-4.2.14-bin

@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

(unfortunately we cannot tell ofborg to only build a package for one platform yet)

@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

OK, let's ignore darwin for now, looks like the builder is out of resources.

Thank you for fixing this!

@xeji xeji merged commit 104014c into NixOS:master Sep 23, 2018
xeji pushed a commit that referenced this pull request Sep 23, 2018
Pull in the Debian patches for AArch64 support, and a bonus patch for
reproducible builds while we're at it.

(cherry picked from commit 104014c)
@xeji
Copy link
Contributor

xeji commented Sep 23, 2018

backported: 18a7c1d

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: hdf4

Partial log (click to expand)

/nix/store/syv5d0imq30j4qqjrqb9zwkcrbbwql36-hdf-4.2.14-bin

@andrew-d
Copy link
Contributor Author

@xeji - And thank you for the help!

@andrew-d andrew-d deleted the andrew/hdf4-aarch64 branch September 23, 2018 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants