Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libav: remove /bin/sh dependencies #36583

Merged
merged 1 commit into from Mar 9, 2018
Merged

libav: remove /bin/sh dependencies #36583

merged 1 commit into from Mar 9, 2018

Conversation

xeji
Copy link
Contributor

@xeji xeji commented Mar 8, 2018

Motivation for this change

impurity resulted in a Hydra build fail on 18.03.

/cc ZHF #36453

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@xeji
libav: remove /bin/sh dependencies
adbd25c 
that resulted in some Hydra build fails
@xeji
Copy link
Contributor Author

xeji commented Mar 8, 2018

cc @shlevy please review

@dtzWill
Copy link
Member

dtzWill commented Mar 8, 2018

bash almost certainly should be a nativeBuildInput

@xeji
Copy link
Contributor Author

xeji commented Mar 8, 2018

One of the outputs installed in bin/ is a shell script, so I considered bash a runtime dependency.

@Mic92
Copy link
Member

Mic92 commented Mar 8, 2018

@GrahamcOfBorg build libav

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Package ‘libpciaccess-0.14’ in /Users/graham/nix-borg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/servers/x11/xorg/default.nix:1115 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Partial log (click to expand)

shrinking /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib/libavresample.so.2.1.0
shrinking /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib/libavcodec.so.56.1.0
shrinking /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib/libavformat.so.56.1.0
shrinking /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib/libavfilter.so.5.0.0
shrinking /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib/libavdevice.so.55.0.0
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12/lib
patching script interpreter paths in /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12
checking for references to /build in /nix/store/zixsmp9lx5v8pdc385q7mkpc1qng0hk4-libav-11.12...
/nix/store/f3q2skh16nhvbarv4d0pw96hzm7vyh97-libav-11.12-bin

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Partial log (click to expand)

shrinking /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib/libavutil.so.54.3.0
shrinking /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib/libavcodec.so.56.1.0
shrinking /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib/libswscale.so.3.0.0
shrinking /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib/libavdevice.so.55.0.0
shrinking /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib/libavformat.so.56.1.0
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12/lib 
patching script interpreter paths in /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12
checking for references to /tmp/nix-build-libav-11.12.drv-0 in /nix/store/mprmpvav9imdj0lk0q6ychip2k7x23rb-libav-11.12...
/nix/store/npfki5l5f8r3h19k7d38zlr5c5f8199i-libav-11.12-bin

shlevy
shlevy approved these changes Mar 9, 2018
View reviewed changes
Copy link
Member

@shlevy shlevy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. We will eventually have a notion of patchBuildtimeShebangs that will be more appropriate in postPatch, but since it doesn't exist yet this will just fail on cross. We can fix that later.

pkgs/development/libraries/libav/default.nix
@@ -45,7 +45,11 @@ let
++ optional (vpxSupport && hasPrefix "0.8." version) ./vpxenc-0.8.17-libvpx-1.5.patch
;

preConfigure = "patchShebangs doc/texi2pod.pl";
postPatch = ''
patchShebangs .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ericson2314 FYI another point that needs patchBuildShebangs

@shlevy shlevy merged commit adbd25c into NixOS:master Mar 9, 2018
@Mic92
Copy link
Member

Mic92 commented Mar 9, 2018 •
edited

@shlevy Do you know what the motivation behind /bin/sh == busybox was? I don't see the advantage of breaking things while even increasing the closure size (we need bash anyway).

@7c6f434c
Copy link
Member

7c6f434c commented Mar 9, 2018

@Mic92 well, a statically-linked busybox could help to solve #1424 completely.

@xeji xeji deleted the libav-shebangs branch March 9, 2018 08:13
@shlevy
Copy link
Member

shlevy commented Mar 9, 2018

@Mic92 I think the main motivation is specifically static linking... bringing in a binary and all of its dependent libraries is sometimes a pain and sometimes impossible to do right, as @7c6f434c points out.

And to be honest, if it were up to me there'd be no /bin/sh in the sandbox at all (or it would just immediately error out with a "nice" error message). Many years ago I tried to argue that system(3) should take sh from PATH...

@Mic92
Copy link
Member

Mic92 commented Mar 9, 2018

@shlevy yes taking /bin/sh away would cause less issues then the current approach.

@Mic92
Copy link
Member

Mic92 commented Mar 9, 2018 •
edited

@shlevy could we at least agree on something like dash then we would share a common ground with ubuntu/debian.

@shlevy
Copy link
Member

shlevy commented Mar 9, 2018

My answer is always going to be in favor of things failing with /bin/sh, sorry 😄 I see the short-term value in a sane common base (though why dash instead of a fully POSIX busybox I don't see), but in the long term /bin/sh dependencies always bite us one way or another. Just don't depend on /bin/sh!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shlevy shlevy shlevy approved these changes

Assignees
No one assigned
Labels
10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@xeji @dtzWill @Mic92 @GrahamcOfBorg @7c6f434c @shlevy