Replace netcat-openbsd with libressl #39634

matthewbauer · 2018-04-28T00:36:34Z

Motivation for this change

Libressl comes with the netcat program. This is much more portable and from the exact same source as Debian's version.

GrahamcOfBorg · 2018-04-28T00:50:57Z

Success on x86_64-darwin (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/54manzdzmz76b478pjqw6brvs5jn4m7y-libressl-2.5.5-bin/bin
patching script interpreter paths in /nix/store/54manzdzmz76b478pjqw6brvs5jn4m7y-libressl-2.5.5-bin
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/fgs67frjn3qz4fnkzqc12k1fmpfnim8c-libressl-2.5.5-dev/lib
patching script interpreter paths in /nix/store/fgs67frjn3qz4fnkzqc12k1fmpfnim8c-libressl-2.5.5-dev
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/8v9sxmabj4xbwhxnpd2haqjl6ksi57aq-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/8v9sxmabj4xbwhxnpd2haqjl6ksi57aq-libressl-2.5.5
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/s01404lk42kdhxkaxqqiafhh5rnl4j2l-libressl-2.5.5-man

GrahamcOfBorg · 2018-04-28T00:52:21Z

Success on x86_64-linux (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5
checking for references to /build in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5...
shrinking RPATHs of ELF executables and libraries in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man
gzipping man pages under /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man/share/man/
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man
checking for references to /build in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man...
/nix/store/cnnm8az8knygq84z1swvgirq7ngx6lk3-libressl-2.5.5-bin
/nix/store/v8xvjqcj8mavgnv1ir79gix09xwwv91n-netcat-gnu-0.7.1

GrahamcOfBorg · 2018-04-28T00:52:29Z

Success on aarch64-linux (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5
checking for references to /build in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5...
shrinking RPATHs of ELF executables and libraries in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man
gzipping man pages under /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man/share/man/
strip is /nix/store/j7d4mr0ikv974ig7yzhknpsq288js4bs-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man
checking for references to /build in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man...
/nix/store/w502vgdw0xvvlaqx7a7829a9znjhwx25-libressl-2.5.5-bin
/nix/store/4688g72pxm23drwza5paqikyqihkvfff-netcat-gnu-0.7.1

dtzWill · 2018-04-28T06:02:26Z

Woohoo! LGTM! Thank you! Looks good w/musl!

Mic92 · 2018-04-28T08:36:05Z

pkgs/development/libraries/libressl/default.nix

@@ -11,6 +11,8 @@ let
      inherit sha256;
    };

+    configureFlags = [ "--enable-nc" ];
+
    enableParallelBuilding = true;

    outputs = [ "bin" "dev" "out" "man" ];


Maybe a separate output for netcat? It feels weird to me that if I install netcat I also get a ssl library.

Yes i can look into it. My hope was that it would be small enough to not matter though.

The problem with somethig like this is that it’s not obvious what is in the netcat output. For instance should nc.1 go in man or netcat?

Everything that belongs to netcat should also go there.

@Mic92

I still feel weird about doing this because it seems a little hacky but this was requested by @Mic92 and seems understandable to not want to mix up libressl outputs with netcat stuff.

aszlig · 2018-04-29T21:36:39Z

@matthewbauer: Be sure to note this in the release notes, because this is incompatible with openbsd-netcat. Most options are compatible, but for example there is no -q option in the version from libressl. The latter already breaks our hibernate test, so this should have a release note entry so that people are aware of this.

In the meantime, I'm going to fix the hibernate test...

matthewbauer · 2018-04-29T21:40:05Z

Ok will do! They looked identical when I tried them but thanks for catching that.

matthewbauer · 2018-04-29T21:41:34Z

Actually if it does really need it, we can probably still apply this patch (with minor modifications):

https://sources.debian.org/patches/netcat-openbsd/1.105-7/0006-quit-timer.patch/

Which enables the -q flag. We would definitely want to upstream it to OpenBSD. I strongly dislike Debian's tendency of "patch hording".

@matthewbauer

I'm not sure why 024b501 used -q 0 because even netcat-openbsd has the -N flag which IMO is the better way to shutdown the socket on EOF. Our default netcat implementation has changed once again[1] in 3c3b822 and we're now using LibreSSL's implementation, which doesn't have a -q flag. See #39634 for the pull request introducing the switch. [1]: #19982 Signed-off-by: aszlig <aszlig@nix.build> Cc: @matthewbauer, @dtzWill, @Mic92

matthewbauer · 2018-04-29T22:53:09Z

@aszlig Are the Nixpkgs release notes no longer used? I can definitely put the release note in nixos/doc/manual/rl-1809.xml just want to make sure that is the right place.

aszlig · 2018-04-29T23:12:40Z

@matthewbauer: This is mainly a NixOS issue anyway, because netcat-openbsd is part of NixOS by default so scripts and services that depend on it might break due to the change. As for the release notes in general... we don't have a real nixpkgs release, only NixOS releases which ship a certain snapshot of nixpkgs with it.

@matthewbauer

I'm not sure why 024b501 used -q 0 because even netcat-openbsd has the -N flag which IMO is the better way to shutdown the socket on EOF. Our default netcat implementation has changed once again[1] in 3c3b822 and we're now using LibreSSL's implementation, which doesn't have a -q flag. See NixOS#39634 for the pull request introducing the switch. [1]: NixOS#19982 Signed-off-by: aszlig <aszlig@nix.build> Cc: @matthewbauer, @dtzWill, @Mic92

linas · 2019-05-09T21:46:26Z

FYI, we tripped over the missing -q flag in netcat here: opencog/opencog-nix#28

dudebout · 2020-01-01T19:21:14Z

Note that this makes netcat hard to install from nix-env because it is provided in the nc output which is not part of outputsToInstall. A workaround is to use an overlay like:

self: super:
{
    netcat = super.libressl.overrideAttrs (oldAttrs: {
    meta = oldAttrs.meta // { outputsToInstall = [ "nc" ]; };
  });
}

dudebout · 2020-01-02T01:30:41Z

Alternatively define a separate derivation for netcat-libressl as follows:

  # Hack around nix-env's inability to install outputs that are not in
  # outputsToInstall: nc(1) is in the "nc" output of the libressl package, but
  # "nc" is not in the outputsToInstall
  stdenv.mkDerivation {
    pname = "netcat-libressl";
    version = libressl.version;
    unpackPhase = "true";
    installPhase = ''
      mkdir $out
      cp -r ${libressl.nc}/* $out
    '';
  }

This makes it clearer which version of netcat it actually is, and where netcat is coming from when running nix-env -q.

sebastianblunt · 2022-04-16T07:07:22Z

FWIW #101601 - the one from debian appears to be more portable.

matthewbauer added 3 commits April 27, 2018 19:32

netcat-gnu: move to pkgs/tools/networking/netcat

99b8d10

libressl: build netcat

949bb98

netcat-openbsd: remove

3c3b822

GrahamcOfBorg added 8.has: clean-up 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100 labels Apr 28, 2018

Mic92 reviewed Apr 28, 2018

View reviewed changes

libressl: move netcat stuff to '.nc' output

c214848

I still feel weird about doing this because it seems a little hacky but this was requested by @Mic92 and seems understandable to not want to mix up libressl outputs with netcat stuff.

matthewbauer merged commit 6583ec5 into NixOS:master Apr 29, 2018

Radivarig mentioned this pull request May 5, 2019

[opencog] ShellUTest test fails (nc: invalid option -- 'q') opencog/opencog-nix#28

Closed

dudebout mentioned this pull request Jan 1, 2020

document nix-env bug relating to multiple output installation #76794

Merged

Artturin mentioned this pull request May 8, 2022

netcat-openbsd: init at 1.218-5 #171953

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace netcat-openbsd with libressl #39634

Replace netcat-openbsd with libressl #39634

matthewbauer commented Apr 28, 2018 •

edited

GrahamcOfBorg commented Apr 28, 2018

GrahamcOfBorg commented Apr 28, 2018

GrahamcOfBorg commented Apr 28, 2018

dtzWill commented Apr 28, 2018

Mic92 Apr 28, 2018

matthewbauer Apr 28, 2018

matthewbauer Apr 28, 2018

Mic92 Apr 28, 2018

aszlig commented Apr 29, 2018

matthewbauer commented Apr 29, 2018

matthewbauer commented Apr 29, 2018 •

edited

matthewbauer commented Apr 29, 2018

aszlig commented Apr 29, 2018

linas commented May 9, 2019

dudebout commented Jan 1, 2020

dudebout commented Jan 2, 2020

sebastianblunt commented Apr 16, 2022

Replace netcat-openbsd with libressl #39634

Replace netcat-openbsd with libressl #39634

Conversation

matthewbauer commented Apr 28, 2018 • edited

Motivation for this change

GrahamcOfBorg commented Apr 28, 2018

GrahamcOfBorg commented Apr 28, 2018

GrahamcOfBorg commented Apr 28, 2018

dtzWill commented Apr 28, 2018

Mic92 Apr 28, 2018

Choose a reason for hiding this comment

matthewbauer Apr 28, 2018

Choose a reason for hiding this comment

matthewbauer Apr 28, 2018

Choose a reason for hiding this comment

Mic92 Apr 28, 2018

Choose a reason for hiding this comment

aszlig commented Apr 29, 2018

matthewbauer commented Apr 29, 2018

matthewbauer commented Apr 29, 2018 • edited

matthewbauer commented Apr 29, 2018

aszlig commented Apr 29, 2018

linas commented May 9, 2019

dudebout commented Jan 1, 2020

dudebout commented Jan 2, 2020

sebastianblunt commented Apr 16, 2022

matthewbauer commented Apr 28, 2018 •

edited

matthewbauer commented Apr 29, 2018 •

edited