yggdrasil: init at 0.3.8 and add service module #63863

gazally · 2019-06-27T17:06:01Z

Motivation for this change

Use an encrypted ipv6 overlay network.

Things done

See also #63855 by @Lassulus, which has a different derivation for yggdrasil which also works with the service module. This is my first attempt to write a Go derivation so I'll be happy to rebase this pull request on #63855 if that is preferable.

nixos/modules/services/networking/yggdrasil.nix

pkgs/tools/networking/yggdrasil/default.nix

JohnAZoidberg · 2019-06-27T18:44:11Z

Make sure to use the email address that is linked to your GitHub account for the commits, then they're linked to your profile account.

pkgs/tools/networking/yggdrasil/default.nix

kalbasit · 2019-06-27T21:16:19Z

pkgs/tools/networking/yggdrasil/default.nix

+    homepage = "https://yggdrasil-network.github.io/";
+    license = licenses.lgpl3;
+    maintainers = with maintainers; [ gazally ];
+    platforms = platforms.linux;


Suggested change

platforms = platforms.linux;

I've just force pushed a commit which I think addresses all your issues with the package. Please take a look. I'm still working on the service module.

nixos/modules/services/networking/yggdrasil.nix

gazally · 2019-07-05T17:01:12Z

I just pushed a new version which applies many of systemd's sandboxing features to the yggdrasil service. I've got this running on 4 computers in 2 locations including one Raspberry Pi (aarch64) and have verified that local multicast peering and remote peering both work.

aanderse · 2019-07-07T00:33:01Z

Great work! Any chance for a NixOS test?

gazally · 2019-07-07T17:15:50Z

Any chance for a NixOS test?

I also think it's a good idea, so I'll find myself a Perl tutorial and see what I can do.

Also, I've decided that it would be better to have service.yggdrasil.config be a Nix attribute set, and then use builtins.toJSON to convert it into a config file for yggdrasil, so I'll be making that change as well unless anyone objects.

gazally · 2019-07-08T14:31:50Z

In this latest push, I have:

Added a test.
Made services.yggdrasil.config into a Nix attribute set.
Added a new option services.yggdrasil.denyDhcpcdInterfaces so dhcpcd broadcasts can be suppressed on TAP interfaces.
Added an assertion on config.networking.enableIPv6.

arcnmx

Nice! I'm looking forward to using this to replace my current ad-hoc package and module. Leaving a few comments aiming to avoid preStart where it's unnecessary and keep the paths used a bit less hard-coded, mostly just because I'm often interested in running multiple yggdrasil services at once.

pkgs/tools/networking/yggdrasil/default.nix

arcnmx · 2019-07-30T16:49:10Z

pkgs/tools/networking/yggdrasil/default.nix

+     -ldflags=
+         -X github.com/yggdrasil-network/yggdrasil-go/src/yggdrasil.buildVersion=${version}
+         -X github.com/yggdrasil-network/yggdrasil-go/src/yggdrasil.buildName=${pname}
+         -s -w


(I know nothing about the go build system but) Should these flags be something included in buildGoModule instead? nixpkgs typically produces stripped binaries (excluding the dontStrip and keepDebugInfo opt-outs) so unless those flags tend to cause widespread problems it may be worth just doing this for all go packages.

@kalbasit I think this is a question for you.

I put those flags in ldflags to make it match what yggdrasil's build script does. I'm also a go build system newbie so I don't know if the way they have set up their build is unusual or not.

nixos/modules/services/networking/yggdrasil.nix

arcnmx · 2019-07-30T17:22:55Z

nixos/modules/services/networking/yggdrasil.nix

+      '';
+
+      serviceConfig = {
+        ExecStart = "${cfg.package}/bin/yggdrasil -useconffile /etc/yggdrasil.conf";


Suggested change

ExecStart = "${cfg.package}/bin/yggdrasil -useconffile /etc/yggdrasil.conf";

ExecStart = "${cfg.package}/bin/yggdrasil -useconffile \${RUNTIME_DIRECTORY}/yggdrasil.conf";

I think? Though this can just refer to the config path directly unless it needs to be merged by preStart.

nixos/modules/services/networking/yggdrasil.nix

arcnmx · 2019-07-30T17:56:47Z

nixos/modules/services/networking/yggdrasil.nix

+    networking.firewall.allowedUDPPorts = mkIf cfg.openMulticastPort [ 9001 ];
+
+    # Make yggdrasilctl available on the command line.
+    environment.systemPackages = [ cfg.package ];


Okay so this one's just a personal wish and probably too opinionated but it'd be neat if we could get this to produce multiple outputs somehow, considering only yggdrasilctl really needs to be in the environment. They could also just be two separate derivations sharing the same source, not sure what's typical/standard for nixpkgs...

One reason to have yggdrasil in the environment is that yggdrasil -genconf and yggdrasil -genconf -json are useful commands to have available when you are working on your yggdrasil configuration.

Yup that's the main reason to run it from outside of the service, though that seems more what nix run nixpkgs.yggdrasil is for rather than making a one-time command a permanent part of the user environment - also ideally the nixos module documents all the settings so you wouldn't even have to do that!

Lassulus · 2019-08-11T22:20:50Z

so whats the state of this? should we merge this now and work on the multiple instance stuff later? or does someone wants to implement it now?

aanderse · 2019-08-11T22:24:59Z

Would the multiple instance stuff cause breaking api changes?

arcnmx · 2019-08-12T00:51:36Z

I wouldn't bother treating my comments as anything other than discussion points; I'm particularly interested due to how I use yggdrasil but that's not really relevant - and I can of course always continue to use a stripped-down module. The main areas I pointed out were paths that were a bit strangely hard-coded or inefficient/unnecessary, but I don't have a good understanding of nixos conventions or style regarding whether any of that actually matters (e.g. is polluting /etc bad or do plenty of other modules do it too?)

Also, what was the reason for meta.platforms being removed?

gazally · 2019-08-12T21:02:07Z

Upstream released 0.3.6 last week and I am hoping to find time this week to update the package.

I don't have a good understanding of nixos conventions or style regarding whether any of that actually matters (e.g. is polluting /etc bad or do plenty of other modules do it too?)

I'm in the same boat. I observed that other NixOS modules put things in /etc and did so too.

Also, what was the reason for meta.platforms being removed?

That was an accident. Thanks for catching it.

gazally · 2019-08-15T19:29:39Z

yggdrasil 0.3.5 and 0.3.6 now do not build due to a broken dependency, and upstream has marked it wontfix. They're recommending use of their develop branch or waiting for 0.3.7. Before that happened I was running into #64346 while trying to test the package on my mix of machines running 19.03 and unstable, so I guess I'll wait a bit.

gazally · 2019-09-22T16:38:39Z

I've just pushed an update of the package to the latest version, 0.3.8. There is now an additional commit which removes DynamicUser from the service module. Yggdrasil works with DynamicUser set on systemd 242, but fails to open its network adapter if DynamicUser is set on systemd 243, and systemd 243 is now in Nixpkgs unstable. I have reported this issue to yggdrasil upstream.

gazally · 2019-09-23T18:00:18Z

yggdrasil 0.3.8 does not build on Go 1.13, which is pull request #68135. This will be fixed in yggdrasil 0.3.9.

Lassulus · 2019-10-16T14:00:45Z

soo, I guess we can merge this now and fix the building on go1.13 later?

arcnmx · 2019-10-16T15:02:49Z

yggdrasil 0.3.10 is available now, and presumably includes the go 1.13 fix.

aanderse · 2019-10-26T11:45:59Z

maintainers/maintainer-list.nix

@@ -2350,6 +2350,11 @@
    github = "gavinrogers";
    name = "Gavin Rogers";
  };
+  gazally = {


You need to add your github id to this as part of recent requirements.

ehmry · 2019-10-26T11:53:57Z

Merged to master, but githubId is still missing.

gazally requested a review from infinisil as a code owner June 27, 2019 17:06

ofborg bot added 6.topic: nixos 8.has: module (update) labels Jun 27, 2019

gazally force-pushed the yggdrasil branch from 59615ce to 1c69a8b Compare June 27, 2019 17:38

Lassulus reviewed Jun 27, 2019

View reviewed changes

nixos/modules/services/networking/yggdrasil.nix Outdated Show resolved Hide resolved

Lassulus reviewed Jun 27, 2019

View reviewed changes

nixos/modules/services/networking/yggdrasil.nix Show resolved Hide resolved

Lassulus reviewed Jun 27, 2019

View reviewed changes

nixos/modules/services/networking/yggdrasil.nix Outdated Show resolved Hide resolved

ofborg bot added the 8.has: package (new) label Jun 27, 2019

ofborg bot requested a review from kalbasit June 27, 2019 17:54

ofborg bot added 11.by: package-maintainer 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Jun 27, 2019

Lassulus reviewed Jun 27, 2019

View reviewed changes

nixos/modules/services/networking/yggdrasil.nix Show resolved Hide resolved

Lassulus reviewed Jun 27, 2019

View reviewed changes

pkgs/tools/networking/yggdrasil/default.nix Outdated Show resolved Hide resolved

kalbasit requested changes Jun 27, 2019

View reviewed changes

aanderse reviewed Jun 30, 2019

View reviewed changes

nixos/modules/services/networking/yggdrasil.nix Outdated Show resolved Hide resolved

nixos/modules/services/networking/yggdrasil.nix Outdated Show resolved Hide resolved

gazally force-pushed the yggdrasil branch from 1c69a8b to 776c460 Compare June 30, 2019 15:06

ofborg bot requested review from Lassulus and kalbasit June 30, 2019 15:17

ofborg bot added 10.rebuild-darwin: 1-10 and removed 10.rebuild-darwin: 0 labels Jun 30, 2019

Lassulus mentioned this pull request Jul 1, 2019

yggdrasil: init at 0.3.5 #63855

Closed

10 tasks

gazally force-pushed the yggdrasil branch from 776c460 to ca7e2ca Compare July 5, 2019 15:55

gazally force-pushed the yggdrasil branch from ca7e2ca to 0925067 Compare July 5, 2019 17:45

gazally mentioned this pull request Jul 5, 2019

Default admin socket location and sandboxing yggdrasil with systemd yggdrasil-network/yggdrasil-go#449

Open

gazally mentioned this pull request Jul 7, 2019

nixos/systemd-confinement: Fix DynamicUser support #64405

Draft

10 tasks

gazally force-pushed the yggdrasil branch from 0925067 to f4b89e6 Compare July 8, 2019 14:24

arcnmx reviewed Jul 30, 2019

View reviewed changes

gazally force-pushed the yggdrasil branch from 1209ac8 to 3d43b41 Compare July 31, 2019 00:50

gazally added 4 commits September 21, 2019 16:43

maintainers: add gazally

42dd643

yggdrasil: init at 0.3.8

8dd6394

nixos/yggdrasil: add service

6235832

nixos/yggdrasil: remove DynamicUser for systemd 243

0092074

gazally force-pushed the yggdrasil branch from 3d43b41 to 0092074 Compare September 22, 2019 16:30

gazally changed the title ~~yggdrasil: init at 0.3.5 and add service module~~ yggdrasil: init at 0.3.8 and add service module Sep 22, 2019

ehmry self-assigned this Oct 26, 2019

ehmry approved these changes Oct 26, 2019

View reviewed changes

aanderse requested changes Oct 26, 2019

View reviewed changes

ehmry closed this Oct 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

yggdrasil: init at 0.3.8 and add service module #63863

yggdrasil: init at 0.3.8 and add service module #63863

gazally commented Jun 27, 2019

JohnAZoidberg commented Jun 27, 2019

kalbasit Jun 27, 2019

gazally Jun 30, 2019

gazally commented Jul 5, 2019

aanderse commented Jul 7, 2019

gazally commented Jul 7, 2019

gazally commented Jul 8, 2019

arcnmx left a comment

arcnmx Jul 30, 2019

gazally Jul 30, 2019

arcnmx Jul 30, 2019

arcnmx Jul 30, 2019

gazally Jul 30, 2019

arcnmx Jul 31, 2019

Lassulus commented Aug 11, 2019

aanderse commented Aug 11, 2019

arcnmx commented Aug 12, 2019

gazally commented Aug 12, 2019

gazally commented Aug 15, 2019

gazally commented Sep 22, 2019

gazally commented Sep 23, 2019

Lassulus commented Oct 16, 2019

arcnmx commented Oct 16, 2019

aanderse Oct 26, 2019

ehmry commented Oct 26, 2019

	ExecStart = "${cfg.package}/bin/yggdrasil -useconffile /etc/yggdrasil.conf";
	ExecStart = "${cfg.package}/bin/yggdrasil -useconffile \${RUNTIME_DIRECTORY}/yggdrasil.conf";

yggdrasil: init at 0.3.8 and add service module #63863

yggdrasil: init at 0.3.8 and add service module #63863

Conversation

gazally commented Jun 27, 2019

Motivation for this change

Things done

JohnAZoidberg commented Jun 27, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gazally commented Jul 5, 2019

aanderse commented Jul 7, 2019

gazally commented Jul 7, 2019

gazally commented Jul 8, 2019

arcnmx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lassulus commented Aug 11, 2019

aanderse commented Aug 11, 2019

arcnmx commented Aug 12, 2019

gazally commented Aug 12, 2019

gazally commented Aug 15, 2019

gazally commented Sep 22, 2019

gazally commented Sep 23, 2019

Lassulus commented Oct 16, 2019

arcnmx commented Oct 16, 2019

Choose a reason for hiding this comment

ehmry commented Oct 26, 2019