nixos/nfs: Allow Kerberized NFS #73989

kwohlfahrt · 2019-11-23T22:08:48Z

Motivation for this change

Currently, NFS mounts secured with Kerberos do not work (see #72722).

Things done

Notify maintainers

cc @tfc - regarding the Perl -> Python test port
cc @edolstra as maintainer of the previous nfs test

rnhmjoj

You have to update nixos/tests/all-tests.nix to build the nfs/ directory, instead of nfs.nix.

nixos/tests/nfs/kerberos.nix

rnhmjoj · 2019-11-23T23:01:41Z

nixos/tests/nfs/kerberos.nix

+      client.fail("echo bla >> /data/foo")
+
+      client.succeed("echo root_pw | kinit root", "echo bla >> /data/foo")
+      server.succeed("test -e /data/foo")


Maybe use wait_for_file here

test -e is what the plain NFS test uses, I assume because NFS should be synchronous here (i.e. by the time the file is visible on the client, it has been created on the server).

nixos/tests/all-tests.nix

kwohlfahrt · 2019-11-25T15:27:28Z

As noted in the issue, there are two remaining problems:

file uid/gids are all messed up. This can be fixed by running nfs-idmapd on the client, but docs suggest that this is no longer necessary as it is replaced by the nfsidmap binary. Indeed, the service is masked if services.nfs.server.enable = false. EDIT: This is nfs4 idmapd fails to operate without additional configuration (clone of #34638 for nfs4) #68106
It is not possible to write to files as root, even if the corresponding kerberos principal exists. This may be intended behavior.

nixos/tests/nfs/simple.nix

kwohlfahrt · 2019-11-25T16:28:53Z

I appear to have CI errors, but the message in details is not very clear to me: attribute 'x86_64-linux' missing, at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/eval-0-gleber.ewr1.nix.ci/nixos/release-combined.nix:50:43. How do I fix this?

kwohlfahrt · 2019-11-26T21:51:38Z

This PR has expanded in scope a bit - there is now a kernel patch to fix the hard-coded ID request-key path. This touches a few other issues, namely #68106 and #34638, though those still need a bit of module work to ensure keyutils is present for example.

tfc · 2019-11-27T07:41:36Z

nixos/tests/nfs/kerberos.nix

+      server.succeed("mkdir -p /data/alice")
+      server.succeed("chown alice:users /data/alice")
+
+      # set up kerberos database


the comments here that tell what happens on a higher level could all be subtests, so this information even appears in the log.

Oh sorry i already stated this in another comment in the same PR and didn't remember.

I've changed most of the other ones. Here, this is really set-up code that is not related to the functionality under test. Is subtest intended for this use as well?

rnhmjoj · 2019-11-27T08:39:30Z

This PR has expanded in scope a bit - there is now a kernel patch to fix the hard-coded ID request-key path. This touches a few other issues, namely #68106 and #34638, though those still need a bit of module work to ensure keyutils is present for example.

I had no idea this could get this complicated. Thanks for all the work you've done, keep it up!

rnhmjoj · 2019-11-27T09:06:26Z

I appear to have CI errors, but the message in details is not very clear to me: attribute 'x86_64-linux' missing, at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/eval-0-gleber.ewr1.nix.ci/nixos/release-combined.nix:50:43. How do I fix this?

I can't try right now because I don't have nearly ram to run an evaluation but you could try to rebase your branch with current master. Maybe at the point you did a checkout master itself was broken.

ofBorg runs something like this

$ nix-instantiate nixos/release-combined.nix -A tested

kwohlfahrt · 2019-11-27T11:13:47Z

I figured out the build failure - I copied the test structure from the kerberos tests, which I wrote last year. It turns out they were never enabled in release-combined.nix, because I didn't know this was a necessary step, and the way I'd been calling them doesn't work.

kwohlfahrt · 2019-11-27T15:09:16Z

All of the tests here are OK now. Remaining question are:

How should /etc/request-key.conf be generated? The issues linked above require different lines in this file.
should keyutils be part of the default systemPackages?

rnhmjoj · 2019-11-27T15:30:18Z

should keyutils be part of the default systemPackages?

If it's only needed for kerberos I would say to install it behind a guard like krb5.enable.

kwohlfahrt · 2019-11-27T15:47:14Z

@rnhmjoj - no, its needed for id-mapping in general. This is not turned on by default for sec=sys (i.e. not kerberos), but such a configuration is possible, see here. I assume this is the reason for #68106 as it doesn't mention kerberos anywhere.

It is also needed for CIFS, at least with kerberos, see #34638. I'm not familiar enough with it to know if this can also be necessary without kerberos - the man page doesn't mention it so it seems likely.

We can test for the presence of a NFS/CIFS filesystem before installing it, which seems sensible.

Mic92 · 2019-11-28T09:40:38Z

nixos/tests/nfs/kerberos.nix

+          "systemctl restart kadmind.service kdc.service",
+      )
+      for unit in ["kadmind", "kdc"]:
+          server.wait_for_unit(f"{unit}.service")


nitpick: same lines of code / easier to read:

Suggested change

server.wait_for_unit(f"{unit}.service")

server.wait_for_unit("kadmind.service")

server.wait_for_unit("kdc.service")

Nits picked. Anything left blocking merging?

Necessary for nfs server with kerberos auth

This is necessary for id mapping to work with NFS + Kerberos, and also touches NixOS#68106 and 634638.

A patch is necessary upstream to support multiple configs via symlinks in /etc/request-key.d Once that is done, we can add support for CIFS as well

Mic92 · 2019-12-13T10:21:59Z

@GrahamcOfBorg test nfs

Mic92 · 2019-12-13T10:23:08Z

@GrahamcOfBorg test nfs.simple nfs.kerberos

Mic92 · 2019-12-13T10:25:41Z

@GrahamcOfBorg test nfs3.simple nfs4.simple nfs4.kerberos

kwohlfahrt mentioned this pull request Nov 23, 2019

NFS4 server with Kerberos does not work #72722

Closed

ofborg bot added the 6.topic: nixos label Nov 23, 2019

rnhmjoj requested changes Nov 24, 2019

View reviewed changes

kwohlfahrt force-pushed the nfs-tests-python branch 2 times, most recently from db4977f to 52f6a9e Compare November 24, 2019 14:44

rnhmjoj reviewed Nov 24, 2019

View reviewed changes

nixos/tests/all-tests.nix Outdated Show resolved Hide resolved

kwohlfahrt force-pushed the nfs-tests-python branch 3 times, most recently from dd33065 to 913a10b Compare November 25, 2019 15:20

kwohlfahrt changed the title ~~[WIP] Allow Kerberized NFS~~ nixos/nfs: Allow Kerberized NFS Nov 25, 2019

kwohlfahrt marked this pull request as ready for review November 25, 2019 15:57

tfc reviewed Nov 25, 2019

View reviewed changes

nixos/tests/nfs/simple.nix Outdated Show resolved Hide resolved

kwohlfahrt force-pushed the nfs-tests-python branch from 913a10b to 62856e8 Compare November 26, 2019 20:13

tfc reviewed Nov 27, 2019

View reviewed changes

kwohlfahrt force-pushed the nfs-tests-python branch from e7b8e45 to 7a6ec9e Compare November 27, 2019 14:59

ofborg bot requested a review from abbradar November 27, 2019 15:16

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 501+ 10.rebuild-linux: 1001-2500 labels Nov 27, 2019

kwohlfahrt force-pushed the nfs-tests-python branch from 8f8e04d to 6ea57ed Compare November 27, 2019 21:38

kwohlfahrt requested a review from alyssais as a code owner November 27, 2019 21:38

ofborg bot removed 6.topic: python 6.topic: qt/kde 6.topic: ruby 8.has: documentation 2.status: merge conflict labels Nov 27, 2019

ofborg bot requested review from andir and lovek323 November 27, 2019 22:10

ofborg bot added 10.rebuild-linux: 5001+ and removed 10.rebuild-linux: 1001-2500 labels Nov 27, 2019

alyssais removed their request for review November 28, 2019 09:05

Mic92 reviewed Nov 28, 2019

View reviewed changes

FRidh added this to Needs review in Staging Nov 30, 2019

kwohlfahrt added 3 commits December 12, 2019 12:23

nixos/nfs: convert test to python

59a0e58

nfs-utils: build svcgssd

7b83883

Necessary for nfs server with kerberos auth

linux: patch request-key binary path

ea55a2d

This is necessary for id mapping to work with NFS + Kerberos, and also touches NixOS#68106 and 634638.

kwohlfahrt force-pushed the nfs-tests-python branch from 6ea57ed to d17c834 Compare December 12, 2019 15:34

kwohlfahrt added 3 commits December 12, 2019 15:42

nixos/nfs: test nfs with kerberos authentication

0dce66a

nfs: set up request-key for id mapping

b1c10bc

A patch is necessary upstream to support multiple configs via symlinks in /etc/request-key.d Once that is done, we can add support for CIFS as well

keyutils: patch to allow symlinks in config dir

e0b9b5f

kwohlfahrt force-pushed the nfs-tests-python branch from d17c834 to e0b9b5f Compare December 12, 2019 15:42

nfs-utils: link nfs tests

e71bd33

Mic92 merged commit 1c329c9 into NixOS:staging Dec 13, 2019

Staging automation moved this from Needs review to Done Dec 13, 2019

tfc mentioned this pull request Dec 13, 2019

nixos/tests: Leftover tasks towards unified Python integration tests #72828

Closed

kwohlfahrt mentioned this pull request Dec 14, 2019

nfs4 idmapd fails to operate without additional configuration (clone of #34638 for nfs4) #68106

Closed

tbenst mentioned this pull request Dec 17, 2019

cifs-utils mount.cifs fails if using kerberos auth #34638

Open

kwohlfahrt deleted the nfs-tests-python branch December 18, 2019 10:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/nfs: Allow Kerberized NFS #73989

nixos/nfs: Allow Kerberized NFS #73989

kwohlfahrt commented Nov 23, 2019 •

edited

rnhmjoj left a comment

rnhmjoj Nov 23, 2019

kwohlfahrt Nov 24, 2019

kwohlfahrt commented Nov 25, 2019 •

edited

kwohlfahrt commented Nov 25, 2019

kwohlfahrt commented Nov 26, 2019

tfc Nov 27, 2019

tfc Nov 27, 2019

kwohlfahrt Nov 27, 2019

rnhmjoj commented Nov 27, 2019

rnhmjoj commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

rnhmjoj commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

Mic92 Nov 28, 2019

kwohlfahrt Dec 12, 2019

Mic92 commented Dec 13, 2019

Mic92 commented Dec 13, 2019

Mic92 commented Dec 13, 2019

	server.wait_for_unit(f"{unit}.service")
	server.wait_for_unit("kadmind.service")
	server.wait_for_unit("kdc.service")

nixos/nfs: Allow Kerberized NFS #73989

nixos/nfs: Allow Kerberized NFS #73989

Conversation

kwohlfahrt commented Nov 23, 2019 • edited

Motivation for this change

Things done

Notify maintainers

rnhmjoj left a comment

Choose a reason for hiding this comment

rnhmjoj Nov 23, 2019

Choose a reason for hiding this comment

kwohlfahrt Nov 24, 2019

Choose a reason for hiding this comment

kwohlfahrt commented Nov 25, 2019 • edited

kwohlfahrt commented Nov 25, 2019

kwohlfahrt commented Nov 26, 2019

tfc Nov 27, 2019

Choose a reason for hiding this comment

tfc Nov 27, 2019

Choose a reason for hiding this comment

kwohlfahrt Nov 27, 2019

Choose a reason for hiding this comment

rnhmjoj commented Nov 27, 2019

rnhmjoj commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

rnhmjoj commented Nov 27, 2019

kwohlfahrt commented Nov 27, 2019

Mic92 Nov 28, 2019

Choose a reason for hiding this comment

kwohlfahrt Dec 12, 2019

Choose a reason for hiding this comment

Mic92 commented Dec 13, 2019

Mic92 commented Dec 13, 2019

Mic92 commented Dec 13, 2019

kwohlfahrt commented Nov 23, 2019 •

edited

kwohlfahrt commented Nov 25, 2019 •

edited