nixos/ceph: Add a service module that enables the configuration of ceph through nixos #35299

lejonet · 2018-02-21T22:13:23Z

All 5 daemon types can be enabled and configured through the module and the module both creates tho ceph.conf required but also creates and enables specific services for each daemon, based on the systemd service files that upstream provides.

Motivation for this change

In addition to the ceph package being broken, there isn't any module to configure a ceph cluster through nixos, which I'm aiming to fix.
I've successfully setup and am running a ceph cluster, on nixos, configured soley through this module.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

…ph through nixos All 5 daemon types can be enabled and configured through the module and the module both creates tho ceph.conf required but also creates and enables specific services for each daemon, based on the systemd service files that upstream provides.

…nality of both daemons and cluster

lejonet · 2018-02-26T19:11:07Z

@Mic92 I've now added a testcase, as requested

grahamc · 2018-02-26T19:45:51Z

Please add the test to https://github.com/NixOS/nixpkgs/blob/master/nixos/release.nix#L219

Mic92 · 2018-02-26T19:45:54Z

nixos/tests/ceph.nix

@@ -0,0 +1,147 @@
+{ system ? builtins.currentSystem }:
+
+with import ../lib/testing.nix { inherit system; };


Can you add yourself to maintainers? https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/mesos.nix#L3

I've added myself to the maintainers list, at L145, I should probably restructure the test after that mesos test, I just did it this way because I used networking.nix as a template.

grahamc · 2018-02-26T19:49:50Z

nixos/modules/services/network-filesystems/ceph.nix

+          [ "0" "1" ];
+        '';
+        description = ''
+          A list of rados gateway daemons that should have a service created.


A very minor nit is that "should have a service created" sounds imperative. Instead, these are describing not what should happen but what services will exist, right?

I actually had a hard time wording that and just decided to write something at the time and continue with the rest of the module, honestly.

I'm open to suggestions of better wording, the wording isn't just imperative, it sounds wonky.

lejonet · 2018-02-26T21:42:00Z

@grahamc Test added to the main list in release.nix: 4f91fc4

lejonet · 2018-02-28T20:28:32Z

@Mic92 I've now restructured the test to the structure of the mesos test with commit cc84fe5

Mic92 · 2018-03-01T00:04:59Z

@GrahamcOfBorg test ceph

GrahamcOfBorg · 2018-03-01T00:05:15Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

error: while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/release.nix:19:72, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/attrsets.nix:282:43:
while evaluating ‘hydraJob’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/customisation.nix:168:14, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/release.nix:19:80:
while evaluating the attribute ‘meta’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/customisation.nix:173:24:
while evaluating the attribute ‘maintainers’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/tests/ceph.nix:4:5:
undefined variable ‘lejonet’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/tests/ceph.nix:4:21

GrahamcOfBorg · 2018-03-01T00:05:19Z

Success on aarch64-linux (full log)

Partial log (click to expand)

Mic92 · 2018-03-01T00:05:26Z

Now ceph should be built already.

Mic92 · 2018-03-01T00:06:50Z

nixos/tests/ceph.nix

+import ./make-test.nix ({pkgs, ...}: rec {
+  name = "All-in-one-basic-ceph-cluster";
+  meta = with pkgs.stdenv.lib.maintainers; {
+    maintainers = [ lejonet ];


typo in the name?

Whatcha mean?

That handle (lejonet) does not exists in lib/maintainers.nix

Fixed: 29e78ce

Mic92 · 2018-03-01T00:08:27Z

nixos/modules/services/network-filesystems/ceph.nix

+      LimitNOFILE = 1048576;
+      LimitNPROC = 1048576;
+      Environment = "CLUSTER=${clusterName}";
+      ExecReload = "/bin/kill -HUP $MAINPID";


should be ${coreutils}/bin/kill -HUP $MAINPID

…he proper path

grahamc · 2018-03-01T13:33:52Z

Did we ever verify that the test passed?

Mic92 · 2018-03-01T13:49:47Z

@grahamc I only executed them locally.

Mic92 · 2018-03-01T13:50:47Z

@grahamc the build bot would have had a timeout because ceph was not yet build. I tested it on a slightly older head (older in hours).

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Feb 21, 2018

nixos/ceph: Add testcase for a simple ceph cluster, to verify functio…

473066a

…nality of both daemons and cluster

Mic92 reviewed Feb 26, 2018

View reviewed changes

grahamc reviewed Feb 26, 2018

View reviewed changes

lejonet added 2 commits February 26, 2018 21:02

nixos/ceph: Clarify the examples and descriptions

9382e32

nixos/ceph: Add ceph test to the main list of tests

4f91fc4

nixos/ceph: Restructure ceph test

cc84fe5

Mic92 reviewed Mar 1, 2018

View reviewed changes

nixos/ceph: Prepend /bin/kill with ${pkgs.coreutil} so that it gets t…

e71d500

…he proper path

lejonet requested review from edolstra, edwtjo, Ericson2314, FRidh, nbp, orivej, peti and rycee as code owners March 1, 2018 11:01

lejonet requested review from ttuegel and zimbatm as code owners March 1, 2018 11:01

Mic92 removed request for zimbatm, peti, edwtjo, orivej, ttuegel, rycee, Ericson2314, edolstra, nbp and FRidh March 1, 2018 11:14

lejonet added 2 commits March 1, 2018 12:19

nixos/ceph: Fix typo pkgs.coreutil -> pkgs.coreutils

8b089f8

maintainers: Add myself to maintainers file

29e78ce

Mic92 merged commit 565f22d into NixOS:master Mar 1, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/ceph: Add a service module that enables the configuration of ceph through nixos #35299

nixos/ceph: Add a service module that enables the configuration of ceph through nixos #35299

lejonet commented Feb 21, 2018 •

edited

lejonet commented Feb 26, 2018

grahamc commented Feb 26, 2018 •

edited

Mic92 Feb 26, 2018

lejonet Feb 26, 2018

grahamc Feb 26, 2018 •

edited

lejonet Feb 26, 2018

lejonet commented Feb 26, 2018

lejonet commented Feb 28, 2018

Mic92 commented Mar 1, 2018

GrahamcOfBorg commented Mar 1, 2018

GrahamcOfBorg commented Mar 1, 2018

Mic92 commented Mar 1, 2018

Mic92 Mar 1, 2018

lejonet Mar 1, 2018

Mic92 Mar 1, 2018 •

edited

lejonet Mar 1, 2018

Mic92 Mar 1, 2018

grahamc commented Mar 1, 2018

Mic92 commented Mar 1, 2018

Mic92 commented Mar 1, 2018

		@@ -0,0 +1,147 @@
		{ system ? builtins.currentSystem }:

		with import ../lib/testing.nix { inherit system; };

nixos/ceph: Add a service module that enables the configuration of ceph through nixos #35299

nixos/ceph: Add a service module that enables the configuration of ceph through nixos #35299

Conversation

lejonet commented Feb 21, 2018 • edited

Motivation for this change

Things done

lejonet commented Feb 26, 2018

grahamc commented Feb 26, 2018 • edited

Mic92 Feb 26, 2018

Choose a reason for hiding this comment

lejonet Feb 26, 2018

Choose a reason for hiding this comment

grahamc Feb 26, 2018 • edited

Choose a reason for hiding this comment

lejonet Feb 26, 2018

Choose a reason for hiding this comment

lejonet commented Feb 26, 2018

lejonet commented Feb 28, 2018

Mic92 commented Mar 1, 2018

GrahamcOfBorg commented Mar 1, 2018

GrahamcOfBorg commented Mar 1, 2018

Mic92 commented Mar 1, 2018

Mic92 Mar 1, 2018

Choose a reason for hiding this comment

lejonet Mar 1, 2018

Choose a reason for hiding this comment

Mic92 Mar 1, 2018 • edited

Choose a reason for hiding this comment

lejonet Mar 1, 2018

Choose a reason for hiding this comment

Mic92 Mar 1, 2018

Choose a reason for hiding this comment

grahamc commented Mar 1, 2018

Mic92 commented Mar 1, 2018

Mic92 commented Mar 1, 2018

lejonet commented Feb 21, 2018 •

edited

grahamc commented Feb 26, 2018 •

edited

grahamc Feb 26, 2018 •

edited

Mic92 Mar 1, 2018 •

edited