nextcloud module: init #44994

globin · 2018-08-13T20:30:45Z

Motivation for this change

Nextcloud module

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- N/A macOS
- N/A other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
N/A Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
N/A Tested execution of all binary files (usually in ./result/bin/)
N/A Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

GrahamcOfBorg · 2018-08-13T20:45:13Z

Success on aarch64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/0102ygq93pk2w3clb57f4rdva2qh1xzd-nextcloud-13.0.5
copying path '/nix/store/0102ygq93pk2w3clb57f4rdva2qh1xzd-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/0102ygq93pk2w3clb57f4rdva2qh1xzd-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-13T20:45:29Z

Success on x86_64-darwin (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5
copying path '/nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-13T20:45:47Z

Success on x86_64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5
copying path '/nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5

Ma27 · 2018-08-13T23:28:39Z

I originally checked @fpletz's commit in July using our fork as base (mayflower@c6bc13c#commitcomment-29652614).

I'll recheck with the NixOS fork and give feedback about this %)

Ma27 · 2018-08-13T23:37:45Z

the test seems to fail as it's missing the following commit: mayflower@dc1158a

The commit doesn't apply automatically onto NixOS master due to minor changes in the Nginx module. I rebased it in my fork and filed a PR here: mayflower#68

Locations need an order because nginx will try to match them in the order they appear in the config file. This reuses the sortProperties function that is also used by the module system for `mkOrder` uses.

globin · 2018-08-14T11:37:33Z

@GrahamcOfBorg test nextcloud

GrahamcOfBorg · 2018-08-14T11:40:43Z

Success on x86_64-linux (full log)

Attempted: tests.nextcloud

Partial log (click to expand)

nextcloud: exit status 1
syncing
nextcloud: running command: sync
nextcloud: exit status 0
test script finished in 59.01s
cleaning up
killing nextcloud (pid 600)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/x8iwvhc8k8vp2mqwlz8q5dyyd5kvl9kb-vm-test-run-nextcloud

GrahamcOfBorg · 2018-08-14T11:43:43Z

Failure on aarch64-linux (full log)

Attempted: tests.nextcloud

Partial log (click to expand)

nextcloud: exit status 22
nextcloud: output:
error: command `curl -sSf http://nextcloud/login' did not succeed (exit code 22)
command `curl -sSf http://nextcloud/login' did not succeed (exit code 22)
cleaning up
killing nextcloud (pid 631)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/3hhral9yc1mia7miz2m8497cppa4bb9m-vm-test-run-nextcloud.drv' failed with exit code 255
error: build of '/nix/store/3hhral9yc1mia7miz2m8497cppa4bb9m-vm-test-run-nextcloud.drv' failed

GrahamcOfBorg · 2018-08-14T11:50:04Z

Success on x86_64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5
copying path '/nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5' from 'https://nix-cache.s3.amazonaws.com'...
/nix/store/5bsf1qkh1g9rrj9r5n97fkkrb52xi403-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-14T11:50:15Z

Success on aarch64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

/nix/store/0102ygq93pk2w3clb57f4rdva2qh1xzd-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-14T11:50:30Z

Success on x86_64-darwin (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5
copying path '/nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/x086pkxpwcs7r0c6dapbjpfva5rna5xq-nextcloud-13.0.5

flokli · 2018-08-15T11:47:07Z

There's also #44839, which seems to have similar functionality. Could you have a look?

Closes NixOS#44839 Co-authored-by: Janne Heß <janne@hess.ooo>

globin · 2018-08-19T22:37:16Z

Incorporated the improvements in #44839 into this PR (added @dasJ as co-author, too for those changes), I think this should be good to go

GrahamcOfBorg · 2018-08-19T22:50:56Z

Success on x86_64-darwin (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/3s9z2fh2xbp1qbg7va6zxzb4d1ilz2hm-nextcloud-13.0.5
copying path '/nix/store/3s9z2fh2xbp1qbg7va6zxzb4d1ilz2hm-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/3s9z2fh2xbp1qbg7va6zxzb4d1ilz2hm-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-19T22:51:00Z

Success on x86_64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/h4zp69vnw716zvz8d82370pmcv26kpa8-nextcloud-13.0.5
copying path '/nix/store/h4zp69vnw716zvz8d82370pmcv26kpa8-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/h4zp69vnw716zvz8d82370pmcv26kpa8-nextcloud-13.0.5

GrahamcOfBorg · 2018-08-19T22:51:06Z

Success on aarch64-linux (full log)

Attempted: nextcloud

Partial log (click to expand)

these paths will be fetched (34.80 MiB download, 155.64 MiB unpacked):
  /nix/store/yk8ndxvgzy84qigff9dgh9i812hf4w73-nextcloud-13.0.5
copying path '/nix/store/yk8ndxvgzy84qigff9dgh9i812hf4w73-nextcloud-13.0.5' from 'https://cache.nixos.org'...
/nix/store/yk8ndxvgzy84qigff9dgh9i812hf4w73-nextcloud-13.0.5

dasJ · 2018-08-20T18:44:49Z

Good to go, I'll close my PR once this is merged

flokli · 2018-08-22T14:13:04Z

@globin I gave this a try, configured nginx to only serve nextcloud on localhost:8080, with a traefik in front listening on ports 80 and 443.

Was able to get some nextcloud web interface, it however complains about "access through untrusted domain". According to nextclouds forum, trusted domains are something that can be set from configuration.

Would it make sense to expose that as an option too, defaulting to

[ options.services.nextcloud.hostName ]

if unset?

flokli · 2018-08-23T10:57:56Z

Looks like nextcloud writes its own list of trusted domains on first request to that configuration file.
After removing the configuration file and switching configuration again, it happily picked up the domain I was accessing it from and added it to the configuration file.

Does nextcloud necessary needs to be able to write that configuration on it's own, or is that something we could explicitly configure with Nix, too? Would look a bit cleaner to me if possible…

globin · 2018-08-24T14:16:00Z

We already write the autoconfig.php and override.config.php for that, and as you have noticed nextcloud modifies the config.php at various stages including initial config. IMHO how it is implemented now is the best we can get from combining static configuration from the nixos for overrides and initial config and the dynamic configuration approach nextcloud includes.

globin · 2018-08-25T08:54:17Z

If there aren't any further comments I'm going to merge this tomorrow if nobody else beats me to it.

flokli · 2018-08-25T14:34:09Z

nixos/modules/services/web-apps/nextcloud.nix

+      dbtableprefix = mkOption {
+        type = types.str;
+        default = "";
+        description = "Table Prefix in Nextlcoud Database";


flokli · 2018-08-25T14:37:02Z

@globin did you get it to work with mysql or postgresql database?

I do get SQLSTATE[HY000] [2002] No such file or directory, when configuring mysql, suggesting php is unable to find the socket file.

Also didn't test caching, webfinger and file uploads.

globin · 2018-08-26T00:47:47Z

I'm using it with similar config to this on two machines (redacted):

  services.nextcloud = {
    enable = true;
    hostName = "nextcloud.example.com";
    nginx.enable = true;
    https = true;
    autoconfig = {
      dbtype = "pgsql";
      dbname = "nextcloud";
      dbuser = "nextcloud";
      dbpass = "password";
      dbhost = "localhost";
      adminlogin = "admin";
      adminpass = "password";
    };
  };
  services.postgresql = {
      enable = true;
      initialScript = pkgs.writeText "psql-init" ''
        CREATE ROLE nextcloud WITH LOGIN PASSWORD 'password';
        CREATE DATABASE nextcloud WITH OWNER nextcloud;
      '';
    };

flokli · 2018-08-26T05:13:19Z

That's my config:

          services.nextcloud = {
            enable = true;
            https = true;
            nginx.enable = true;
            hostName = "nextcloud.example.com";
            autoconfig= {
              adminpass = "password";
              dbtype = "mysql";
              dbname = "nextcloud";
              dbuser = "root";
              dbpass = "";
              dbhost = "localhost";
            };
          };
          services.mysql = {
            enable = true;
            ensureDatabases = ["nextcloud"];
            ensureUsers = [{ ensurePermissions = { "nextcloud.*" = "ALL PRIVILEGES"; }; name = "nextcloud"; }];
            package = pkgs.mysql;
          };
          services.nginx.virtualHosts."nextcloud.example.com".listen = [
            { addr = "127.0.0.1"; port = 8080;  ssl = false; }
          ];

For some reason, The nextcloud user isn't able to login (using the mysql cli), but root works from there, and mysql listens both on *:3306 and /var/run/mysqld/mysqld.sock (which is my I configured the root user for nextcloud for now). However, the error message from PHP suggests something with the PHP configuration / loaded modules (?) is wrong.

Ma27

I just started to set this up on my private server and I found some minor nitpicks worth considering. I wanted to discuss about this first, but if you agree with my comments I'd volunteer to implement it myself.

Furthermore I think that some documentation might be helpful (@globin's comment about how to setup nextcloud with postgres might have saved me some time). I'd volunteer for this as well, but I'd do this in a separate PR as I'd love to see this merged soon and for writing docs for the NixOS manual takes some time IMHO.

Ma27 · 2018-08-31T23:00:56Z

nixos/modules/services/web-apps/nextcloud.nix

+    autoconfig = {
+      dbtype = mkOption {
+        type = types.str;
+        default = "sqlite";


how about using an enum here? This is especiall helpful when viewing the options in man configuration.nix.

Ma27 · 2018-08-31T23:02:26Z

nixos/modules/services/web-apps/nextcloud.nix

+        default = "nextcloud";
+        description = "Database User";
+      };
+      dbpass = mkOption {


How about adding another option passwordFile (to avoid storing dbpass in the nix store) and read the contents in the autoconfig.php using file_get_contents?

Same for admin password.

eqyiel · 2018-09-03T03:14:36Z

nixos/modules/services/web-apps/nextcloud.nix

+        phpPackage = pkgs.php71;
+        pools.nextcloud = let
+          phpAdminValues = concatStringsSep "\n" (flip mapAttrsToList phpOptions
+            (k: v: "php_admin_value[${k}] = ${v}"));


Something weird is going on here:

[root@cloud:~]# cat /nix/store/jkqz260cfm0335fvpyflpvdhj1hgkqjp-nextcloud-occ/bin/nextcloud-occ #! /nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/bash cd /nix/store/wni0plnr91xlcrz2lfwr4dzlxlmb058r-nextcloud-13.0.5 exec /nix/store/bgwy6ysr8wffmy74xn9pj978gxas4n7j-sudo-1.8.23/bin/sudo -u nextcloud \ NEXTCLOUD_CONFIG_DIR="/data/var/lib/nextcloud/config" \ /nix/store/bmwiwi69556177ndfdg0nc6pg0j84ky1-php-7.1.21/bin/php \ -c /nix/store/vlpanb3f5iigwkigr3ywg6zzryywkwng-php.ini\ occ $* [root@cloud:~]# cat /nix/store/vlpanb3f5iigwkigr3ywg6zzryywkwng-php.ini extension=/nix/store/vfkv108a0zya61b4gxxyym4jj9fr9pyk-php-apcu-5.1.11/lib/php/extensions/apcu.so zend_extension = opcache.so opcache.enable = 1 k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v k = v

It could be related to this line?

Actually, it's due the literal k = v here: https://github.com/NixOS/nixpkgs/pull/44994/files#diff-fc6e7fa5e8f92a630f0738fa2fc4e9faR20 😅

Whoops, nice find!

eqyiel · 2018-09-03T03:17:17Z

@globin Thanks for your efforts, I've been trying this on my server and it's good except for this one thing: https://github.com/NixOS/nixpkgs/pull/44994/files#r214568515

flokli · 2018-09-09T20:57:33Z

I'd really like to see this in 18.09.

@globin do you have time to address the mentioned things in the coming days?

Otherwise, I could also follow-up from here…

globin · 2018-09-20T16:27:14Z

@flokli sorry for not having time, would be great if you could finish it up! Ideally also just cherry-pick in the update to nextcloud 14!

Mic92 · 2018-09-20T22:48:09Z

nixos/modules/services/web-apps/nextcloud.nix

+    post_max_size = cfg.maxUploadSize;
+    memory_limit = cfg.maxUploadSize;
+  } // cfg.phpOptions;
+  phpOptionsStr = phpOptionsExtensions + (concatStringsSep "\n" (mapAttrsToList (k: v: "k = v") phpOptions));


- "k = v" + "${k} = ${v}"

toKeyValue in lib/generators.nix might help.

flokli · 2018-09-23T13:17:07Z

@eqyiel thanks for picking up. @globin, I think this can be closed.

fpletz and others added 3 commits August 13, 2018 22:24

nextcloud module: init

ca049a6

nextcloud module: improve documentation

f8e0a78

nextcloud: add tests

e5e3e98

globin added the 8.has: module (new) label Aug 13, 2018

globin added this to the 18.09 milestone Aug 13, 2018

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Aug 13, 2018

nixos module: add priority for locations

e205e98

Locations need an order because nginx will try to match them in the order they appear in the config file. This reuses the sortProperties function that is also used by the module system for `mkOrder` uses.

globin changed the title ~~Nextcloud~~ nextcloud module: init Aug 14, 2018

flokli mentioned this pull request Aug 15, 2018

nixos/nextcloud: Init #44839

Closed

9 tasks

nextcloud: incorporate improvements from NixOS#44839

9bdd7af

Closes NixOS#44839 Co-authored-by: Janne Heß <janne@hess.ooo>

globin force-pushed the nextcloud branch from 1b4afaf to 9bdd7af Compare August 19, 2018 22:35

flokli reviewed Aug 25, 2018

View reviewed changes

Ma27 reviewed Aug 31, 2018

View reviewed changes

eqyiel reviewed Sep 3, 2018

View reviewed changes

globin mentioned this pull request Sep 9, 2018

nextcloud: 13.0.6 -> 14.0.0 #46408

Closed

9 tasks

Mic92 reviewed Sep 20, 2018

View reviewed changes

eqyiel mentioned this pull request Sep 22, 2018

nextcloud module: init #47159

Merged

9 tasks

flokli mentioned this pull request Sep 23, 2018

php: set mysql socket path if mysql[i] or pdo_mysql support is enabled #47220

Merged

9 tasks

Mic92 closed this Sep 23, 2018

globin deleted the nextcloud branch July 22, 2019 13:26

nextcloud module: init #44994

nextcloud module: init #44994

Conversation

globin commented Aug 13, 2018

Motivation for this change

Things done

GrahamcOfBorg commented Aug 13, 2018

GrahamcOfBorg commented Aug 13, 2018

GrahamcOfBorg commented Aug 13, 2018

Ma27 commented Aug 13, 2018

Ma27 commented Aug 13, 2018 • edited

globin commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

flokli commented Aug 15, 2018

globin commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

dasJ commented Aug 20, 2018

flokli commented Aug 22, 2018 • edited

flokli commented Aug 23, 2018 • edited

globin commented Aug 24, 2018

globin commented Aug 25, 2018

Choose a reason for hiding this comment

flokli commented Aug 25, 2018

globin commented Aug 26, 2018

flokli commented Aug 26, 2018

Ma27 left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eqyiel commented Sep 3, 2018

flokli commented Sep 9, 2018

globin commented Sep 20, 2018

Mic92 Sep 20, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flokli commented Sep 23, 2018

Ma27 commented Aug 13, 2018 •

edited

flokli commented Aug 22, 2018 •

edited

flokli commented Aug 23, 2018 •

edited

Ma27 left a comment •

edited

Mic92 Sep 20, 2018 •

edited