nixos/nextcloud: fix postgresql/redis test #68441

jonringer · 2019-09-10T18:43:49Z

Motivation for this change

While reviewing ##68435, I noticed one of the tests were failing on master.

redis.user option was removed here #67845

Things done

Notify maintainers

cc @flokli @eqyiel @infinisil

jonringer · 2019-09-10T20:52:50Z

@GrahamcOfBorg test nextcloud.with-postgresql-and-redis

jonringer · 2019-09-10T20:56:35Z

[13:56:11] jon@jon-workstation ~/projects/nixpkgs (fix-nextcloud-test)
$ nix-build -A nixosTests.nextcloud
...
/nix/store/2nxdzpzg9qzcy939x90sslrlp7xl3yaa-vm-test-run-nextcloud-basic
/nix/store/yijda41za53xfjzp9yhchbwy8prvbzrd-vm-test-run-nextcloud-with-mysql-and-memcached
/nix/store/3qs676zszr24ra2vyk1c4mfhmhwaa639-vm-test-run-nextcloud-with-postgresql-and-redis

[13:56:47] jon@jon-workstation ~/projects/nixpkgs (master)
$ nix-build -A nixosTests.nextcloud
...

error: The option `services.redis.user' is used but not defined.
(use '--show-trace' to show detailed location information)

flokli

Can you remove the chown-redis-socket service, and instead broaden permissions on the file socket, or switch to link-local networking?

nixos/tests/nextcloud/with-postgresql-and-redis.nix

flokli · 2019-09-11T02:11:28Z

nixos/tests/nextcloud/with-postgresql-and-redis.nix

@@ -46,7 +46,7 @@ in {
      systemd.services.redis = {
        preStart = ''
          mkdir -p /var/run/redis
-          chown ${config.services.redis.user}:${config.services.nginx.group} /var/run/redis
+          chown redis:${config.services.nginx.group} /var/run/redis


If we want the test to keep using the file socket (and not link-local networking instead, we should instead set unixsocketperm in the test to something more permissive.

I'm honestly not sure what you mean, :)

do you mind writing what you think would be appropriate?

These lines seem to exist only to get php to be able to connect to redis.

Further down, configureRedis = pkgs.writeScript "configure-redis" runs nextcloud-occ and configures nextcloud to connect to redis at /var/run/redis/redis.sock.

This preStart tries to extend the existing redis systemd service to obtain a less restrictive /run/redis folder (which contains the unix socket file).
However, /run/redis and its permissions are managed by systemd's RuntimeDirectory nowadays (and /var/run/ is a symlink to /run`), so this doesn't have the desired effect.

I'd suggest to drop the preStart, services.redis.unixSocket and services.redis.extraConfig, and configure nextcloud to connect to localhost:6379.

makes more sense, i don't dabble much with systemd, it's next on my plate. Thanks!

Do you feel confident to update the PR with this suggestions?

I do, just slammed in my personal life with obligations (friends visiting from out of town)

No worries :-)

nixos/tests/nextcloud/with-postgresql-and-redis.nix

jonringer · 2019-10-03T04:02:26Z

hmm, I forgot about this... sorry @flokli

jonringer · 2019-10-03T05:57:17Z

still not entirely sure what I'm doing :)

jonringer · 2019-10-03T06:01:00Z

Current Failure :(

3.9/bin/php -c /nix/store/byhjdjvfa11ymah191qcykdlr5iph83x-php.ini occ config:system:set memcache.locking --value \OC\Memcache\Redis --type string
nextcloud# [  677.983812] sudo[1603]: pam_unix(sudo:session): session opened for user nextcloud by (uid=0)
nextcloud# [  680.283712] sudo[1603]: pam_unix(sudo:session): session closed for user nextcloud
nextcloud: exit status 1
nextcloud: output: System config value redis => host set to string /var/run/redis/redis.sock
System config value redis => port set to integer 0
System config value memcache.local set to string \OC\Memcache\Redis
An unhandled exception has been thrown:
RedisException: Redis server went away in /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/Memcache/Redis.php:54
Stack trace:
#0 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/Memcache/Redis.php(54): Redis->get('40f5fd180ff279c...')
#1 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/App/InfoParser.php(57): OC\Memcache\Redis->get('/var/lib/nextcl...')
#2 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/App/AppManager.php(397): OC\App\InfoParser->parse('/var/lib/nextcl...')
#3 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/legacy/app.php(550): OC\App\AppManager->getAppInfo('files', false, NULL)
#4 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/AppFramework/App.php(66): OC_App::getAppInfo('files')
#5 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/legacy/app.php(235): OC\AppFramework\App::buildAppNamespace('files')
#6 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/private/legacy/app.php(119): OC_App::registerAutoloading('files', '/var/lib/nextcl...')
#7 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/base.php(640): OC_App::loadApps(Array)
#8 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/lib/base.php(1056): OC::init()
#9 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/console.php(46): require_once('/nix/store/grn4...')
#10 /nix/store/grn44xcvqlbmvz32ql7vmg6wh9ql66h3-nextcloud-16.0.5/occ(11): require_once('/nix/store/grn4...')
#11 {main}
(19.91 seconds)
error: Attempt to end element "line" with "nest" tag at /nix/store/6ngv6mgfrjxf93yfrzxpra2wy7dzxcym-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 56.
(682.54 seconds)
Attempt to end element "line" with "nest" tag at /nix/store/6ngv6mgfrjxf93yfrzxpra2wy7dzxcym-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 56.
cleaning up
killing nextcloud (pid 597)
killing client (pid 609)
(0.00 seconds)
Attempt to end element "line" with "logfile" tag at /nix/store/6ngv6mgfrjxf93yfrzxpra2wy7dzxcym-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 28.
END failed--call queue aborted, <__ANONIO__> line 20507.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/z2h2lb5qqnk118gav9c6hlcsqj575xjj-vm-test-run-nextcloud-with-postgresql-and-redis.drv' failed with exit code 22
error: build of '/nix/store/z2h2lb5qqnk118gav9c6hlcsqj575xjj-vm-test-run-nextcloud-with-postgresql-and-redis.drv' failed

jonringer · 2019-10-12T22:13:17Z

@flokli I really have no idea what I'm doing at this point. I tried getting the services to communicate over localhost and just hit a wall with it "not being able to find the nextcloud user"

flokli · 2019-10-13T11:49:57Z

There seems to be some breakage with connecting to postgresql during the installation:

nextcloud# [   13.133268] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]: 2019-10-13 11:44:25.831 GMT [800] FATAL:  password authentication failed for user "nextcloud"
nextcloud# [   13.135977] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]: 2019-10-13 11:44:25.831 GMT [800] DETAIL:  User "nextcloud" has no password assigned.
nextcloud# [   13.138797] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]:   Connection matched pg_hba.conf line 4: "host  all all ::1/128      md5"
nextcloud# [   13.152370] Nextcloud[796]: {"reqId":"TxG4oADpWtmci6Q9AKaR","level":2,"time":"2019-10-13T11:44:25+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Error occurred while checking PostgreSQL version, assuming >= 9","userAgent":"--","version":""}
nextcloud# [   13.159860] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]: 2019-10-13 11:44:25.850 GMT [801] FATAL:  password authentication failed for user "nextcloud"
nextcloud# [   13.163490] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]: 2019-10-13 11:44:25.850 GMT [801] DETAIL:  User "nextcloud" has no password assigned.
nextcloud# [   13.166003] c2r8xhhiw77ijzq2b6l6nbpfaq17lfq6-unit-script-postgresql-start[657]:   Connection matched pg_hba.conf line 4: "host  all all ::1/128      md5"
nextcloud: running command: systemctl --no-pager show "multi-user.target"

This is broken already without redis.

After the installation fails, we still seem to try to run the occ commands, even though the installation fails.

@Ma27, @Ekleog, could you take a look at this?

Ma27 · 2019-10-13T11:52:01Z

Can take a look at this later that day :)

Ma27 · 2019-10-13T12:33:06Z

Just pushed a patch that fixes the test locally. @flokli @jonringer would you mind re-reviewing? :)

jonringer · 2019-10-13T16:21:29Z

can't believe i didn't see the port line... oops ;)

Ma27 · 2019-10-13T16:46:06Z

Thanks! Backported as 3fdca39, c725b28.

ofborg bot added 6.topic: nixos 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Sep 10, 2019

flokli requested changes Sep 11, 2019

View reviewed changes

lheckemann added this to the 19.09 milestone Sep 12, 2019

jonringer force-pushed the fix-nextcloud-test branch from 10f7a32 to 6d330ce Compare October 3, 2019 05:56

nixos/nextcloud: fix postgresql/redis test

5e8ae58

jonringer force-pushed the fix-nextcloud-test branch from 6d330ce to 5e8ae58 Compare October 12, 2019 22:07

Ma27 added the 9.needs: port to stable A PR needs a backport to the stable release. label Oct 13, 2019

Ma27 self-assigned this Oct 13, 2019

nixos/nextcloud: fix postgresql test

81414c0

Ma27 approved these changes Oct 13, 2019

View reviewed changes

Ma27 merged commit 927204c into NixOS:master Oct 13, 2019

Ma27 added 8.has: port to stable A PR already has a backport to the stable release. and removed 9.needs: port to stable A PR needs a backport to the stable release. labels Oct 13, 2019

jonringer deleted the fix-nextcloud-test branch October 13, 2019 16:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/nextcloud: fix postgresql/redis test #68441

nixos/nextcloud: fix postgresql/redis test #68441

jonringer commented Sep 10, 2019 •

edited

jonringer commented Sep 10, 2019

jonringer commented Sep 10, 2019 •

edited

flokli left a comment

flokli Sep 11, 2019

jonringer Oct 3, 2019

jonringer Oct 3, 2019

flokli Oct 11, 2019

jonringer Oct 12, 2019

flokli Oct 12, 2019

jonringer Oct 12, 2019

flokli Oct 12, 2019

jonringer commented Oct 3, 2019

jonringer commented Oct 3, 2019

jonringer commented Oct 3, 2019

jonringer commented Oct 12, 2019

flokli commented Oct 13, 2019

Ma27 commented Oct 13, 2019

Ma27 commented Oct 13, 2019

jonringer commented Oct 13, 2019

Ma27 commented Oct 13, 2019

nixos/nextcloud: fix postgresql/redis test #68441

nixos/nextcloud: fix postgresql/redis test #68441

Conversation

jonringer commented Sep 10, 2019 • edited

Motivation for this change

Things done

Notify maintainers

jonringer commented Sep 10, 2019

jonringer commented Sep 10, 2019 • edited

flokli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonringer commented Oct 3, 2019

jonringer commented Oct 3, 2019

jonringer commented Oct 3, 2019

jonringer commented Oct 12, 2019

flokli commented Oct 13, 2019

Ma27 commented Oct 13, 2019

Ma27 commented Oct 13, 2019

jonringer commented Oct 13, 2019

Ma27 commented Oct 13, 2019

jonringer commented Sep 10, 2019 •

edited

jonringer commented Sep 10, 2019 •

edited