nexus: fix setup and nixos test #40522

Ma27 · 2018-05-14T20:27:45Z

Motivation for this change

The original nexus derivation required /run/sonatype-work/nexus3
which explicitly depended on the NixOS path structure.

This would break nexus for everyone using nixpkgs on a non-NixOS
system, additionally the module never created /run/sonatype-work, so
the systemd unit created in services.nexus fails as well. The issue
wasn't actively known as the nixos/nexus test wasn't registered in
Hydra (see #40257).

This patch contains the following changes:

Adds tests.nexus to release.nix to run the test on Hydra.
Makes JVM parameters configurable: by default all JVM options were located
in result/bin/nexus.vmoptions which made it quite hard to patch
these parameters. Now it's possible to override all parameters by
running VM_OPTS_FILE=custom-nexus.vmoptions ./result/bin/nexus run
(after patching the nexus shell script), additionally it's possible
to override these parameters with services.nexus.vmoptions.
Bumped Nexus from 3.5.1 to 3.11.0
Run the nexus test on Hydra with callTest in nixos/release.nix,
furthermore the test checks if the UI is available on the specified
port.
Added myself as maintainer for the NixOS test and the package to have
some more people in case of further breakage.

Added sufficient disk space to the nexus test, otherwise the service
fails with the following errors:

com.orientechnologies.orient.core.exception.ODatabaseException: Cannot create database 'accesslog'
com.orientechnologies.orient.core.exception.OLowDiskSpaceException: Error occurred while executing
  a write operation to database 'accesslog' due to limited free space on the disk (242 MB). The database
  is now working in read-only mode. Please close the database (or stop OrientDB), make room on your hard
  drive and then reopen the database. The minimal required space is 256 MB. Required space is now set to
  256MB (you can change it by setting parameter storage.diskCache.diskFreeSpaceLimit) .

/cc @ironpinguin @xeji

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

GrahamcOfBorg · 2018-05-14T20:39:18Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: nexus

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-14T20:41:57Z

Success on x86_64-linux (full log)

Attempted: nexus

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01/lib  /nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01/bin
patching script interpreter paths in /nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01
/nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01/bin/nexus.orig: interpreter directive changed from "/bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
/nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01/bin/.nexus-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01...
warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy
/nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01

xeji · 2018-05-14T22:59:01Z

nixos/modules/services/web-apps/nexus.nix

+          -Dkaraf.startLocalConsole=false
+        '';
+
+        description = "Options for the JVM written to `nexus.jvmopts`";


Can you add a link to where the nexus-specific jvm options are explained in the nexus docs ?

xeji · 2018-05-14T23:02:07Z

nixos/tests/nexus.nix

@@ -6,29 +6,27 @@
 import ./make-test.nix ({ pkgs, ...} : {


Please update/clarify the comments at the top of this file describing what this test does (github doesn't let me comment on lines not changed by this PR).

xeji · 2018-05-14T23:05:30Z

nixos/tests/nexus.nix

-
-    print $server->execute("sudo -u nexus groups");
-    $server->mustSucceed("sudo -u nexus groups | grep nexus | grep users");
-
    $server->waitForOpenPort(8081);


This takes quite long on my local machine because the service starts up slowly. May time out on Hydra. But let's see before we make this more complicated.

xeji · 2018-05-14T23:13:14Z

nixos/tests/nexus.nix

    $server->waitForOpenPort(8081);
+
+    $server->succeed("curl 127.0.0.1:8081/nexus");


Please use curl --fail because by default, curl will exit with 0 even when the server gives a 404 or 500 ...
I would also add some code to sanity check the html output, like | grep -q 'Nexus Repository Manager'.

@ironpinguin

The original `nexus` derivation required `/run/sonatype-work/nexus3` which explicitly depended on the NixOS path structure. This would break `nexus` for everyone using `nixpkgs` on a non-NixOS system, additionally the module never created `/run/sonatype-work`, so the systemd unit created in `services.nexus` fails as well. The issue wasn't actively known as the `nixos/nexus` test wasn't registered in Hydra (see NixOS#40257). This patch contains the following changes: * Adds `tests.nexus` to `release.nix` to run the test on Hydra. * Makes JVM parameters configurable: by default all JVM options were located in `result/bin/nexus.vmoptions` which made it quite hard to patch these parameters. Now it's possible to override all parameters by running `VM_OPTS_FILE=custom-nexus.vmoptions ./result/bin/nexus run` (after patching the `nexus` shell script), additionally it's possible to override these parameters with `services.nexus.vmoptions`. * Bumped Nexus from 3.5.1 to 3.11.0 * Run the `nexus` test on Hydra with `callTest` in `nixos/release.nix`, furthermore the test checks if the UI is available on the specified port. * Added myself as maintainer for the NixOS test and the package to have some more people in case of further breakage. * Added sufficient disk space to the `nexus` test, otherwise the service fails with the following errors: ``` com.orientechnologies.orient.core.exception.ODatabaseException: Cannot create database 'accesslog' com.orientechnologies.orient.core.exception.OLowDiskSpaceException: Error occurred while executing a write operation to database 'accesslog' due to limited free space on the disk (242 MB). The database is now working in read-only mode. Please close the database (or stop OrientDB), make room on your hard drive and then reopen the database. The minimal required space is 256 MB. Required space is now set to 256MB (you can change it by setting parameter storage.diskCache.diskFreeSpaceLimit) . ``` /cc @ironpinguin @xeji

Ma27 · 2018-05-15T11:52:56Z

@xeji thanks a lot for your feedback!

This takes quite long on my local machine because the service starts up slowly. May time out on Hydra. But let's see before we make this more complicated.

Unfortunately yes, it takes some time to start the nexus service (especially the first time in a new environment).
But you're right, let's see how it works on Hydra and then check if we have to improve the test :)

xeji · 2018-05-15T11:56:40Z

@GrahamcOfBorg test nexus

GrahamcOfBorg · 2018-05-15T11:57:03Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.nexus

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-15T12:01:08Z

Success on x86_64-linux (full log)

Attempted: nexus

Partial log (click to expand)

/nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01

GrahamcOfBorg · 2018-05-15T12:01:11Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: nexus

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-15T12:02:38Z

Success on x86_64-linux (full log)

Attempted: tests.nexus

Partial log (click to expand)

server: exit status 1
syncing
server: running command: sync
server: exit status 0
test script finished in 331.46s
cleaning up
killing server (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/3z1k0jsfmypbwnq06phlg6c4d3yrjml5-vm-test-run-nexus

xeji · 2018-05-15T12:05:24Z

let's check if this builds on Darwin as well

@GrahamcOfBorg build nexus

GrahamcOfBorg · 2018-05-15T12:05:34Z

Success on x86_64-linux (full log)

Attempted: nexus

Partial log (click to expand)

/nix/store/qwq11zdliyv8azdmyr2gyk74qk0yspi5-nexus-3.11.0-01

GrahamcOfBorg · 2018-05-15T12:05:41Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: nexus

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-15T12:06:05Z

Success on x86_64-darwin (full log)

Attempted: nexus

Partial log (click to expand)

'.install4j/user/guava-21.0.jar' -> '/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/.install4j/user/guava-21.0.jar'
'.install4j/user/install4j-common-1.1.1.jar' -> '/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/.install4j/user/install4j-common-1.1.1.jar'
'.install4j/user/install4j-slf4j-1.1.1.jar' -> '/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/.install4j/user/install4j-slf4j-1.1.1.jar'
'.install4j/user/slf4j-api-1.7.25.jar' -> '/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/.install4j/user/slf4j-api-1.7.25.jar'
post-installation fixup
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/lib  /nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/bin
patching script interpreter paths in /nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01
/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/bin/nexus.orig: interpreter directive changed from "/bin/sh" to "/nix/store/r8bx3qf1bpncb14i9gzma4vr089pc3pv-bash-4.4-p19/bin/sh"
/nix/store/931ixwcd7zlmk7y3r6lxi5jsilfiav73-nexus-3.11.0-01/bin/.nexus-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/r8bx3qf1bpncb14i9gzma4vr089pc3pv-bash-4.4-p19/bin/sh"

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels May 14, 2018

xeji reviewed May 14, 2018

View reviewed changes

Ma27 force-pushed the fix-nexus branch from b9af276 to c769f9b Compare May 15, 2018 11:49

xeji merged commit 91365cd into NixOS:master May 15, 2018

Ma27 deleted the fix-nexus branch May 15, 2018 13:19

xeji mentioned this pull request May 15, 2018

Fix failing NixOS tests #40257

Closed

41 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nexus: fix setup and nixos test #40522

nexus: fix setup and nixos test #40522

Ma27 commented May 14, 2018 •

edited

GrahamcOfBorg commented May 14, 2018

GrahamcOfBorg commented May 14, 2018

xeji May 14, 2018

xeji May 14, 2018

xeji May 14, 2018

xeji May 14, 2018

Ma27 commented May 15, 2018

xeji commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

xeji commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

		$server->waitForOpenPort(8081);

		$server->succeed("curl 127.0.0.1:8081/nexus");

nexus: fix setup and nixos test #40522

nexus: fix setup and nixos test #40522

Conversation

Ma27 commented May 14, 2018 • edited

Motivation for this change

Things done

GrahamcOfBorg commented May 14, 2018

GrahamcOfBorg commented May 14, 2018

xeji May 14, 2018

Choose a reason for hiding this comment

xeji May 14, 2018

Choose a reason for hiding this comment

xeji May 14, 2018

Choose a reason for hiding this comment

xeji May 14, 2018

Choose a reason for hiding this comment

Ma27 commented May 15, 2018

xeji commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

xeji commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

Ma27 commented May 14, 2018 •

edited