solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump #49283

aanderse · 2018-10-27T16:33:11Z

Fix for issue #33876 by updating solr from an ancient version to the most recent.

I have only tested that solr actually starts running, but I have not indexed anything yet. Looking for feedback from anyone who is familiar with solr. I'll be asking a colleague at work who is familiar with solr to test this out and then report back. Will be adding a NixOS test when I get a chance.

Motivation for this change

Security issues on ancient version of solr. Version bump to latest for fix.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

andir

Overall it doesn't look bad but could you also add a nixos VM test verifying that this module works as expected?

nixos/modules/services/search/solr.nix

pkgs/servers/search/solr/default.nix

nixos/tests/solr.nix

andir · 2018-10-29T08:57:29Z

@domenkozar are you still using familiar with solr? This is trying to resurrect the whole thing with a test. If you have any input that would be appreciated.

aanderse · 2018-10-31T01:15:42Z

@andir Given this entirely breaks compatibility with previous module/package, but previous module/package is now entirely unsupported by upstream (and has CVEs against it...) what is the plan as far as which versions of NixOS this will land in?

andir · 2018-10-31T10:55:22Z

@aanderse I'd say master and then the next release until there is a good reason to replace the old (& broken) module on 18.09.

aanderse · 2018-10-31T12:02:35Z

@andir sounds good.

@GrahamcOfBorg test solr

GrahamcOfBorg · 2018-10-31T12:02:51Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.solr

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-10-31T12:04:05Z

Success on x86_64-linux (full log)

Attempted: tests.solr

Partial log (click to expand)

machine: exit status 1
syncing
machine: running command: sync
machine: exit status 0
test script finished in 41.47s
cleaning up
killing machine (pid 597)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/hn8in1czvbr5j3qmzpz9n4h1r5gni6xf-vm-test-run-solr

andir · 2018-10-31T13:07:35Z

@aanderse I did rebase, and moved a few changes around. I also had to bump the module id since something else was merged with 307 already.

@GrahamcOfBorg test solr

When the tests complete this is good for me.

GrahamcOfBorg · 2018-10-31T13:07:50Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.solr

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

andir · 2018-10-31T13:08:29Z

@aanderse would you like writing a changelog entry for the (re-)introduction of solr? :)

GrahamcOfBorg · 2018-10-31T13:08:40Z

Success on x86_64-linux (full log)

Attempted: tests.solr

Partial log (click to expand)

machine: exit status 1
syncing
machine: running command: sync
machine: exit status 0
test script finished in 36.85s
cleaning up
killing machine (pid 597)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/4piih9b917n00k6snghscz9v5ar9nb2k-vm-test-run-solr

GrahamcOfBorg · 2018-10-31T13:20:26Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: solr

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-10-31T13:20:54Z

Success on x86_64-linux (full log)

Attempted: solr

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/bin
patching script interpreter paths in /nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/server/scripts/cloud-scripts/zkcli.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/server/scripts/cloud-scripts/snapshotscli.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/example/exampledocs/test_utf8.sh: interpreter directive changed from "/bin/sh" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/sh"
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/contrib/prometheus-exporter/bin/solr-exporter: interpreter directive changed from "/bin/sh" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/sh"
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/bin/.post-wrapped: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0/bin/.solr-wrapped: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
checking for references to /build in /nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0...
/nix/store/8lmj2yjns5cpgjcdxckxia1jgf96xw8m-solr-7.5.0

aanderse · 2018-11-03T00:48:49Z

@andir I hope this entry will suffice. If not please let me know. Thank you.

andir · 2018-11-03T12:46:53Z

Can you cleanup the git history a bit like I did a few days before you pushed your latest changes. I think the merge of master should go away and also a few of those commits should be stashed together.

aanderse · 2018-11-03T15:42:24Z

@andir Sorry for the problems, have studied up on my git history rewriting enough to accomplish as requested.

GrahamcOfBorg · 2018-11-03T15:43:55Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: solr

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-11-03T15:44:19Z

Success on x86_64-linux (full log)

Attempted: solr

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/bin
patching script interpreter paths in /nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/server/scripts/cloud-scripts/snapshotscli.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/server/scripts/cloud-scripts/zkcli.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/bin/.post-wrapped: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/bin/.solr-wrapped: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/bash"
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/example/exampledocs/test_utf8.sh: interpreter directive changed from "/bin/sh" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/sh"
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0/contrib/prometheus-exporter/bin/solr-exporter: interpreter directive changed from "/bin/sh" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0...
/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0

…ersion bump, NixOS test included

GrahamcOfBorg · 2018-11-03T17:27:19Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: solr

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

aanderse · 2018-11-03T17:27:40Z

Just to be sure after rewriting history... @GrahamcOfBorg test solr

GrahamcOfBorg · 2018-11-03T17:27:48Z

Success on x86_64-linux (full log)

Attempted: solr

Partial log (click to expand)

/nix/store/d2kgqgl08navv2npc2fw2p5s55lcbq61-solr-7.5.0

aanderse mentioned this pull request Oct 27, 2018

apache solr: CVE-2017-12629 & CVE-2017-3163 #33876

Closed

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 27, 2018

andir requested changes Oct 27, 2018

View reviewed changes

nixos/modules/services/search/solr.nix Outdated Show resolved Hide resolved

nixos/modules/services/search/solr.nix Outdated Show resolved Hide resolved

pkgs/servers/search/solr/default.nix Outdated Show resolved Hide resolved

andir requested changes Oct 29, 2018

View reviewed changes

nixos/tests/solr.nix Show resolved Hide resolved

aanderse force-pushed the solr branch from f324e1a to ca8d192 Compare October 31, 2018 00:43

GrahamcOfBorg added 2.status: merge conflict and removed 2.status: merge conflict labels Oct 31, 2018

andir force-pushed the solr branch 2 times, most recently from b0111c4 to a7c5838 Compare October 31, 2018 13:06

andir approved these changes Oct 31, 2018

View reviewed changes

andir changed the title ~~WIP: solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump~~ solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump Oct 31, 2018

GrahamcOfBorg added 8.has: changelog 8.has: documentation 2.status: merge conflict labels Nov 3, 2018

aanderse force-pushed the solr branch from 58a4220 to cd51c5a Compare November 3, 2018 15:05

GrahamcOfBorg added 6.topic: xfce The Xfce Desktop Environment and removed 6.topic: xfce The Xfce Desktop Environment 2.status: merge conflict labels Nov 3, 2018

solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in v…

1b725de

…ersion bump, NixOS test included

aanderse force-pushed the solr branch from b14e9f4 to 1b725de Compare November 3, 2018 17:15

andir merged commit c891dac into NixOS:master Nov 4, 2018

aanderse deleted the solr branch November 4, 2018 13:48

aanderse mentioned this pull request Nov 7, 2018

SOLR version is way out of date #24306

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump #49283

solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump #49283

aanderse commented Oct 27, 2018

andir left a comment

andir commented Oct 29, 2018

aanderse commented Oct 31, 2018

andir commented Oct 31, 2018

aanderse commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

andir commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

andir commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

aanderse commented Nov 3, 2018

andir commented Nov 3, 2018

aanderse commented Nov 3, 2018 •

edited

GrahamcOfBorg commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

aanderse commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump #49283

solr: 4.10.3 -> 7.5.0, refactor service to reflect major changes in version bump #49283

Conversation

aanderse commented Oct 27, 2018

Motivation for this change

Things done

andir left a comment

Choose a reason for hiding this comment

andir commented Oct 29, 2018

aanderse commented Oct 31, 2018

andir commented Oct 31, 2018

aanderse commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

andir commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

andir commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

GrahamcOfBorg commented Oct 31, 2018

aanderse commented Nov 3, 2018

andir commented Nov 3, 2018

aanderse commented Nov 3, 2018 • edited

GrahamcOfBorg commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

aanderse commented Nov 3, 2018

GrahamcOfBorg commented Nov 3, 2018

aanderse commented Nov 3, 2018 •

edited