Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

services.logstash: default options, examples and address update #21311

Merged
merged 2 commits into from Jan 25, 2017

Conversation

makefu
Copy link
Contributor

@makefu makefu commented Dec 20, 2016

Motivation for this change

services.logstash contains obsolete defaults,examples and uses address instead of listenAddress. Cleaning up these defaults will result in a logstash service which works out of the box and the documentation will show a sane example config

Things done

@mention-bot
Copy link

@makefu, thanks for your PR! By analyzing the history of the files in this pull request, we identified @offlinehacker, @cstrahan and @edolstra to be potential reviewers.

@Mic92
Copy link
Member

Mic92 commented Dec 21, 2016

Unrelated to this patch, but the tests currently fail for me:

$  nix-build '<nixpkgs/nixos/tests/logstash.nix>'
NameError: uninitialized constant Jars
one# [   32.497571] logstash[677]:   const_missing at org/jruby/RubyModule.java:2719
one# [   32.498752] logstash[677]:     require_jar at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/lib/bootstrap/patches/jar_dependencies.rb:5
one# [   32.501440] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/jruby/lib/ruby/shared/jopenssl/load.rb:13
one# [   32.505639] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.507431] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/jruby/lib/ruby/shared/openssl.rb:1
one# [   32.512397] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.516173] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/jruby/lib/ruby/shared/openssl.rb:1
one# [   32.521032] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.522909] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/patches/str
onger_openssl_defaults.rb:1
one# [   32.525991] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.527777] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/patches/str
onger_openssl_defaults.rb:2
one# [   32.530681] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.532471] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/patches.rb:
1
one# [   32.537405] logstash[677]:         require at org/jruby/RubyKernel.java:1040
one# [   32.540805] logstash[677]:          (root) at /nix/store/5axbmfa473x1iiyka3qrdvqcz6hkj18n-logstash-2.4.0/lib/bootstrap/environment.rb:68
one# [   32.579793] systemd[1]: logstash.service: Main process exited, code=exited, status=1/FAILURE

The default configuration made logstash stopped after startup. Presumably stdin is closed in the service. I added user/group support so that logstash does not need to be run as root:

diff --git a/nixos/modules/services/logging/logstash.nix b/nixos/modules/services/logging/logstash.nix
index 74ad037834..b1a8aa8784 100644
--- a/nixos/modules/services/logging/logstash.nix
+++ b/nixos/modules/services/logging/logstash.nix
@@ -38,6 +38,16 @@ in
         description = "Logstash package to use.";
       };
 
+      extraGroups = mkOption {
+        type = types.listOf types.str;
+        default = [];
+        example = ["systemd-journal"];
+        description = ''
+          List of groups which logstash user is added to. This might be used to
+          grant logstash access to certain types of logs.
+        '';
+      };
+
       plugins = mkOption {
         type = types.listOf types.path;
         default = [ ];
@@ -77,7 +87,7 @@ in
 
       inputConfig = mkOption {
         type = types.lines;
-        default = ''stdin { type => "example" }'';
+        default = ''file { path => "/var/log/boot.log" }'';
         description = "Logstash input configuration.";
         example = ''
           # Read from journal
@@ -123,12 +133,26 @@ in
   ###### implementation
 
   config = mkIf cfg.enable {
+    users.extraUsers.logstash = {
+      home = "/var/lib/logstash";
+      extraGroups = cfg.extraGroups;
+      createHome = true;
+    };
     systemd.services.logstash = with pkgs; {
       description = "Logstash Daemon";
       wantedBy = [ "multi-user.target" ];
-      environment = { JAVA_HOME = jre; };
+      environment = {
+        JAVA_HOME = jre;
+        HOME = "/var/lib/logstash";
+      };
       path = [ pkgs.bash ];
+      preStart = ''
+        install -d -m 755 -o logstash /var/lib/logstash
+        chown -R logstash /var/lib/logstash
+      '';
       serviceConfig = {
+        User = "logstash";
+        PermissionsStartOnly = true;
         ExecStart =
           "${cfg.package}/bin/logstash agent " +
           "-w ${toString cfg.filterWorkers} " +

@makefu
Copy link
Contributor Author

makefu commented Dec 21, 2016

Error in travis-ci is:

subprocess.CalledProcessError: Command '['git', 'merge', '981553cbb8cebbd90dd26d1fceeeb270760a18a2', '-qm', 'Nox automatic merge']' returned non-zero exit status 1

@pSub
Copy link
Member

pSub commented Jan 12, 2017

I've restarted the Travis job, as this looks like a temporary problem. @makefu can you reproduce the error reported by @Mic92 and if so do you plan to fix it?

@makefu
Copy link
Contributor Author

makefu commented Jan 13, 2017

@pSub i cannot reproduce the build error described by @Mic92 https://gist.github.com/makefu/63f4defda611a4f859a843c8a542b1e1
The patch by mic also makes use of /var/log/boot.log which in my case is not available. What we can do is simply use the generator input to have "something".

Adding a separate user for logstash was not in scope of this PR - it was simply for cleaning up insane defaults.

@pSub
Copy link
Member

pSub commented Jan 13, 2017

At least the travis job is fine now. I will try to test this locally in the next few days.

@globin globin merged commit 117e554 into NixOS:master Jan 25, 2017
@makefu
Copy link
Contributor Author

makefu commented Jan 25, 2017

thanks!

@makefu makefu deleted the services/logstash branch January 25, 2017 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants