Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/graylog: minor fixes, add test #64463

Merged
merged 3 commits into from Jul 21, 2019
Merged

nixos/graylog: minor fixes, add test #64463

merged 3 commits into from Jul 21, 2019

Conversation

Ma27
Copy link
Member

@Ma27 Ma27 commented Jul 9, 2019

Motivation for this change

This PR consists of three changes:

  • 3464c60: ensure that the directory which contains the node-id files (/var/lib/graylog/server by default) exists. If the directory doesn't exist, graylog's startup breaks.
  • 5dad370: update SystemdJournal2Gelf (a tool which sends journal logs from systemd to graylog) to the latest version. Also modified the package to use buildGoModule now.
  • 875f533 : write a simple test for graylog which ensures that an input can be created and a log message can be sent to it. Further details are explained in the commit message.
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Until now the startup failed with an error like this:

```
com.github.joschi.jadconfig.ValidationException: Parent path /var/lib/graylog/server for Node ID file at /var/lib/graylog/server/node-id is not a directory
```

This happens since `graylog.service` ensures that `/var/lib/graylog`
exists, however it doesn't take care of the directory for
`cfg.nodeIdFile`.
@Ma27
Copy link
Member Author

Ma27 commented Jul 9, 2019

@GrahamcOfBorg test graylog
cc @fadenb

@Ma27
Copy link
Member Author

Ma27 commented Jul 9, 2019

@GrahamcOfBorg test graylog
Updated the PR to use a free elasticsearch version in the graylog test.

Ma27 added 2 commits July 9, 2019 23:57
Basic test which confirms new inputs can be created and that messages
can be sent to a UDP-GELF input using `netcat`.

This test requires 4GB of RAM to avoid issues due insufficient
memory (please refer to `nixos/tests/elk.nix` for a detailed explanation of
the issue) for elasticsearch.

Also it's ensured that elasticsearch has an open HTTP port for communication
when starting `graylog`. This is a workaround to ensure that all services
are started in proper order, even in test environments with less power.
However this shouldn't be implemented in the `nixos/graylog` module as
this might be harmful when using elasticsearch clusters that require e.g.
authentication and/or run on different servers.
Copy link
Member

@kalbasit kalbasit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the Go package, LGTM. I have not reviewed/tested the module.

@Ma27
Copy link
Member Author

Ma27 commented Jul 9, 2019

@GrahamcOfBorg build systemd-journal2gelf

@grahamc regarding the broken test (https://logs.nix.ci/?key=nixos/nixpkgs.64463&attempt_id=b0ca9079-3d92-4a62-a55c-bb597f39de50): it seems as there's a timeout in systemd while waiting to get graylog started. This is because I hard a hard time getting graylog to connect with elasticsearch in the test which is why I figured that the "until-sleep" hack is the best solution for the test. As I'm not that familiar with NixOS's build infrastructure: do you have any suggestion on how we should work around these timeouts? In the end it'll always take a long time to get the services started in environments with less power I guess.

@ofborg ofborg bot requested a review from kalbasit July 9, 2019 22:16
@fpletz
Copy link
Member

fpletz commented Jul 21, 2019

@GrahamcOfBorg test graylog

@fpletz fpletz merged commit 376b5fd into NixOS:master Jul 21, 2019
@Ma27 Ma27 deleted the graylog-test branch July 21, 2019 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants