Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade the ELK stack #26252

Merged
merged 1 commit into from
Jun 13, 2017
Merged

Upgrade the ELK stack #26252

merged 1 commit into from
Jun 13, 2017

Conversation

basvandijk
Copy link
Member

@basvandijk basvandijk commented May 30, 2017

Please don't merge yet. This is a RFC with a known bug that I'm in the process of fixing.
EDIT: the PR is finished now and the test passes successfully.

This PR is based on the elasticsearch-5.4 upgrade (#25857). It contributes the following:

  • Add elasticsearch5 courtesy of @apeschar
  • Add kibana5 and logstash5
  • Upgrade the elastic beats to 5.4
  • Make sure all elastic products use the same version (see elk5Version)
  • Replace the logstash test with a test for the full ELK stack

Currently logstash5 has issues because I see the following when running the testsuite:

$ nix-build nixos/tests/elk.nix
...
one# [   14.909622] logstash[551]: Bundler::PathError: The path `/nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/logstash-core` does not exist.
one# [   14.910711] logstash[551]:         load_spec_files at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/source/path.rb:162
one# [   14.911810] logstash[551]:             local_specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/source/path.rb:92
one# [   14.913219] logstash[551]:                   specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/source/path.rb:100
one# [   14.918250] logstash[551]:   converge_locked_specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:539
one# [   14.920212] logstash[551]:                    each at org/jruby/RubyArray.java:1613
one# [   14.920873] logstash[551]:                    each at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/jruby/lib/ruby/1.9/forwardable.rb:201
one# [   14.926447] logstash[551]:   converge_locked_specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:526
one# [   14.929248] logstash[551]:                 resolve at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:187
one# [   14.932264] logstash[551]:                   specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:132
one# [   14.935266] logstash[551]:               specs_for at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:177
one# [   14.938382] logstash[551]:         requested_specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:166
one# [   14.941245] logstash[551]:         requested_specs at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/environment.rb:18
one# [   14.944477] logstash[551]:                   setup at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/runtime.rb:13
one# [   14.948441] logstash[551]:                   setup at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler.rb:122
one# [   14.953275] logstash[551]:                  setup! at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/lib/bootstrap/bundler.rb:64
one# [   14.955125] logstash[551]:                  (root) at /nix/store/99i6v7s28km91b5ys85n99fx53fyl935-logstash-5.4.0/lib/bootstrap/environment.rb:67
one# [   14.965465] systemd[1]: logstash.service: Main process exited, code=exited, status=1/FAILURE
one# [   14.966192] systemd[1]: logstash.service: Unit entered failed state.
one# [   14.966724] systemd[1]: logstash.service: Failed with result 'exit-code'.

It would be nice if somebody could help me debug this because I'm not a Ruby developer. Pinging maintainers of logstash: @offlinehacker @wjlroe

@mention-bot
Copy link

@basvandijk, thanks for your PR! By analyzing the history of the files in this pull request, we identified @globin, @fadenb and @offlinehacker to be potential reviewers.

@fpletz fpletz added 2.status: merge conflict This PR has merge conflicts with the target branch 8.has: package (update) This PR updates a package to a newer version labels May 31, 2017
@basvandijk
Copy link
Member Author

@fpletz I rebased it on master.

@fpletz fpletz added 2.status: work-in-progress This PR isn't done and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels May 31, 2017
@basvandijk basvandijk force-pushed the elk5 branch 3 times, most recently from 1811a6f to 6ac8da3 Compare May 31, 2017 13:17
@basvandijk
Copy link
Member Author

I fixed the logstash issue and the test now passes successfully. I both tested the new logstash-5.4 as well as the current logstash-2.4.

@fpletz this PR is now ready to be reviewed so the work-in-progress label can be removed.

@pSub pSub removed the 2.status: work-in-progress This PR isn't done label May 31, 2017
* Add kibana5 and logstash5
* Upgrade the elastic beats to 5.4
* Make sure all elastic products use the same version
  (see elk5Version)
* Add a test for the ELK stack
@fpletz
Copy link
Member

fpletz commented Jun 13, 2017

Thanks!

@fpletz fpletz merged commit 2444eab into NixOS:master Jun 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.has: package (update) This PR updates a package to a newer version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants