Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Munin: Fix broken plugins #31916

Closed
wants to merge 4 commits into from
Closed

Munin: Fix broken plugins #31916

wants to merge 4 commits into from

Conversation

orbekk
Copy link
Contributor

@orbekk orbekk commented Nov 21, 2017

Motivation for this change

The munin-node service used wrapProgram to inject environment variables.
This doesn't work because munin plugins depend on argv[0], which is
overwritten when the executable is a script with a shebang line.

Instead, rely on environment variables passed to the systemd units. This
requires another minor patch to munin.

Also fixes #23049 by disabling the
apc_nis plugin.

Things done

The munin.nix test passes.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@orbekk orbekk changed the title Munin Munin: Fix broken plugins Nov 21, 2017
@domenkozar
Copy link
Member

@jpierre03 @bjornfor

@bjornfor bjornfor self-assigned this Nov 22, 2017
@bjornfor
Copy link
Contributor

This reminds me that I tried to fix this myself, in https://github.com/bjornfor/nixpkgs/commits/munin-master. But while testing I found a regression that I couldn't solve/pinpoint, so I dropped it.

Looking at this PR, I see the same issue I was having. The list of enabled plugins change! Look in /etc/munin/plugins with/without this PR and see the difference.

$ diff -u before-pr after-pr
--- before-pr	2017-11-22 21:05:02.266009841 +0100
+++ after-pr	2017-11-22 21:05:02.267009844 +0100
@@ -7,10 +7,21 @@
 df
 df_abs
 df_inode
+diskstat_iops_sda
+diskstat_iops_sdb
+diskstat_iops_sdc
+diskstat_iops_sdd
+diskstat_latency_sda
+diskstat_latency_sdb
+diskstat_latency_sdc
+diskstat_latency_sdd
+diskstat_throughput_sda
+diskstat_throughput_sdb
+diskstat_throughput_sdc
+diskstat_throughput_sdd
 entropy
 forks
 fw_packets
-hddtemp_smartctl
 if_docker0
 if_enp3s0
 if_enp4s0
@@ -28,7 +39,6 @@
 interrupts
 irqstats
 load
-meminfo
 memory
 munin_stats
 munin_update
@@ -42,11 +52,11 @@
 nfsd4
 open_files
 open_inodes
-port_36113
-port_38357
-port_40641
-port_42059
+port_33867
+port_34535
+port_34809
 port_51413
+port_59903
 port_apcupsd
 port_domain
 port_git

BTW, I don't think we should remove "apc_nis" plugin. It's easier to disable it from user config than to bring it back if deleted by the NixOS module. (Also, I have APC UPS.)

@bjornfor
Copy link
Contributor

I guess the only thing strange in that diff is that the hddtemp_smartctl and meminfo plugins no longer work after applying this PR.

Add a new patch (adding_sconfdir_munin-node.patch) to be able to
configure the location of plugin-conf.d (otherwise it has to be
configured at build time).  This patch is very similar to the
existing 'adding_servicedir_munin-node.patch'.
@orbekk
Copy link
Contributor Author

orbekk commented Nov 24, 2017

@bjornfor I found two issues. Both hddtemp_smartctl and meminfo work for me now.

Some plugins should run as root, hddtemp and meminfo happen to be such plugins. Added plugin level configuration to solve this. Without the wrappers, munin uses user-level state directories. This was broken with the preserve_environment.patch, so I updated it.

Some plugins are still broken (e.g., the munin plugin), but as far as I can tell there are no more regressions on the two systems I tested.

(Btw. I don't know why this worked before. :/)

@bjornfor
Copy link
Contributor

@orbekk: Great! Can you squash that last commit so each commit in the PR represents a good state? And
explain in the commit message what was needed to get hddtemp_smartctl and meminfo to work.

@grahamc
Copy link
Member

grahamc commented Nov 24, 2017

@GrahamcOfBorg test munin

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: x86_64-linux

one: exit status 1
syncing
one: running command: sync
one: exit status 0
test script finished in 123.80s
cleaning up
killing one (pid 143)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/tmp/nix-build-vm-test-run-munin.drv-0/vde1.ctl': Directory not empty
/nix/store/rz107zr4n2y0x0q8xlj75y82ng2j8b23-vm-test-run-munin

The munin-node service used wrapProgram to inject environment variables.
This doesn't work because munin plugins depend on argv[0], which is
overwritten when the executable is a script with a shebang line (example
below).

This commit removes the wrappers and instead passes the required
environment variables to munin-node.

Eliminating the wrappers resulted in some broken plugins, e.g., meminfo
and hddtemp_smartctl. That was fixed with the per-plugin configuration.

Example:

  The plugin if_eth0 is a symlink to /.../plugins/if_, which uses $0
  to determine that it should monitor traffic on the eth0 interface.

  if_ is a wrapped program, and runs `exec -a "$0" .if_-wrapped`

  .if_-wrapped has a "#!/nix/.../bash" line, which results in bash
  changing $0, and as a result the plugin thinks my interface
  is called "-wrapped".
@orbekk
Copy link
Contributor Author

orbekk commented Nov 24, 2017

@bjornfor Done. I'd also like to make a few extra configuration variables for Munin, but I can send a follow-up request for that.

@bjornfor
Copy link
Contributor

Applied to master with bd3e49a and 3 parents. Thanks!

I'll backport this to release-17.09 after a round of testing.

@bjornfor bjornfor closed this Nov 25, 2017
@bjornfor
Copy link
Contributor

Backported to NixOS 17.09 with 340f306 and 3 parents.

@jpierre03
Copy link
Contributor

thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

munin-node service: apc_nis plugin is enabled and throws errors
6 participants