nixos/nginx: use nginxfmt and gixy #57979

4z3 · 2019-03-20T18:01:37Z

Motivation for this change

Currently nginx.conf gets formatted by an awk script. This PR replaces that script by writeNginxConfig which uses nginx-config-formatter to format, and gixy to analyze nginx configurations.

Things done

Mic92 · 2019-03-21T14:51:31Z

pkgs/build-support/writers/default.nix

+    cp "$textPath" $out
+    ${pkgs.nginx-config-formatter}/bin/nginxfmt $out
+    ${pkgs.gnused}/bin/sed -i '/^$/d' $out
+    ${pkgs.gixy}/bin/gixy $out


How likely is gixy to break valid configuration?

Should there be a opt-out flag?

I'm not sure how likely it is, but I'd argue that broken configurations should be fixed instead. Alternatively, writeNginxConfig can be overridden. This might not be the most convenient solution, but it's also not too painful either. Basically a writeNginxConfig = self.writeText would be sufficient. Anyway, if it's really desired, we can make it optional, or even pass arguments to gixy (but I haven't needed that myself before).

I guess we could test on master and see how many people would complain about it.

here to complain about it, breaks generic forward proxy configs

TomSmeets

Fails with:

error: attribute 'writeNginxConfig' missing

nixos/modules/services/web-servers/nginx/default.nix

Mic92 · 2019-03-24T06:34:16Z

@GrahamcOfBorg test nginx

Izorkin · 2019-04-08T12:02:34Z

Error work this code

location /example { allow 192.168.0.0/16; deny all; }

result

  location /example {
    allow 192.168.0.0/16;
    deny all;
  }
  ;

character is added ;

Mic92 · 2019-04-08T12:30:33Z

@Izorkin is it nginx or gixy that rejects the generated code?

Izorkin · 2019-04-08T12:36:59Z

Nginx error:

апр 08 14:55:45 web.elven.pw systemd[1]: Starting Nginx Web Server...
апр 08 14:55:45 web.elven.pw pnzhwc1q1is8vbdkg34ksicqq9jh5chc-unit-script-nginx-pre-start[2430]: nginx: [emerg] unexpected ";" in /nix/store/glvppaxv9gqg9vj179ph9rmj8yg41d72-nginx.conf:151

Mic92 · 2019-04-08T12:38:10Z

Ok. So it is nginx itself.

jb55 · 2019-04-18T23:33:48Z

@Mic92

I guess we could test on master and see how many people would complain about it.

I can't upgrade to latest unstable because of this. I get no useful error messages:

builder for '/nix/store/d2hphy6y9hn4wsj513b70vw8w3hnbzvp-nginx.conf.drv' failed with exit code 1; last 10 log lines:
   }
  }
  
  ==================== Summary ===================
  Total issues:
      Unspecified: 0
      Low: 0
      Medium: 1
      High: 1
cannot build derivation '/nix/store/7r84gqm501i30qqfhzvdvlv1bzg6fsr7-unit-nginx.service.drv': 1 dependencies couldn't be built

Izorkin · 2019-04-19T06:03:55Z

@jb55 need full log. Example:

building all machine configurations...
unpacking 'https://github.com/nix-community/NUR/archive/master.tar.gz'...
these derivations will be built:
  /nix/store/vg217jgnrfmm2kk9nk10rjvbvnqhavk7-nginx.conf.drv
  /nix/store/4k3hn2nvgf21bx551iq0zzf3ib4vl9a5-unit-script-nginx-pre-start.drv
  /nix/store/csixd8qnkwy9llqard5gpilxd52234hb-unit-nginx.service.drv
  /nix/store/z2r5caxs8g0d98ibx8mf779rm5s7l1mm-system-units.drv
  /nix/store/gsjrv7d56f8ipf98bqv95ad5ln420lx0-etc.drv
  /nix/store/wgfp73fldfzcsp8n20ii2yc9mi8wwfmy-nixos-system-my_site-19.09pre176321.4c4ad72a43c.drv
  /nix/store/56wkdkff2w42zfx2063sg978idh0g2s8-nixops-machines.drv
building '/nix/store/vg217jgnrfmm2kk9nk10rjvbvnqhavk7-nginx.conf.drv'...

==================== Results ===================

>> Problem: [http_splitting] Possible HTTP-Splitting vulnerability.
Severity: HIGH
Description: Using variables that can contain "\n" or "\r" may lead to http injection.
Additional info: https://github.com/yandex/gixy/blob/master/docs/en/plugins/httpsplitting.md
Reason: At least variable "$uri" can contain "\n"
Pseudo config:

server {
        server_name my_site;

        location @backend_prober {
                proxy_pass http://nginx-unit-13/$uri?args;
        }
}

==================== Summary ===================
Total issues:
    Unspecified: 0
    Low: 0
    Medium: 0
    High: 1

builder for '/nix/store/vg217jgnrfmm2kk9nk10rjvbvnqhavk7-nginx.conf.drv' failed with exit code 1
cannot build derivation '/nix/store/csixd8qnkwy9llqard5gpilxd52234hb-unit-nginx.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/z2r5caxs8g0d98ibx8mf779rm5s7l1mm-system-units.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/gsjrv7d56f8ipf98bqv95ad5ln420lx0-etc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/wgfp73fldfzcsp8n20ii2yc9mi8wwfmy-nixos-system-my_site-19.09pre176321.4c4ad72a43c.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/56wkdkff2w42zfx2063sg978idh0g2s8-nixops-machines.drv': 1 dependencies couldn't be built
error: build of '/nix/store/56wkdkff2w42zfx2063sg978idh0g2s8-nixops-machines.drv' failed
Traceback (most recent call last):
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/bin/..nixops-wrapped-wrapped", line 990, in <module>
    args.op()
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/bin/..nixops-wrapped-wrapped", line 411, in op_deploy
    max_concurrent_activate=args.max_concurrent_activate)
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/lib/python2.7/site-packages/nixops/deployment.py", line 1056, in deploy
    self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/lib/python2.7/site-packages/nixops/deployment.py", line 1045, in run_with_notify
    f()
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/lib/python2.7/site-packages/nixops/deployment.py", line 1056, in <lambda>
    self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/lib/python2.7/site-packages/nixops/deployment.py", line 996, in _deploy
    self.configs_path = self.build_configs(dry_run=dry_run, repair=repair, include=include, exclude=exclude)
  File "/nix/store/dmcxlqi7fwb4jws8c1qrzncfvqibvp0m-nixops-1.6.1/lib/python2.7/site-packages/nixops/deployment.py", line 664, in build_configs
    raise Exception("unable to build all machine configurations")
Exception: unable to build all machine configurations

jb55 · 2019-04-20T00:17:25Z

lzorkin: I need the full log too, I have no idea how to get that. I just reverted these commits for now.

Mic92 · 2019-04-23T10:46:10Z

@jb55 use nix log /nix/store/d2hphy6y9hn4wsj513b70vw8w3hnbzvp-nginx.conf.drv to get the full log.

build-support writers: add writeNginxConfig

7437268

4z3 requested a review from infinisil as a code owner March 20, 2019 18:01

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Mar 20, 2019

Mic92 requested a review from fpletz March 21, 2019 14:49

Mic92 reviewed Mar 21, 2019

View reviewed changes

TomSmeets suggested changes Mar 23, 2019

View reviewed changes

nixos/modules/services/web-servers/nginx/default.nix Outdated Show resolved Hide resolved

nixos/nginx: use writeNginxConfig

59fac1a

4z3 force-pushed the writeNginxConfig branch from 499c755 to 59fac1a Compare March 23, 2019 10:16

GrahamcOfBorg added 10.rebuild-linux: 1-10 and removed 10.rebuild-linux: 0 labels Mar 23, 2019

Mic92 merged commit 6dd7483 into NixOS:master Apr 4, 2019

Mic92 mentioned this pull request Apr 8, 2019

errounous semicolon after single-line location block slomkowski/nginx-config-formatter#16

Closed

Izorkin mentioned this pull request Apr 8, 2019

nixos/nginx: fix error in writeNginxConfig #59181

Merged

10 tasks

aanderse mentioned this pull request Apr 10, 2019

Nginx formatter #34985

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/nginx: use nginxfmt and gixy #57979

nixos/nginx: use nginxfmt and gixy #57979

4z3 commented Mar 20, 2019 •

edited

Mic92 Mar 21, 2019

Mic92 Mar 21, 2019

4z3 Mar 21, 2019

Mic92 Mar 21, 2019

acuteaura Feb 25, 2024

TomSmeets left a comment •

edited

Mic92 commented Mar 24, 2019

Izorkin commented Apr 8, 2019

Mic92 commented Apr 8, 2019

Izorkin commented Apr 8, 2019

Mic92 commented Apr 8, 2019

jb55 commented Apr 18, 2019

Izorkin commented Apr 19, 2019

jb55 commented Apr 20, 2019 •

edited

Mic92 commented Apr 23, 2019

nixos/nginx: use nginxfmt and gixy #57979

nixos/nginx: use nginxfmt and gixy #57979

Conversation

4z3 commented Mar 20, 2019 • edited

Motivation for this change

Things done

Mic92 Mar 21, 2019

Choose a reason for hiding this comment

Mic92 Mar 21, 2019

Choose a reason for hiding this comment

4z3 Mar 21, 2019

Choose a reason for hiding this comment

Mic92 Mar 21, 2019

Choose a reason for hiding this comment

acuteaura Feb 25, 2024

Choose a reason for hiding this comment

TomSmeets left a comment • edited

Choose a reason for hiding this comment

Mic92 commented Mar 24, 2019

Izorkin commented Apr 8, 2019

Mic92 commented Apr 8, 2019

Izorkin commented Apr 8, 2019

Mic92 commented Apr 8, 2019

jb55 commented Apr 18, 2019

Izorkin commented Apr 19, 2019

jb55 commented Apr 20, 2019 • edited

Mic92 commented Apr 23, 2019

4z3 commented Mar 20, 2019 •

edited

TomSmeets left a comment •

edited

jb55 commented Apr 20, 2019 •

edited