Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/caddy: add support for caddy v2 (take 2) #97217

Merged
merged 7 commits into from Sep 8, 2020

Conversation

sephii
Copy link
Contributor

@sephii sephii commented Sep 5, 2020

Motivation for this change

The caddy module allows you to use Caddy 2 but it's broken (#92992). This PR is a new take at #86686, which has been dormant for a few weeks now. Since I'd really like to get Caddy 2 support in 20.09 I'm opening a new PR with the requested changes.

The tests pass when running the tests interactively (nix-build caddy.nix -A driver and then start_all() followed by test_script()), but they fail when running them non-interactively (nix-build caddy.nix) because the test directory created in the test cannot be found. Since I can't figure out why, any help is appreciated. (fixed!)

I would also like to bump the version to 2.1.1 instead of 2.0, but doing it gives me some go errors I don't know how to fix (eg. gopkg.in/yaml.v2@v2.3.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt), so if someone knows how to make it work, it would be nice. (fixed!)

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

pkgs/servers/caddy/default.nix Outdated Show resolved Hide resolved
@Br1ght0ne
Copy link
Member

Thanks @sephii, I fully support your effort. I'll take a look at the tests.

@sephii sephii mentioned this pull request Sep 7, 2020
Br1ght0ne
Br1ght0ne previously approved these changes Sep 7, 2020
Copy link
Member

@Br1ght0ne Br1ght0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides failing tests, LGTM

@Br1ght0ne Br1ght0ne dismissed their stale review September 7, 2020 13:05

Forgot about failing tests

@KamilaBorowska
Copy link
Member

@ofborg test caddy

@KamilaBorowska
Copy link
Member

KamilaBorowska commented Sep 8, 2020

From what I can tell, the issue is that Nix doesn't recognize paths in a configuration file as being inputs. caddy-config.json looks like this:

{
    "apps": {
        "http": {
            "servers": {
                "srv0": {
                    "automatic_https": {
                        "skip": [
                            "localhost"
                        ]
                    },
                    "listen": [
                        ":80"
                    ],
                    "routes": [
                        {
                            "handle": [
                                {
                                    "handler": "subroute",
                                    "routes": [
                                        {
                                            "handle": [
                                                {
                                                    "handler": "vars",
                                                    "root": "/nix/store/496k1zx0li2mp94xnydn2smk1gbybw2n-testdir"
                                                },
                                                {
                                                    "encodings": {
                                                        "gzip": {}
                                                    },
                                                    "handler": "encode"
                                                },
                                                {
                                                    "handler": "file_server",
                                                    "hide": [
                                                        "/nix/store/69yh559dagwrhfa8bahpm9v99sd2qkg8-Caddyfile"
                                                    ]
                                                }
                                            ]
                                        }
                                    ]
                                }
                            ],
                            "match": [
                                {
                                    "host": [
                                        "localhost"
                                    ]
                                }
                            ],
                            "terminal": true
                        }
                    ]
                }
            }
        },
        "tls": {
            "automation": {
                "policies": [
                    {
                        "issuer": {
                            "ca": "https://acme-v02.api.letsencrypt.org/directory",
                            "email": "",
                            "module": "acme"
                        }
                    }
                ]
            }
        }
    }
}

Neither /nix/store/496k1zx0li2mp94xnydn2smk1gbybw2n-testdir nor /nix/store/69yh559dagwrhfa8bahpm9v99sd2qkg8-Caddyfile actually exist on a virtual machine.

This likely occurs due to importJSON here. Merging JSON files should likely be done as a derivation (probably using jq), not from Nix language.

  adaptedConfig = importJSON (pkgs.runCommand "caddy-config-adapted.json" { } ''
    ${cfg.package}/bin/caddy adapt \
      --config ${configFile} --adapter ${cfg.adapter} > $out
  '');
  configJSON = pkgs.writeText "caddy-config.json" (builtins.toJSON
    (recursiveUpdate adaptedConfig tlsConfig));

};
}];
};
adaptedConfig = importJSON (pkgs.runCommand "caddy-config-adapted.json" { } ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid use of importJSON here and merge JSON files in a derivation. The following should work.

  adaptedConfig = pkgs.runCommand "caddy-config-adapted.json" { } ''
    ${cfg.package}/bin/caddy adapt \
      --config ${configFile} --adapter ${cfg.adapter} > $out
  '';
  tlsJSON = pkgs.writeText "tls.json" (builtins.toJSON tlsConfig);
  configJSON = pkgs.runCommand "caddy-config.json" { } ''
    ${pkgs.jq}/bin/jq -s '.[0] * .[1]' ${adaptedConfig} ${tlsJSON} > $out
  '';

@sephii
Copy link
Contributor Author

sephii commented Sep 8, 2020

The tests should now pass, thanks for the help!

@KamilaBorowska
Copy link
Member

@ofborg test caddy

@ofborg ofborg bot requested a review from Br1ght0ne September 8, 2020 07:45
@Br1ght0ne Br1ght0ne merged commit 45d7f59 into NixOS:master Sep 8, 2020
@sephii sephii deleted the nixos-caddy-v2-migration branch September 8, 2020 08:27
@lf-
Copy link
Member

lf- commented Sep 9, 2020

I don't think this is a regression caused by this PR, but for some reason, it's emitting a StartLimitIntervalSec in the wrong section (should be under [Unit] as of some time in 2017 according to the mailing list).

Here's the generated service file:

[Unit]
After=network-online.target
Description=Caddy web server
Wants=network-online.target

[Service]
# ... snip ...
StartLimitIntervalSec=14400
TimeoutStopSec=5s
Type=simple
User=caddy

@sephii
Copy link
Contributor Author

sephii commented Sep 9, 2020

You're right, it should be moved to the Unit section along with StartLimitBurst. I think you should file an issue, since this problem is present in other services (grepping for StartLimitIntervalSec and StartLimitBurst reveals that other modules are using these options, and that they're also in the Service section).

@lf-
Copy link
Member

lf- commented Sep 9, 2020

@sephii I should file a PR and fix all of them as it's relatively trivial. ~~Not sure when I'll have time to do that though.~~Well, the following bug is a blocker on my work, so I guess I have time now!

@lf-
Copy link
Member

lf- commented Sep 9, 2020

In terms of actual bugs in this PR though:

@sephii

There's a bug introduced in this PR where we are doing a bit of hamfisted overwriting of the tls key in our generated configs, breaking tls internal settings in the config section. Here's a repro:

{pkgs, ...}: {
    services.caddy.package = pkgs.caddy;
    services.caddy.enable = true;
    services.caddy.agree = true;
    services.caddy.email = "letsencrypt@catgirl.enterprises";
    services.caddy.ca = "https://acme-staging-v02.api.letsencrypt.org/directory";
    services.caddy.config = ''
      test.catgirl.enterprises:443 {
        encode zstd gzip
        log
        tls internal
        reverse_proxy localhost:3000
      }
    '';
}

is making the following bad config:

{
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [
            ":443"
          ],
          "routes": [
            {
              "match": [
                {
                  "host": [
                    "test.catgirl.enterprises"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "handle": [
                        {
                          "encodings": {
                            "gzip": {},
                            "zstd": {}
                          },
                          "handler": "encode"
                        },
                        {
                          "handler": "reverse_proxy",
                          "upstreams": [
                            {
                              "dial": "localhost:3000"
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ],
              "terminal": true
            }
          ],
          "logs": {}
        }
      }
    },
    "tls": {
      "automation": {
        "policies": [
          {
            "issuer": {
              "ca": "https://acme-staging-v02.api.letsencrypt.org/directory",
              "email": "letsencrypt@catgirl.enterprises",
              "module": "acme"
            }
          }
        ]
      }
    }
  }
}

Compare the tls section to what caddy adapt produces for the equivalent caddyfile with the content of config:

{
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [
            ":443"
          ],
          "routes": [
            {
              "match": [
                {
                  "host": [
                    "test.catgirl.enterprises"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "handle": [
                        {
                          "encodings": {
                            "gzip": {},
                            "zstd": {}
                          },
                          "handler": "encode"
                        },
                        {
                          "handler": "reverse_proxy",
                          "upstreams": [
                            {
                              "dial": "localhost:3000"
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ],
              "terminal": true
            }
          ],
          "logs": {}
        }
      }
    },
    "tls": {
      "automation": {
        "policies": [
          {
            "subjects": [
              "test.catgirl.enterprises"
            ],
            "issuer": {
              "module": "internal"
            }
          }
        ]
      }
    }
  }
}

@lf-
Copy link
Member

lf- commented Sep 9, 2020

Update: it looks like it's that for some reason jq is not merging arrays while doing a * recursive merge. I don't know enough jq to fix it though, sorry. I'll instead go fix the systemd unit option issues [EDIT: done this fix, see #97512] :)

@sephii
Copy link
Contributor Author

sephii commented Sep 9, 2020

I think the following jq should do the trick: jq -s '.[0] * {"apps": {"tls": {"automation": {"policies": (if .[0].apps.tls.automation.policies == .[1]?.apps.tls.automation.policies then .[0].apps.tls.automation.policies else (.[0].apps.tls.automation.policies + .[1]?.apps.tls.automation.policies) end)}}}}'. I don't know jq either so there might be a better way to do it. Can you please try and let me know, so I can submit a patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants