Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nomad: 0.12.2 -> 0.12.3 #96414

Merged
merged 2 commits into from Sep 9, 2020
Merged

nomad: 0.12.2 -> 0.12.3 #96414

merged 2 commits into from Sep 9, 2020

Conversation

amaxine
Copy link
Member

@amaxine amaxine commented Aug 26, 2020

Motivation for this change

Bumping to 0.12.3

Because of #95885 this should also redefine both versions of nomad to build with go 1.14, as that's the expectation from upstream, but I'm unsure on how to do that.
I understand that usually I'd be able to just specify buildGoPackage = buildGo114Package, but because it was requested the individual versions are implemented as files, I'm not entirely sure how to achieve that - any help would be appreciated.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@zowoq
Copy link
Contributor

zowoq commented Aug 26, 2020

Because of #95885 this should also redefine both versions of nomad to build with go 1.14, as that's the expectation from upstream

Unless it actually requires go 1.14 for some reason I'd suggest not changing it.

@endocrimes
Copy link
Member

endocrimes commented Aug 26, 2020

@zowoq Fwiw there have been a bunch of issues in the past when they diverge due to subtle changes in the Go standard library, especially around underlying platform support. This is especially true of a release like go1.15 that changes a lot of networking, cryptography, and introduces new panics (e.g changes to context.WithXXX). I'd very much like to see us do the right thing here and mirror upstreams go version.

@zowoq
Copy link
Contributor

zowoq commented Aug 26, 2020

I'd like to see a reason beyond it's what upstream is using for this to be pinned.

@endocrimes
Copy link
Member

@zowoq (For reference, I'm the maintainer of this package and a former member of the Nomad Core Team). Nomad is critical infrastructure software with a lot of security sensitive aspects, building it against an untested and unsupported go version can cause a lot of subtle issues that aren't going to be covered by any testing we can do in NixOS, and can make it significantly harder for the upstream to reproduce when they occur on a different go version to what they are currently shipping with. This has happened in the past, and I have (personally) lost days of my life to tracking those issues down.

Even ignoring the issues related to making upstream maintainers lives more difficult, go1.15 includes a lot of changes that could potentially cause issues with Nomad as I previously stated. Publishing untested software combinations to our users is actively pretty dangerous for these categories of applications.

@worldofpeace
Copy link
Contributor

Hi, pinning packages to a specific version for like a compiler is something that is done soo very commonly in nixpkgs that I don't ever think about it. Plus, there is no downside to this for a leaf package that I can think of. As long as it's documented why in the expression so people know why in the future go crazy.

I understand that usually I'd be able to just specify buildGoPackage = buildGo114Package, but because it was requested the individual versions are implemented as files, I'm not entirely sure how to achieve that - any help would be appreciated.

I'm not sure I understand this. do u mean u want 0.12 and 0.11 to both do that but in one place? Because that would be to just change buildGoPackage to buildGo114Package in the generic expression.

@zowoq
Copy link
Contributor

zowoq commented Aug 26, 2020

pinning packages to a specific version for like a compiler is something that is done soo very commonly in nixpkgs that I don't ever think about it

I think it is relatively uncommon in go packages compared to the rest of nixpkgs.

@worldofpeace
Copy link
Contributor

pinning packages to a specific version for like a compiler is something that is done soo very commonly in nixpkgs that I don't ever think about it

I think it is relatively uncommon in go packages compared to the rest of nixpkgs.

I recall prs before and after 20.03 branched off where we had to cleanup old go overrides, and I think lots of prs from @kalbasit to make programs that break with newer versions to use old ones (or vice versa). What is uncommon doesn't mean it won't or could be necessary.

@kalbasit
Copy link
Member

I don't have a problem myself with using a different version of Go as long as it's explained with a comment.

@JJJollyjim
Copy link
Member

The bump to go 1.15 broke lots of stuff (e.g. syncthing and ipfs panicked on startup, kubernetes can't make SSL certs and start, the acme nixos-test broke). The go community apparently does not expect or plan for compiler version bumps, and it's absolutely a good idea to be more cautious about it like this.

@amaxine
Copy link
Member Author

amaxine commented Aug 27, 2020

@worldofpeace Thanks, that solution works. I was looking for a way to redefine it here

nomad = nomad_0_11;
nomad_0_11 = callPackage ../applications/networking/cluster/nomad/0.11.nix { };
nomad_0_12 = callPackage ../applications/networking/cluster/nomad/0.12.nix { };

as it's not guaranteed that all major releases in nix would be on the same version of go. But that can be left for when 0.13 is out.

I'll change generic.nix to just use 1.14 for now, and add an appropriate comment in the file in a bit.

@endocrimes
Copy link
Member

Bumped this PR to include a comment about version pinning and started pinning go versions.

@zowoq
Copy link
Contributor

zowoq commented Sep 9, 2020

Can you include a link that lists what go version upstream is using for each version please?

@endocrimes
Copy link
Member

@zowoq
Copy link
Contributor

zowoq commented Sep 9, 2020

Sorry, I meant some more like documentation for downstream builders with the supported go versions?

And can you include with commit pinning the versions?

@endocrimes
Copy link
Member

@zowoq There isn't any - Nomad for the most part expects people to be using binary builds produced via HashiCorp's release infrastructure. The only exceptions to that are the Debian, Void, and Nix packages that I've seen.

The commit references this PR, so I don't think we really need to amend it into there tbh.

This commit pins go versions for nomad 0.11 and 0.12. Future versions of
Nomad should have their versions pinned from the beginning, even if they
support the latest-at-the-time version of Go to prevent accidental
version bumps on unsupported go versions.

See #96414 for further discussion
around this change.
@worldofpeace worldofpeace merged commit 4a3e33a into NixOS:master Sep 9, 2020
@amaxine amaxine deleted the nomad_0.12.3 branch September 10, 2020 08:43
jonringer pushed a commit to jonringer/nixpkgs that referenced this pull request Dec 17, 2020
This commit pins go versions for nomad 0.11 and 0.12. Future versions of
Nomad should have their versions pinned from the beginning, even if they
support the latest-at-the-time version of Go to prevent accidental
version bumps on unsupported go versions.

See NixOS#96414 for further discussion
around this change.

(cherry picked from commit 329a922)
jonringer pushed a commit that referenced this pull request Dec 17, 2020
This commit pins go versions for nomad 0.11 and 0.12. Future versions of
Nomad should have their versions pinned from the beginning, even if they
support the latest-at-the-time version of Go to prevent accidental
version bumps on unsupported go versions.

See #96414 for further discussion
around this change.

(cherry picked from commit 329a922)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants