Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubernetes: 1.9.1 -> 1.10 #38445

Closed
wants to merge 1 commit into from
Closed

kubernetes: 1.9.1 -> 1.10 #38445

wants to merge 1 commit into from

Conversation

cstrahan
Copy link
Contributor

@cstrahan cstrahan commented Apr 5, 2018

Motivation for this change

Updates to latest kubernetes release.

The patches were removed, as they're now applied in the latest release.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

/cc @srhb @matejc @offlinehacker

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: kubernetes

Partial log (click to expand)

strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
patching script interpreter paths in /nix/store/imw8pp0hy7s96p8isysx6a9m0b5nq3hy-kubernetes-1.10.0-man
checking for references to /tmp/nix-build-kubernetes-1.10.0.drv-0 in /nix/store/imw8pp0hy7s96p8isysx6a9m0b5nq3hy-kubernetes-1.10.0-man...
shrinking RPATHs of ELF executables and libraries in /nix/store/w12zjqh89jjgi88ylgypj1q99l64l0mh-kubernetes-1.10.0-pause
shrinking /nix/store/w12zjqh89jjgi88ylgypj1q99l64l0mh-kubernetes-1.10.0-pause/bin/pause
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/w12zjqh89jjgi88ylgypj1q99l64l0mh-kubernetes-1.10.0-pause/bin
patching script interpreter paths in /nix/store/w12zjqh89jjgi88ylgypj1q99l64l0mh-kubernetes-1.10.0-pause
checking for references to /tmp/nix-build-kubernetes-1.10.0.drv-0 in /nix/store/w12zjqh89jjgi88ylgypj1q99l64l0mh-kubernetes-1.10.0-pause...
/nix/store/qpgj30za0bxjj0fmph69zgxrvyays02s-kubernetes-1.10.0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: kubernetes

Partial log (click to expand)

strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
patching script interpreter paths in /nix/store/ffidncjv7pqsnbf7ixhz6w8ffxy8fjip-kubernetes-1.10.0-man
checking for references to /build in /nix/store/ffidncjv7pqsnbf7ixhz6w8ffxy8fjip-kubernetes-1.10.0-man...
shrinking RPATHs of ELF executables and libraries in /nix/store/513qksh2srrqamcjs433kzsyh8r599fp-kubernetes-1.10.0-pause
shrinking /nix/store/513qksh2srrqamcjs433kzsyh8r599fp-kubernetes-1.10.0-pause/bin/pause
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/513qksh2srrqamcjs433kzsyh8r599fp-kubernetes-1.10.0-pause/bin
patching script interpreter paths in /nix/store/513qksh2srrqamcjs433kzsyh8r599fp-kubernetes-1.10.0-pause
checking for references to /build in /nix/store/513qksh2srrqamcjs433kzsyh8r599fp-kubernetes-1.10.0-pause...
/nix/store/hfx0r8g5nifxbzvzn9yzbq93avdv8ga6-kubernetes-1.10.0

@Mic92 Mic92 changed the title kubernetes: update to 1.10 kubernetes: 1.9.1 -> 1.10 Apr 5, 2018
@steveej
Copy link
Contributor

steveej commented Apr 25, 2018

I'm wondering why this PR is not moving, it seems to look fine.
If someone tells me how to run the testsuite for kubernetes I will give it a try too.

@srhb
Copy link
Contributor

srhb commented Apr 25, 2018 •
edited

Sorry, I completely forgot I had been pinged for review. I am not in a position to actually test kubernetes anymore, but the changes look sane to me and I am very happy to see the flags moving into a config file!

@steveej
Copy link
Contributor

steveej commented May 1, 2018

The tests seem to run indefinitely without success, including the singlenode ones.
In order to make it work I have tried to bump kubernetes to v1.10.2.

  • I spotted an issue with etcd and TLS and therefore bumped etcd to 3.3.4 as it seemed to have been an upstream issue. The etcd/TLS issue went away, although many other erros remained.
  • One log line says that the program conntrack is missing

At this point I can't tell if it's a misconfiguration or upstream issues that prevent the setup from working, I'm afraid I have to leave this one to the more experienced, as I don't have that much spare time to debug this.

My changes and the logs of running the dns.singlenode tests are in a gist: https://gist.github.com/steveeJ/8abeb3c02a85aeeb1113969bec84541b

@yurrriq yurrriq mentioned this pull request May 14, 2018
Merged
8 tasks
@azazel75
Copy link
Contributor

@steveej where you got the infos about how to run the testsuite?

@steveej
Copy link
Contributor

steveej commented May 20, 2018 •
edited

@azazel75 it's part of the nixos test expressions, which I wasn't familiar with at that point.
Within the nixpkgs repository, you can run it as follows:

nix-build nixos/tests/kubernetes -A dns.singlenode

I can't reproduce my above result at the moment, instead I'm getting this error:

error: anonymous function at /home/steveej/src/github/NixOS/nixpkgs/pkgs/build-support/docker/default.nix:38:5 called without required argument 'imageDigest', at /home/steveej/src/github/NixOS/nixpkgs/nixos/modules/services/cluster/kubernetes/dashboard.nix:11:11
(use '--show-trace' to show detailed location information)

@srhb
Copy link
Contributor

srhb commented May 20, 2018

@steveej 7368487 introduced a new and improved function to pull images, but it requires different arguments (among others, imageDigest)

The tests will need to be updated accordingly. It shouldn't be too hard.

@johanot
Copy link
Contributor

johanot commented May 23, 2018

@steveej and others. Have a look at my branch here: https://github.com/johanot/nixpkgs/tree/kube-1.10 .. I did the following:

  1. rebase with master
  2. updated to kubernetes v1.10.3
  3. updated dockerTools.pullImage statements to pull by digest as @srhb pointed out
  4. updated kube-dns and kube-dashboard revisions
  5. fixed (what seemed to be) the actual problem with the test cases. Various kube components were running out of memory while the tests were running. johanot@7a83df0

Can you verify that the test cases run clean for you after pulling my changes? If so, we can start squashing/cherry-picking. :)

@azazel75
Copy link
Contributor

azazel75 commented May 23, 2018 •
edited

Ah, @johanot , you beat me! I too have a similar branch here https://github.com/azazel75/nixpkgs/tree/kube-1.10 . All the enabled tests pass.

From what I can see, the differences between ou two branches are the follow:

  • You've set VMs memory to 2GB and I to 1.5GB
  • I've updated not only the addons versions but also the difinition jsons as they are shipped in 1.10.3
  • I've kept the etcd update from @steveej patch

@johanot
Copy link
Contributor

johanot commented May 23, 2018

@azazel75 Great minds think alike :-) And thanks for updating the manifests for the addons. I was too lazy, apparently.

I agree with your implementation as well, so I think if we can get @steveej to also approve, we can get this PR finally flying.

@azazel75
Copy link
Contributor

Great! Just to know, what's the policy for the update of packages in 18.03 in Nixos? i.e. will this land on 18.03 or I will have to backport it myself? Keeping a server on unstable is a bad idea :-)

@srhb
Copy link
Contributor

srhb commented May 23, 2018 •
edited

@azazel75 I am not keen on backporting what is essentially a major version in Kubernetes' terms to stable.

You can however pull in these changes on top of your existing 18.03 server.

@johanot
Copy link
Contributor

johanot commented May 23, 2018

@azazel75 What @srhb says... and... This doesn't mean that you have to keep your server on unstable. Just port the changes into a custom module + overlay in your configuration until 18.09 becomes stable.

@steveej
Copy link
Contributor

steveej commented May 23, 2018

@johanot @azazel75 this is great news, thanks to both of you!
I ran the dns and rbac tests locally against @johanot's kube-1.10 branch with success.

@cstrahan would it be possible for you to incorporate the changes discussed above or would you prefer to close this PR?

@cstrahan
Copy link
Contributor Author

cstrahan commented May 25, 2018 via email

@johanot johanot mentioned this pull request May 25, 2018
Merged
8 tasks
@srhb
Copy link
Contributor

srhb commented May 25, 2018

Closing this in favour of #41073. Thanks for the effort everyone!

@srhb srhb closed this May 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
2.status: work-in-progress 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@cstrahan @GrahamcOfBorg @steveej @srhb @azazel75 @johanot @xeji