Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixos: kubernetes fixes #37218

Merged
merged 1 commit into from Apr 4, 2018
Merged

nixos: kubernetes fixes #37218

merged 1 commit into from Apr 4, 2018

Conversation

cstrahan
Copy link
Contributor

  • Fix reference CNI plugins

    • The plugins were split out of the upstream cni repo around version
      0.6.0

  • Fix RBAC and DNS tests

    • Fix broken apiVersion fields
    • Change plugin linking to look in ${package}/bin rather than
      ${package.plugins}

  • Initial work towards a working e2e test

    • Test still fails, but at least the expression evaluates now

Continues @srhb's work in #37199

Fixes #37199

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@cstrahan cstrahan mentioned this pull request Mar 17, 2018
Closed
8 tasks
@cstrahan cstrahan force-pushed the kube-test-fix branch 2 times, most recently from d69358f to e359258 Compare March 17, 2018 06:12
@cstrahan
Copy link
Contributor Author

@GrahamcOfBorg test kubernetes.dns kubernetes.rbac

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: tests.kubernetes.dns, tests.kubernetes.rbac

No log is available.

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: tests.kubernetes.dns, tests.kubernetes.rbac

No log is available.

@cstrahan
Copy link
Contributor Author

@grahamc Any idea why OfBorg is reporting success, while the logs show a 404 served up by nginx?

@grahamc
Copy link
Member

grahamc commented Mar 17, 2018

...yikes... this is ugly!

@grahamc
Copy link
Member

grahamc commented Mar 17, 2018

The problem is these evaluate to nothing:

[nix-shell:~/projects/nixpkgs]$ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate ./nixos/release.nix -A tests.kubernetes.rbac --option restrict-eval true --option build-timeout 1800 --arg supportedSystem '["x86_64-linux"]' --show-trace

[nix-shell:~/projects/nixpkgs]$ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate ./nixos/release.nix -A tests.kubernetes.dns --option restrict-eval true --option build-timeout 1800 --arg supportedSystem '["x86_64-linux"]' --show-trace

so evaluation "Succeeds" but then there is nothing to do. I'll write a test and fix this!

@grahamc grahamc mentioned this pull request Mar 17, 2018
Open
grahamc
grahamc reviewed Mar 17, 2018
View reviewed changes
nixos/release.nix Outdated
tests.kubernetes.dns = callSubTestsOnTheseSystems ["x86_64-linux"] tests/kubernetes/dns.nix {};
## kubernetes.e2e should eventually replace kubernetes.rbac when it works
#tests.kubernetes.e2e = callSubTestsOnTheseSystems ["x86_64-linux"] tests/kubernetes/e2e.nix {};
tests.kubernetes.rbac = callSubTestsOnTheseSystems ["x86_64-linux"] tests/kubernetes/rbac.nix {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note callSubTestsOnTheseSystems might need recurseForDerivations applied to it to make this work.

@cstrahan
Copy link
Contributor Author

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Partial log (click to expand)

Merge failed

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Partial log (click to expand)

Merge failed

@cstrahan
Copy link
Contributor Author

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:299:26

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:302:27

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:302:27

@GrahamcOfBorg
Copy link

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:299:26

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:302:27

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:302:27

@cstrahan
nixos: kubernetes fixes
709b6f6 
* Fix reference CNI plugins
  * The plugins were split out of the upstream cni repo around version
    0.6.0

* Fix RBAC and DNS tests
  * Fix broken apiVersion fields
  * Change plugin linking to look in ${package}/bin rather than
    ${package.plugins}

* Initial work towards a working e2e test
  * Test still fails, but at least the expression evaluates now

Continues @srhb's work in NixOS#37199

Fixes NixOS#37199
@cstrahan
Copy link
Contributor Author

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
�[31;1merror:�[0m attribute 'singlenode' in selection path 'tests.kubernetes.dns.singlenode' not found

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
�[31;1merror:�[0m attribute 'multinode' in selection path 'tests.kubernetes.rbac.multinode' not found

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
�[31;1merror:�[0m attribute 'singlenode' in selection path 'tests.kubernetes.rbac.singlenode' not found

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)

cannot build derivation '/nix/store/pkm7dq6b6qnsb8k46iywxqsjs7xcidgp-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/aawf1ly7j262rrlrxidrwvxh0p9mj283-nixos-test-driver-kubernetes-dns-multinode.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/jx3kc668x3ydr120k48c4w3d46yj09sx-nixos-test-driver-kubernetes-dns-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/wgv64py0y1rwclvv0jg3qcyfkbbyfsc2-nixos-test-driver-kubernetes-rbac-multinode.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/wj6qhqrqxy20lbbgfw2pcrdxdwjs61l3-nixos-test-driver-kubernetes-rbac-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/dc8zhjfjqylhmhqwkm7klnq5dhq33nzr-vm-test-run-kubernetes-dns-multinode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9k0ckg7fchcp0lscja73lamrpnprqr4m-vm-test-run-kubernetes-dns-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ykzh4p0ax100cm4clx77w563jzcsfq0r-vm-test-run-kubernetes-rbac-multinode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/kdbwqf3ynzlbrb6vqc9h39sywii6cbaz-vm-test-run-kubernetes-rbac-singlenode.drv': 1 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/9k0ckg7fchcp0lscja73lamrpnprqr4m-vm-test-run-kubernetes-dns-singlenode.drv', '/nix/store/dc8zhjfjqylhmhqwkm7klnq5dhq33nzr-vm-test-run-kubernetes-dns-multinode.drv', '/nix/store/kdbwqf3ynzlbrb6vqc9h39sywii6cbaz-vm-test-run-kubernetes-rbac-singlenode.drv', '/nix/store/ykzh4p0ax100cm4clx77w563jzcsfq0r-vm-test-run-kubernetes-rbac-multinode.drv' failed

@cstrahan
Copy link
Contributor Author

@grahamc Do you know why OfBorg is failing here?

qemu-system-x86_64: failed to initialize KVM: Permission denied

@7c6f434c
Copy link
Member

7c6f434c commented Mar 30, 2018 via email

@cstrahan
Copy link
Contributor Author

cstrahan commented Apr 4, 2018

The tests passed locally, so I'm going to stop sitting on this and merge.

@cstrahan cstrahan merged commit 5c066e2 into NixOS:master Apr 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grahamc grahamc grahamc left review comments

Assignees
No one assigned
Labels
6.topic: nixos 8.has: module (update) 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@cstrahan @GrahamcOfBorg @grahamc @7c6f434c