Github integration tests #179
Conversation
|
Good news: I managed to shove this test in a VM using nix-test.
Just need to find a way to run this in travis or hydra and we should be good. [edit] looks like vim is getting crazy and is inserting tabs a bit everywhere, I'll fix this before merging. Sorry about that... |
Small CI updateSo, I've been trying to run this on both Travis and CircleCI. TravisThe VM successfully boots up, but for some reason, the rabbitmq service is not able to start. Can't say if this is related to this or if it's a weird behavior triggered by the odd setup (qemu machine in travis). Full log here. We were so close, it's kind of frustrating... Circle CIThe VM can't even boot up, it seems unable to mount the vda disk. I guess it's a dead end, full log here. Hydra?I guess the only remaining solution is Hydra. Sadly, hydra can't build PR so far, which kind of negate this kind of e2e test purpose. Ofborg Itself@LnL7 mentionned this on IRC. This is probably our best shot actually. However, we'll need a nixpkgs checkout along with this repo to build the tests. Any suggestions? |
|
@NinjaTrappeur Hydra can build PRs, we're building our PRs with hydra for around two years. See relevant module https://github.com/NixOS/hydra/blob/master/src/lib/Hydra/Plugin/GithubPulls.pm. You can also find an example here written for demonstration purposes https://github.com/aycanirican/declarative-hydra-test. |
| @@ -65,4 +65,6 @@ in { | |||
| cd .. | |||
| mv ofborg $out | |||
| ''; | |||
|
|
|||
| ofborg.integrationTests = pkgs.callPackage ./e2e-tests/default.nix {}; | |||
There was a problem hiding this comment.
This should probably be moved to release.nix, that's what hydra uses.
| @@ -0,0 +1,33 @@ | |||
| import <nixpkgs/nixos/tests/make-test.nix> ({ pkgs, ...}: rec { | |||
There was a problem hiding this comment.
This uses nixpkgs from NIX_PATH instead of the version used by the build itself and won't work on hydra, depending on the jobset configuration.
|
|
||
| # Kind of a dirty hack... We call evaluation-filter to create | ||
| # the exchange/queue routes. | ||
| evaluation-filter "${configPath}" & |
There was a problem hiding this comment.
This entire script doesn't do much other than calling ofborg binaries, using a rust binary for the test itself would enable sharing code for setup like this, etc.
|
Just stumbled upon that one. Reading this 4 years later, this PR sounds out of place to me. Closing it. |
Hi,
As previously discussed in the IRC room, I have been working on a GitHub E2E test.
I've got a first prototype, this is not ready to merge yet. I mostly open this PR to ask some questions.
Here's what I still need to do before considering this PR as complete.
Provisionning/Run the testing environment
I am still a rookie regarding nix and never used it to provision a testing environment. I saw the nix-test chapter in the nixos manual and thought it would be a good candidate. However, I was unable to find an example of a nix-test use outside of the nixpkgs repository.
We basically just need Ofborg's rust application together with rabbitmq to conduct those tests.
So, dear nix power users, here are the actual questions :) How are you usually writing your E2E tests while using nix? What do you think about /lib/testing.nix? Is there a similar yet better way to generate those tests?
I am open to any suggestions!