Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/test: added verbose output for failed tests #77662

Merged
merged 1 commit into from Jan 17, 2020

Conversation

marijanp
Copy link
Contributor

@marijanp marijanp commented Jan 14, 2020

Motivation for this change

While porting some tests to use the new python test-driver, some of the subtests failed. Finding the exception and the respective subtest was annoyjing for me, since there is a lot of other output printed to the terminal. At a NixOS meetup @0x4A6F approached me and asked me if I could make some changes in the test-driver, such that the failed tests are shown in a clear way.

This was the output before the changes:

1 out of 2 tests succeeded
cleaning up
killing machine (pid 6)

This is the new output:

1 out of 2 tests succeeded
The following tests have failed:
 - Test "root fs is fsckd" failed with error: "Failing for demonstration purposes"
cleaning up
killing machine (pid 6)

I'm sure that there are a few more improvements that can be done to make the output uncluttered. Feel free to comment your ideas.

@tfc @flokli @worldofpeace #72828

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • [x ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@worldofpeace
Copy link
Contributor

Yes yes yes 🎊
I think I've might've also complained in the IRC channel about this, before it was soo difficult to debug tests that have giant outputs (like finding a needle in the haystack). I will try this out.

@0x4A6F
Copy link
Member

0x4A6F commented Jan 16, 2020

Nice and looks good for me, thanks for implementing this.

@flokli flokli merged commit ed0b5b6 into NixOS:master Jan 17, 2020
@marijanp marijanp deleted the verbose-python-test-driver branch January 17, 2020 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants