Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest: introduce runTests function #53142

Closed
wants to merge 2 commits into from
Closed

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Jan 1, 2019

Motivation for this change

A simple function that generates a checkPhase. It includes a
disabledTests parameter to conveniently disable tests.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@grahamc
Copy link
Member

grahamc commented Jan 1, 2019

@GrahamcOfBorg eval

@grahamc
Copy link
Member

grahamc commented Jan 1, 2019

Ofborg's new maintainer ping algorithm would have requested a review from:

  • @nixy for pkgs.python27Packages.astor, pkgs.python37Packages.astor, pkgs.python27Packages.astor, pkgs.python37Packages.astor, pkgs.python27Packages.astor, pkgs.python27Packages.astor, pkgs.python37Packages.astor, pkgs.python37Packages.astor
  • @knedlsepp for pkgs.python37Packages.pandas, pkgs.python27Packages.pandas, pkgs.python27Packages.pandas, pkgs.python37Packages.pandas, pkgs.python37Packages.pandas, pkgs.python37Packages.pandas, pkgs.python27Packages.pandas, pkgs.python27Packages.pandas
  • @FRidh for pkgs.python37Packages.datashape, pkgs.python37Packages.datashape, pkgs.python37Packages.pandas, pkgs.python27Packages.traitlets, pkgs.python37Packages.traitlets, pkgs.python27Packages.pandas, pkgs.python37Packages.traitlets, pkgs.python37Packages.traitlets, pkgs.python27Packages.traitlets, pkgs.python27Packages.pandas, pkgs.python27Packages.datashape, pkgs.python27Packages.traitlets, pkgs.python37Packages.pandas, pkgs.python37Packages.datashape, pkgs.python37Packages.pandas, pkgs.python37Packages.datashape, pkgs.python27Packages.traitlets, pkgs.python37Packages.pandas, pkgs.python27Packages.datashape, pkgs.python27Packages.pandas, pkgs.python27Packages.datashape, pkgs.python37Packages.traitlets, pkgs.python27Packages.pandas, pkgs.python27Packages.datashape
  • @zimbatm for pkgs.python37Packages.pip-tools, pkgs.python27Packages.pip-tools, pkgs.python37Packages.pip-tools, pkgs.python27Packages.pip-tools, pkgs.python37Packages.pip-tools, pkgs.python27Packages.pip-tools, pkgs.python37Packages.pip-tools, pkgs.python27Packages.pip-tools
  • @7c6f434c for pkgs.python37Packages.pandas, pkgs.python27Packages.pandas, pkgs.python27Packages.pandas, pkgs.python37Packages.pandas, pkgs.python37Packages.pandas, pkgs.python37Packages.pandas, pkgs.python27Packages.pandas, pkgs.python27Packages.pandas
  • @catern for pkgs.python37Packages.trio, pkgs.python37Packages.trio, pkgs.python37Packages.trio, pkgs.python37Packages.trio

does this seem reasonable? please discuss on NixOS/ofborg#293

A simple function that generates a `checkPhase`. It includes a
`disabledTests` parameter to conveniently disable tests.
disabledTestIds ? [], # Disable tests that match given node id's. This is useful in case a test in a specific file needs to be disabled.
options ? [], # py.test options.
targets ? [], # py.test files or folders.
variables ? {}, # Environment variables to export.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variables is a bit ambiguities. How about environment or environmentVariables?

@FRidh
Copy link
Member Author

FRidh commented Jan 2, 2019

runTests returns a string, and can therefore unfortunately not be overridden. If, instead, buildPython* would accept a parameter, say, checkConfig, whose value corresponds to an attribute set, then it would be possible to write:

buildPythonPackage {
  ...

  checkConfig = {
    runner = pytest;
    variables.FOO = "bar";
    disabledTests = [
      "test_this"
    ];
  };
}

where the runner gets to decide how to invoke the test.
I don't want to over-engineer it (mainly just have a solution for disabledTests), but considering we have different test runners it may be convenient. At the same time, nose is being used fewer and fewer.

@Mic92
Copy link
Member

Mic92 commented Jan 2, 2019

Just two points here:

  1. Was there a particular reason not to use a build-support hook? This would allow overrideAttrs to work as expected when disabledTests becomes an attribute of the derivation.
  2. Can we get documentation for this function in the manual?

@FRidh
Copy link
Member Author

FRidh commented Jan 2, 2019

Was there a particular reason not to use a build-support hook? This would allow overrideAttrs to work as expected when disabledTests becomes an attribute of the derivation.

Regarding hooks, there are two options:

  1. hook exported by the derivation, pytest, itself.
  2. a separate hook, say pytestRunner.

I don't think 1) is a good solution because pytest may be just some dependency. Using a specific attribute for it would solve that (mypkg.testRunner = pytest;). At that point one might as well go for 2).
Instead of adding new top-level attributes (such as say disabledTests) I prefer to put everything test-related in its own namespace. Are attribute sets exported as associative arrays?

Can we get documentation for this function in the manual?

Of course, once we settle on the solution.

@mmahut
Copy link
Member

mmahut commented Aug 19, 2019

Are there any updates on this pull request, please?

@FRidh
Copy link
Member Author

FRidh commented Aug 19, 2019

Will continue with hook approach.

@FRidh FRidh closed this Aug 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants