WIP: buildPythonPackage: perform tests in separate derivation #90240
Conversation
Experiment to see what is needed to perform tests in a separate
derivation.
$ nix-build -A python3.pkgs.pytest.tests.installation
Ideally, I would like a Python build to consist of the following three
builds:
1) build wheel
2) install wheel
3) test installation
where 3) is optional.
Going there we should probably change the interface significantly,
specifically, we should nomenclature for inputs common to Python users.
All the check related attributes are probably best put in an attribute
set as well, but again, that is a big change in interface.
| @@ -163,15 +157,42 @@ let | |||
| # Python packages built through cross-compilation are always for the host platform. | |||
| disallowedReferences = lib.optionals (python.stdenv.hostPlatform != python.stdenv.buildPlatform) [ python.pythonForBuild ]; | |||
|
|
|||
| passthru = passthru // { | |||
| tests.installation = stdenv.mkDerivation { | |||
There was a problem hiding this comment.
maybe better to include the whole self here stdenv.mkDerivatoin (self // {... })
| nativeBuildInputs = [ | ||
| self | ||
| ] ++ lib.optionals (format == "setuptools") [ | ||
| # Longer-term we should get rid of this and require | ||
| # users of this function to set the `installCheckPhase` or | ||
| # pass in a hook that sets it. | ||
| #setuptoolsCheckHook | ||
| ] ++ checkInputs ++ propagatedBuildInputs; |
There was a problem hiding this comment.
it might be time to just drop this altogether, already commented it out:
| nativeBuildInputs = [ | |
| self | |
| ] ++ lib.optionals (format == "setuptools") [ | |
| # Longer-term we should get rid of this and require | |
| # users of this function to set the `installCheckPhase` or | |
| # pass in a hook that sets it. | |
| #setuptoolsCheckHook | |
| ] ++ checkInputs ++ propagatedBuildInputs; | |
| nativeBuildInputs = [ | |
| self | |
| ] ++ checkInputs ++ propagatedBuildInputs; |
|
having some tests I think is really beneficial for checking for regressions, and it pairs will with I think having separate derivations for tests might be really useful for something like pytorch, in which the test suite is very resource and time intensive. But, only ofborg is realistically going to run it, maybe if Another idea is to have two checkPhases, one that gets ran with the build, and an optional longer test which gets added to I might also be missing what a separate test derivation is trying to solve. |
|
Having the build succeed implying that the tests passed is quite convenient, but perhaps I'm also missing the motivation. One thing that would be useful is downloading a bunch of packages from Hydra but re-running the tests locally (but without recompiling) for packages that use CUDA or CPU-specific features. Perhaps only vaguely related, but one thing I've wished for is an |
|
I marked this as stale due to inactivity. → More info |
stale
bot
added
the
2.status: stale
Experiment to see what is needed to perform tests in a separate
derivation.
Ideally, I would like a Python build to consist of the following three
builds:
where 3) is optional.
Going there we should probably change the interface significantly,
specifically, we should nomenclature for inputs common to Python users.
All the check related attributes are probably best put in an attribute
set as well, but again, that is a big change in interface.
Motivation for this change
Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)nix path-info -Sbefore and after)