Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Enable testing in core scientific python libraries and move to github as src #65344

Closed
wants to merge 5 commits into from
Closed

[WIP] Enable testing in core scientific python libraries and move to github as src #65344

wants to merge 5 commits into from

Conversation

costrouc
Copy link
Member

@costrouc costrouc commented Jul 24, 2019 •
edited

Motivation for this change

There are several motivations for this PR.

Packages I am focusing on: numpy, scipy, numba, dask, sympy, matplotlib, and a few others.

  1. I've noticed that many of the core python scientific libraries in nix are not being currently tested and the derivations could be cleaned up
  2. (Selfish intention) Right now I am running downstream testing for the major python scientific libraries with hydra. https://hydra.aves.io/jobset/python-downstream-testing/master. I am testing to see if any commits on master from these major packages affect packages that depend on them. This is hard to do if the nixpkgs derivation is not based on the repository instead of sdist.

Questions/Opinions

  1. After doing some of this work. I am under the impression that if it is a well maintained project we should almost always prefer to use the git repository instead of the sdist. This guarentees tests being available and allows the user to override a package to a git commit that does not have a git tag.
  2. Is there a prefered amount of time that tests should run in? For example numba and sympy both take around an hour for the full test suite.
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member

FRidh commented Jul 24, 2019

  1. sounds good to me.
  2. when asking how much time can be spend on tests, ask yourself, how important is this package compared to the other packages we have and for how many people? What we want with these tests are basically sanity checks to see that we packaged things correctly. We should not run exhaustive test suites that contain tests for very specific issues.

jonringer
jonringer reviewed Aug 1, 2019
View reviewed changes
pkgs/development/python-modules/numba/default.nix
propagatedBuildInputs = [
numpy
llvmlite
] ++ stdenv.lib.optional (!isPy3k) funcsigs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python3.3 is not available in nixpkgs, should only need to check python2.7 vs python3.5+

Suggested change
] ++ stdenv.lib.optional (!isPy3k) funcsigs
] ++ stdenv.lib.optionals (!isPy3k) [ funcsigs singledispatch ];

@stale
Copy link

stale bot commented Jun 1, 2020

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the
    related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
    irc.freenode.net.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020
@ryantm ryantm marked this pull request as draft October 23, 2020 03:06
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Oct 23, 2020
@stale
Copy link

stale bot commented Apr 26, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonringer jonringer jonringer left review comments

@FRidh FRidh Awaiting requested review from FRidh

@infinisil infinisil Awaiting requested review from infinisil

@thoughtpolice thoughtpolice Awaiting requested review from thoughtpolice

Assignees
No one assigned
Labels
2.status: merge conflict 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: work-in-progress 6.topic: python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@costrouc @FRidh @jonringer @ryantm @SuperSandro2000