Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update-python-libraries: add github fetcher #99635

Merged
merged 1 commit into from Oct 6, 2020

Conversation

jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 5, 2020

Motivation for this change

was tired of it only supporting pypi

I use the nix eval -f <NIXPKGS_ROOT>/default.nix python3Packages.<package>.src.meta.homepage to determine the owner and repo. NIXPKGS_ROOT is determined from script location, not an environment variable though.

Some limitations:

  • If the attr name and directory name differ, it will fail the update
  • can't handle "unstable" checkouts due to version logic not being able to handle it ('2018-08-22' vs '1.2.1')?
  • If a package is disabled for python3, it will also fail due to the evaluation will throw the " is not supported for python3.8"
    • Maybe this is a plus? a previously python2-only library will probably need some manual work to get working.
  • If GITHUB_API_TOKEN (taken from vim plugin conventions) is not given, then user will be limited to 100 api requests; instead of 5000

If an update does fail, the entire update job will still succeed, similar to existing behavior.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@jonringer
Copy link
Contributor Author

cc @FRidh example of running script: #99636

@jonringer
Copy link
Contributor Author

The sha is being incorrectly calculated :(

@jonringer
Copy link
Contributor Author

jonringer commented Oct 5, 2020

ah, had to pass --unpack to the prefetcher, as fetchFromGitHub unpacks the tar.gz into a "source" directory.

The other option would be to change the sha into a known "bad" sha, and then see what nix-build complains about. This would be resilient to people passing "name" to fetchFromGitHub as well, but that's not a very common practice at all

@jonringer
Copy link
Contributor Author

built around 8000 python packages, didn't see any sha errors

Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! If it all works then go ahead and merge it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants