Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildPython*: add updateScript to passthru #50983

Merged
merged 1 commit into from Dec 7, 2018
Merged

buildPython*: add updateScript to passthru #50983

merged 1 commit into from Dec 7, 2018

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Nov 24, 2018

All Python packages now have an updateScript. The script calls
update-python-libraries and passes it the position of the derivation
expression obtained using meta.position. This works fine in case a Nix
expression represents only a single derivation. If there are more in it,
update-python-libraries will fail.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh FRidh mentioned this pull request Nov 24, 2018
Merged
10 tasks
@FRidh FRidh mentioned this pull request Nov 24, 2018
Closed
FRidh
FRidh commented Nov 24, 2018
View reviewed changes
pkgs/development/interpreters/python/mk-python-derivation.nix Outdated
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''
${update-python-libraries} ${filename} --commit
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will have to disable committing in case we run concurrently with #50977

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be nice if update.py handled the committing. Though in my experience git’s locking is pretty bad for things like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree.

@FRidh FRidh mentioned this pull request Nov 24, 2018
Closed
jtojnar
jtojnar reviewed Nov 24, 2018
View reviewed changes
pkgs/development/interpreters/python/mk-python-derivation.nix
passthru = {
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
in writeScript "update-python" ''
in writeScript "update-python" ''
#!${python.stdenv.shell}

jtojnar
jtojnar reviewed Nov 25, 2018
View reviewed changes
pkgs/development/interpreters/python/mk-python-derivation.nix
passthru = {
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''
Copy link
Contributor

@jtojnar jtojnar Nov 25, 2018 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #50977 (comment), you can do

passthru = {
  updateScript = let
    filename = builtins.head (lib.splitString ":" self.meta.position);
  in [ update-python-libraries filename "--commit" ];
};

Mic92
Mic92 reviewed Nov 26, 2018
View reviewed changes
pkgs/development/interpreters/python/update-python-libraries/update-python-libraries.py
# Commits are created sequentially.
if args.commit:
logging.info("Committing updates...")
list(map(lambda x: _commit(**x), filter(bool, results)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this easier to read:

Suggested change
list(map(lambda x: _commit(**x), filter(bool, results)))
for result in results:
if results:
_commit(**result)

or if your really want code golfing:

Suggested change
list(map(lambda x: _commit(**x), filter(bool, results)))
[_commit(**result) for result in results if result]

@FRidh
buildPython*: add updateScript to passthru
56a0ed1 
All Python packages now have an updateScript. The script calls
`update-python-libraries` and passes it the position of the derivation
expression obtained using `meta.position`. This works fine in case a Nix
expression represents only a single derivation. If there are more in it,
`update-python-libraries` will fail.
@FRidh FRidh merged commit 74d0b82 into NixOS:master Dec 7, 2018
@FRidh FRidh deleted the update-script branch December 7, 2018 14:49
@jtojnar
Copy link
Contributor

jtojnar commented Dec 22, 2018 •
edited

This seems to break nix-shell maintainers/scripts/update.nix --argstr path python3.pkgs.pygobject3

No error is printed but if I modify the script, I can see that the updateScript from the expression is overridden by the one from this PR:

INFO:root:Updating packages...
WARNING:root:Path /home/jtojnar/projects/nixpkgs/pkgs/development/python-modules/pygobject/3.nix: url does not point to PyPI.
INFO:root:Finished updating packages.
INFO:root:0 package(s) updated

@FRidh
Copy link
Member Author

FRidh commented Dec 23, 2018

@jtojnar pushed 3d5025b to master

@nixos-discourse
Copy link

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/automating-updates-of-ruby-packages/2996/2

@jtojnar jtojnar added the 6.topic: updaters Tooling for (semi-)automated updating of packages label Jun 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mic92 Mic92 Mic92 left review comments

@jtojnar jtojnar jtojnar left review comments

Assignees
No one assigned
Labels
6.topic: python 6.topic: updaters Tooling for (semi-)automated updating of packages 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@FRidh @jtojnar @nixos-discourse @Mic92 @GrahamcOfBorg