Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasmer: Add cargo hashes for Python 3.6/3.7/3.9 support #109039

Closed

Conversation

esotericnonsense
Copy link
Contributor

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions

@SuperSandro2000
Copy link
Member

And what about python 3.7 and 3.9?

@esotericnonsense esotericnonsense changed the title wasmer: Add cargo hash for Python 3.6 support wasmer: Add cargo hashes for Python 3.6/3.7/3.9 support Jan 12, 2021
@esotericnonsense
Copy link
Contributor Author

latest commit is a slight code clean up to avoid the annoying if-else chain

@esotericnonsense
Copy link
Contributor Author

also use pythonOlder rather than isPy3{6,7,8,9}

@jonringer
Copy link
Contributor

maintaining this is going to be difficult, is there a way we could have a script ran to generate the related hashes?

@esotericnonsense
Copy link
Contributor Author

@jonringer makes sense; are there any other examples in nixpkgs I may be able to look at for reference?

I had just manually run something like this (actually opened in 4 terminals and ran in parallel)

versions=( "36" "37" "38" "39" ); for x in "${versions[@]}"; do nix-build -A "python${x}Packages.wasmer"; done

and then just copy pasted the hashes in. It seems far easier to re-use nix's output than to independently figure out the cargo hash and paste it in.

@SuperSandro2000
Copy link
Member

I wonder why the hash changes at all. Is it selecting dependencies based on python version?

are there any other examples in nixpkgs I may be able to look at for reference?

There are many update.sh files but I did not find any that generated a vendorSha directly. They generate it with vgo2nix or yaml2nix.

It seems far easier to re-use nix's output than to independently figure out the cargo hash and paste it in.

Maybe we can try to build it in the update script and just put the hash in the file?

@@ -20,7 +20,12 @@ let
sha256 = "0302lcfjlw7nz18nf86z6swhhpp1qnpwcsm2fj4avl22rsv0h78j";
};

cargoSha256 = "0d83dniijjq8rc4fcwj6ja5x4hxh187afnqfd8c9fzb8nx909a0v";
cargoSha256 = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does cargo, depending on python version, fetch different dependencies?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another possibility is that there's some reference to the python version, thus the vendor libs get a different hash

Copy link
Contributor

@danieldk danieldk Jan 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that the Python version is used in the name attribute of the Rust derivation. name is used as part of the vendor directory and tarball. So, if the Python version is part of the name, the vendored tarball changes for every Python version.

(Cargo vendoring itself is reproducible modulo formatting changes in the metadata, which we normalize.)

A fix for this issue in the wasmer Python derivation is in #110580.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to close this in favor of daniels PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants