Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unextracted all cabal hashes #31333

Merged
merged 2 commits into from Nov 8, 2017

Conversation

ElvishJerricco
Copy link
Contributor

Motivation for this change

#30290, while useful, is kind of massively complicated, and actually leads to hashes which are different depending on the OS. We can get rid of the crazy C program and special schema, and reduce disk usage if we just leave all the files in the tarball and extract them as needed to their own store paths. This produces unique derivations and therefore unique paths, even given macOS's case insensitivity, without having to conjure those paths manually.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@ElvishJerricco
Copy link
Contributor Author

cc: @shlevy

Copy link
Member

@shlevy shlevy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎆

@shlevy
Copy link
Member

shlevy commented Nov 6, 2017

Barring objections from @peti, ping me in a few days and I'll merge

Copy link
Member

@peti peti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

in fetchFromGitHub (src-spec //
{ postFetch = "${partition-all-cabal-hashes}/bin/partition-all-cabal-hashes $downloadedFile $out";
})
fetchurl {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you prefer fetchurl over fetchFromGitHub here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetchFromGitHub is the whole cause of the problem. It extracts the tarball in full to a directory, which breaks on macOS when that tarball has paths that clash on a case insensitive file system.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I got it! This whole ordeal does make sense now :D

@peti peti merged commit c7203ef into NixOS:master Nov 8, 2017
@Ericson2314
Copy link
Member

Should we remove that extra hook from fetchurl / fetchgit too?

@mightybyte
Copy link
Contributor

@shlevy Any chance we can get this back ported to 17.03?

@shlevy
Copy link
Member

shlevy commented Dec 13, 2017

Sure, ping me on the PR

@ElvishJerricco
Copy link
Contributor Author

@shlevy Should I just open a PR to the 17.03 branch on nixpkgs-channels?

@shlevy
Copy link
Member

shlevy commented Dec 13, 2017

No, the release-17.03 branch here.

@ElvishJerricco
Copy link
Contributor Author

Gotcha.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants