Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetchArchive: init #53879

Closed
wants to merge 7 commits into from
Closed

fetchArchive: init #53879

wants to merge 7 commits into from

Conversation

lukateras
Copy link
Member

@lukateras lukateras commented Jan 13, 2019

Motivation for this change

Even though fetchzip uses stdenv's unpackFile to unpack .{tar{,.*},zip} files, it doesn't expose any convenient way to extend it to handle anything other than that.

Additionally, fetchzip's stripRoot goes beyond ignoring archive metadata/algorithm non-determinism and affects actual archive contents.

This is an attempt at a more explicit API that only fetches, then unpacks with one of inputs. It uses SRI hashes added in Nix 2.2 to avoid code churn that happens to algorithm-specific arg attrs as hash algorithms change.

stripRoot is implemented a la carte as a function that takes a directory and returns its first child.

Short list of changes:

  • add unpackCmd hook to dpkg
  • add SRI hash support to fetchurl
  • add fetchArchive + docs
  • showcase fetchArchive on typora derivation
  • add lib.stripRoot + docs

Fetch examples:

fetchArchive {
  url = https://github.com/NixOS/nix/archive/2.2.1.tar.gz;
  hash = "sha512-U8y9gVgvjowXZ/lTp7fxqRrWFbxZwrbS4LY95XduiM2plvhQ37rJ5TACo/PHStCILSvhzHG19UBZ1bWP62jEgQ==";
}

fetchArchive {
  url = https://github.com/NixOS/nix/archive/2.2.1.zip;
  hash = "sha512-U8y9gVgvjowXZ/lTp7fxqRrWFbxZwrbS4LY95XduiM2plvhQ37rJ5TACo/PHStCILSvhzHG19UBZ1bWP62jEgQ==";
  inputs = [ unzip ];
}

fetchArchive {
  url = https://deb.debian.org/debian/pool/main/a/apt/apt_1.4.8_arm64.deb;
  hash = "sha512-/6NY2ZhfJIKA6Qzg5vBxVaG9qfR7TqF/kK2ntzwMGXGpg2ycopjUAtxLqqRsM87OZQsYS7hklQhHcvx/NVz3dQ==";
  inputs = [ dpkg ];
}

Derivation override example:

nixUnstable.overrideAttrs (super: {
  src = lib.stripRoot (fetchArchive {
    url = https://github.com/NixOS/nix/archive/2.2.1.tar.gz;
    hash = "sha512-U8y9gVgvjowXZ/lTp7fxqRrWFbxZwrbS4LY95XduiM2plvhQ37rJ5TACo/PHStCILSvhzHG19UBZ1bWP62jEgQ==";
  });
})

Maybe this should better be called fetchThenUnpack. Also, instead of lib.stripRoot, stdenv can be tweaked into supporting non-stripped sources.

cc @grahamc, @LnL7, @oxij, @zimbatm

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@edolstra
Copy link
Member

Using SRI hashes will cause problems with NixOS upgrades, since nixos-rebuild uses the old daemon (which might not support SRI hashes) to do the build.

@lukateras
Copy link
Member Author

lukateras commented Jan 13, 2019

This is unfortunately true. What is a reasonable delay for leveraging new Nix features in Nixpkgs, two releases?

@vcunat
Copy link
Member

vcunat commented Jan 13, 2019

Something like two NixOS releases, yes. I suppose it depends on the expected amount of benefit from using the feature. I remember this precedent: #37693

@mmahut
Copy link
Member

mmahut commented Aug 18, 2019

Are there any updates on this pull request, please?

@mmahut
Copy link
Member

mmahut commented Jan 24, 2020

Closing due to inactivity.

@mmahut mmahut closed this Jan 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants