Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nix
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 4ec6eb1fdf51
Choose a base ref
...
head repository: NixOS/nix
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: bf6792c0df21
Choose a head ref
  • 2 commits
  • 4 files changed
  • 1 contributor

Commits on Jul 17, 2017

  1. Make the hashes mirrors used by builtins.fetchurl configurable 

    In particular, this allows it to be disabled in our tests.
    @edolstra
    edolstra committed Jul 17, 2017
    Copy the full SHA
    49304ba View commit details

  2. Always use base-16 for hashed mirror lookups 

    In particular, don't use base-64, which we don't support. (We do have
base-32 redirects for hysterical reasons.)

Also, add a test for the hashed mirror feature.
    @edolstra
    edolstra committed Jul 17, 2017
    Copy the full SHA
    bf6792c View commit details
Showing with 56 additions and 8 deletions.
  1. +28 −0 doc/manual/command-ref/conf-file.xml
  2. +10 −6 src/libstore/builtins.cc
  3. +3 −0 src/libstore/globals.hh
  4. +15 −2 tests/fetchurl.sh
28 changes: 28 additions & 0 deletions doc/manual/command-ref/conf-file.xml
View file
Original file line number Diff line number Diff line change
@@ -660,6 +660,34 @@ password <replaceable>my-password</replaceable>
</varlistentry>


<varlistentry xml:id="conf-hashed-mirrors"><term><literal>hashed-mirrors</literal></term>

<listitem><para>A list of web servers used by
<function>builtins.fetchurl</function> to obtain files by
hash. The default is
<literal>http://tarballs.nixos.org/</literal>. Given a hash type
<replaceable>ht</replaceable> and a base-16 hash
<replaceable>h</replaceable>, Nix will try to download the file
from
<literal>hashed-mirror/<replaceable>ht</replaceable>/<replaceable>h</replaceable></literal>.
This allows files to be downloaded even if they have disappeared
from their original URI. For example, given the default mirror
<literal>http://tarballs.nixos.org/</literal>, when building the derivation

<programlisting>
builtins.fetchurl {
url = https://example.org/foo-1.2.3.tar.xz;
sha256 = "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae";
}
</programlisting>

Nix will attempt to download this file from
<literal>http://tarballs.nixos.org/sha256/2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae</literal>
first. If it is not available there, if will try the original URI.</para></listitem>

</varlistentry>


</variablelist>

</para>
16 changes: 10 additions & 6 deletions src/libstore/builtins.cc
View file
Original file line number Diff line number Diff line change
@@ -38,12 +38,16 @@ void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)

std::shared_ptr<std::string> data;

try {
if (getAttr("outputHashMode") == "flat")
data = fetch("http://tarballs.nixos.org/" + getAttr("outputHashAlgo") + "/" + getAttr("outputHash"));
} catch (Error & e) {
debug(e.what());
}
if (getAttr("outputHashMode") == "flat")
for (auto hashedMirror : settings.hashedMirrors.get())
try {
if (!hasSuffix(hashedMirror, "/")) hashedMirror += '/';
auto ht = parseHashType(getAttr("outputHashAlgo"));
data = fetch(hashedMirror + printHashType(ht) + "/" + Hash(getAttr("outputHash"), ht).to_string(Base16, false));
break;
} catch (Error & e) {
debug(e.what());
}

if (!data) data = fetch(getAttr("url"));

3 changes: 3 additions & 0 deletions src/libstore/globals.hh
View file
Original file line number Diff line number Diff line change
@@ -327,6 +327,9 @@ public:
"Whether builders can acquire new privileges by calling programs with "
"setuid/setgid bits or with file capabilities."};
#endif

Setting<Strings> hashedMirrors{this, {"http://tarballs.nixos.org/"}, "hashed-mirrors",
"A list of servers used by builtins.fetchurl to fetch files by hash."};
};


17 changes: 15 additions & 2 deletions tests/fetchurl.sh
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ clearStore
# Test fetching a flat file.
hash=$(nix-hash --flat --type sha256 ./fetchurl.sh)

outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha256 $hash --no-out-link)
outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha256 $hash --no-out-link --option hashed-mirrors '')

cmp $outPath fetchurl.sh

@@ -14,10 +14,23 @@ clearStore

hash=$(nix hash-file --type sha512 --base64 ./fetchurl.sh)

outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha512 $hash --no-out-link)
outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file://$(pwd)/fetchurl.sh --argstr sha512 $hash --no-out-link --option hashed-mirrors '')

cmp $outPath fetchurl.sh

# Test the hashed mirror feature.
clearStore

hash=$(nix hash-file --type sha512 --base64 ./fetchurl.sh)
hash32=$(nix hash-file --type sha512 --base16 ./fetchurl.sh)

mirror=$TMPDIR/hashed-mirror
rm -rf $mirror
mkdir -p $mirror/sha512
ln -s $(pwd)/fetchurl.sh $mirror/sha512/$hash32

outPath=$(nix-build '<nix/fetchurl.nix>' --argstr url file:///no-such-dir/fetchurl.sh --argstr sha512 $hash --no-out-link --option hashed-mirrors "file://$mirror")

# Test unpacking a NAR.
rm -rf $TEST_ROOT/archive
mkdir -p $TEST_ROOT/archive