Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nix
base: ba334bd6f756
Choose a base ref
...
head repository: NixOS/nix
compare: e3013543d369
Choose a head ref
  • 3 commits
  • 8 files changed
  • 1 contributor

Commits on Dec 7, 2017

  1. nix ls-{nar,store}: Return offset of files in the NAR if known 

    E.g.

  $ nix ls-store --json --recursive --store https://cache.nixos.org /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79 \
    | jq .entries.bin.entries.blender.narOffset
  400
    @edolstra
    edolstra committed Dec 7, 2017
    Copy the full SHA
    338f29d View commit details
    Browse the repository at this point in the history

  2. Provide random access to cached NARs 

    E.g.

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m4.139s

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m0.024s

(Before, the second call took ~0.220s.)

This will use a NAR listing in
/tmp/nars/b0w2hafndl09h64fhb86kw6bmhbmnpm1.ls containing all metadata,
including the offsets of regular files inside the NAR. Thus, we don't
need to read the entire NAR. (We do read the entire listing, but
that's generally pretty small. We could use a SQLite DB by borrowing
some more code from nixos-channel-scripts/file-cache.hh.)

This is primarily useful when Hydra is serving files from an S3 binary
cache, in particular when you have giant NARs. E.g. we had some 12 GiB
NARs, so accessing individuals files was pretty slow.
    @edolstra
    edolstra committed Dec 7, 2017
    Copy the full SHA
    2df9cbe View commit details
    Browse the repository at this point in the history

  3. Fix test

    @edolstra
    edolstra committed Dec 7, 2017
    Copy the full SHA
    e301354 View commit details
    Browse the repository at this point in the history