This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/suggestion-feature-use-bittorent-or-ipfs-to-download-packages/1153/33

It seems conceptually wrong to have LocalStore expose a "file hash" because it has no such concept (nor for that matter does Store). This is also evident from the implementation of LocalStore::queryPathFromFileHash():

return getNarInfoDiskCache()->queryPathFromFileHash(fileHash);

This doesn't touch this which suggests it shouldn't be a method of LocalStore. Also, the .narinfo cache can be nuked at any time so it's not a reliable source of information. Furthermore, if you re-serve the store path, compressing the NAR may produce a different result so relying on the fileHash is not very reliable. It may also be insecure since the signature in the .narinfo file is over the NAR hash, not the file hash. So a MitM could serve a binary cache that contains correct NAR hashes but incorrect file hashes, and then get a node on the network to serve files with those hashes.

However, it's not entirely clear to me why this is needed. When serving the local store, you can generate .narinfo files that contain upstream signatures (these are stored in ValidPaths.sigs in the Nix database). You could filter out any paths/signatures that don't come from cache.nixos.org or whatever.

Thank you for the feedback! I posted this here mainly to gather feedback on the
idea. As you pointed out this isn't really code that should go in as is.

It seems conceptually wrong to have LocalStore expose a "file hash" because
it has no such concept (nor for that matter does Store). This is also
evident from the implementation of LocalStore::queryPathFromFileHash():

return getNarInfoDiskCache()->queryPathFromFileHash(fileHash);

This doesn't touch this which suggests it shouldn't be a method of LocalStore.

I agree. This was a hack that looked like it solved my problem at the time.
(Read below why I preferred this over direct SQLite access)

Also, the .narinfo cache can be nuked at any time so it's not a reliable
source of information.

I accepted that as a caveat for my version. Gotta start somewhere.

Furthermore, if you re-serve the store path, compressing the NAR may produce a
different result so relying on the fileHash is not very reliable.

Yeah, I am skipping compression entirely for now. I probably shouldn't have abused the fileHashes as I ended up doing. I was naively hoping to implement a compatible compression at some point in the future…. Will probably not go down that road any time soon as it requires pre-compressing everything that I provide a .narinfo file for.

It may also be insecure since the signature in the .narinfo file is over the
NAR hash, not the file hash. So a MitM could serve a binary cache that
contains correct NAR hashes but incorrect file hashes, and then get a node on
the network to serve files with those hashes.

But that would require some node to skip the verification of NAR hashes on
import?! That would be an issue in any case. I can see that Nix might be
rejecting those paths and might be stuck there. This probably applies for all
bad actors between the user and the binary cache(s). (Minus HTTPs etc…)

However, it's not entirely clear to me why this is needed. When serving the
local store, you can generate .narinfo files that contain upstream signatures
(these are stored in ValidPaths.sigs in the Nix database). You could filter
out any paths/signatures that don't come from cache.nixos.org or whatever.

The problem with reading from the SQLite database is that as an unpriviledged
process SQLite doesn't really support reading database (unless you own the
file/dir). In any case (reading/writing) SQLite wants to access/create WAL
files. That obviously doesn't work if the process can't write to the directory.
There is a mode (?immutable=1) where it allows reading files but you have to
pledge that the file will not change again. Not really suitable for a system
that might still add/remove paths.

This made adding the one missing lookup to the store protocol sound like a good
way to work around that limitation.

I also didn't want to rely on too much internal knowledge of Nix. One could
argue that the database layout is probably not a public API but then again the
store protocol also isn't really "public"?. I could probably have added a Nix
plugin that duplicated a bit of code from other parts and exposed a custom
interfaces.. That sounded jut wrong at the time (and still does).

Again thank you for the feedback. I can now go back to the drawing board and
figure out my next steps :-)

I marked this as stale due to inactivity. → More info

I closed this issue due to inactivity. → More info