How does this compare to #53436 ?

@Mic92 The universal prefetch of #53436 is focused on providing a command line interface for all available fetchers within Nixpkgs, while my prefetch will just use the source as it is defined, i.e. can remain ignorant about what specific fetcher is being used.

They both have the same goal, to predetermine the hash of a source, but they have different interfaces that serve different use cases better. The TOFU use case should be easier with my prefetcher, because you just point to the package or its source and it will report the correct hash, instead of having to duplicate the arguments given to the fetcher on the command line. For example:

The universal prefetcher:
nix-universal-prefetch fetchFromGitHub --owner samueldr --repo nix-universal-prefetch --rev v0.1.0

My prefetcher:
nix-prefetch nix-universal-prefetch

However the universal prefetcher is a better fit when all this information is already available to you in your update script, then the new hash can already be determined, while my prefetcher would require the update script to first update the package with the new version information in order for it to determine the correct new hashes for the sources.

@Mic92 I have an idea on how to merge the two prefetchers to get the best of both approaches, so I will be closing the PR in the meantime.

@Mic92 I reopened it with a version that is nothing alike the one I originally opened it with. Here are some of its features: https://github.com/msteen/nix-prefetch#features

@bhipple It does also support any package defining a src or srcs attribute, so this makes the use case you described, prefetching the sources before disconnecting from the internet, more convenient.

@danbst It got completions including autcomplete for the fetcher arguments for both bash and zsh.

@7c6f434c I think I have now covered all fetchers I could find with nix-prefetch --list --deep that disabled TLS verification by monkey patching them or for the builtin fetchers, by leveraging nix-prefetch-url (which does verify TLS).

The packaging seems nice. The script logic: so, a version bump requires additionally breaking the hash for prefetching to work? (for example, for fetchFromGitHub)?

a version bump requires additionally breaking the hash for prefetching to work? (for example, for fetchFromGitHub)?

You mean whether you still need to modify the hash (e.g. sha256) attribute passed to fetchFromGitHub? If so, no that's not needed. It will default to a probably wrong hash (zeroed hash):
https://github.com/msteen/nix-prefetch/blob/master/lib/prefetcher.nix#L83-L85
https://github.com/msteen/nix-prefetch/blob/master/src/main.sh#L547

It will just ignore the existing sha256. It will always do this, except when you explicitly supply the hash, or when it can be almost certain that the hash is in fact correct (you have installed something with that hash produced by the version updated derivation, i.e. nothing changed, so no need to refetch). An example of it ignoring the hash in all other cases is already in the examples:

A package with verbose output:

$ nix-prefetch hello --verbose 
The package hello-2.10 will be fetched as follows:
> fetchurl {
>   sha256 = "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i";
>   url = "mirror://gnu/hello/hello-2.10.tar.gz";
> }

The following URLs will be fetched as part of the source:
mirror://gnu/hello/hello-2.10.tar.gz

trying http://ftpmirror.gnu.org/hello/hello-2.10.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  708k  100  708k    0     0  1498k      0 --:--:-- --:--:-- --:--:-- 1498k

0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i

It is redownloading it, because outside the exceptions I just gave, its normally impossible for me to know whether a hash is correct except from recalculating it.

I could try and break it in more pieces and comment it some more, but what it does is check if there is something in the nix store that already refers to the source derivation that is about to be realized (i.e. prefetched). If there exist such derivations, and the outputs of those derivations actually exist in the nix store (this does not have to be the case), then we don't have any reason to refetch it. However it could still be the product of building your package with an updated version but outdated hash (so it would have used the old sources), so I check if there is a root to it (outside from generally temporary result roots), then I can be very certain there is no need to refetch the source.

It would have been a really straightforward check it were not for: NixOS/nix#2631

Its usefulness has gone down now that I found out that there is no way to determine the correctness of a hash or whether it is outdated, other than this approximation. Thinking about it again, I think its better to only do this check via a flag, rather than by default. So I would be removing --force and replace it for a --try-local (or something), what do you think?

@7c6f434c I have added more thorough comments on that piece of script. This is how I updated the Nixpkgs definition just now: nix-prefetch 'with import ./. { overlays = []; }; nix-prefetch' --rev ''. If I update my nix-fetch-update project to work with the latest nix-prefetch, then it could even have been completely automated.

Oh. Nice! Thank you very much for the additions. Now the «model of world» is indeed clear.

I don't know if it is a good idea to mention somewhere (maybe in README?) that what it does is uses n overlay to wrap various commands to ignore the insecure flags.