Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetchFromGitHub: add possibility to fetch release assets #85289

Closed

Conversation

sternenseemann
Copy link
Member

@sternenseemann sternenseemann commented Apr 15, 2020

Motivation for this change

Sometimes it is desireable to download files that are uploaded as assets to accompany a release. This is for example often the case in ocamlPackages, since package authors often generate release tarballs containing the source code, but preprocessed to adjust certain values like version numbers. Instead of using fetchurl all the time, we could now use fetchGitHubRelease.

Instead of ...

src = fetchurl {
  url = "https://github.com/hannesm/duration/releases/download/${version}/duration-${version}.tbz";
  sha256 = "0m9r0ayhpl98g9vdxrbjdcllns274jilic5v8xj1x7dphw21p95h";
};

... we'd have ...

src = fetchGitHubRelease {
  owner = "hannesm";
  repo = pname;
  tag = version;
  sha256 = "07dbirgrypxjg743k4fqw63vzk489ps4m5dncnzfvxjb93xmi9n5";
  asset = "${pname}-${version}.tbz";
};
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@grahamc
Copy link
Member

grahamc commented Apr 15, 2020

I think this is probably not a good direction for fetchFromGitHub: it is not so complicated from the code level, but I think conceptually makes the behavior much more complicated. Instead, maybe a fetchGitHubRelease function?

@Mic92
Copy link
Member

Mic92 commented Apr 15, 2020

What benefits does this abstraction provide over using fetchurl?

src = fetchurl {
  url = "https://github.com/hannesm/duration/releases/download/${version}/duration-${version}.tbz";
  sha256 = "0m9r0ayhpl98g9vdxrbjdcllns274jilic5v8xj1x7dphw21p95h";
};

We don't need things like fetchzip for releases assets since they are not suppose to change.

@sternenseemann
Copy link
Member Author

@grahamc I see your point, probably would make (more) sense.

@Mic92 I guess mostly convenience, since these URLs are kind of annoying too manually insert and construct every time. Also I guess you could argue the same thing for fetchFromGitHub in the first place, as you can also use fetchzip directly. Additionally I think this is a functionality packagers expect from fetchFromGitHub because intuitively it makes little sense why you can't download files using fetchFromGitHub listed directly next to files that you can.

This fetcher downloads release assets attached to a release tag on
github using fetchurl. This functionality was missing form
fetchFromGitHub, but not possible to integrate into it cleanly, since it
would complicate its behavior and also doesn't support private
repositories (without creating a GitHub app for it).
@sternenseemann
Copy link
Member Author

I implemented it now according to @grahamc's suggestion. I'm not sure, if it's worth merging it in this state though, I'd be interested in your opinions.

One idea to improve this would be to add support for fetching release assets of private repositories. This is not possible at all currently, because GitHub always returns a 404 on private release assets, even if you are authenticated. It'd be required to send two API requests in order to get an download URL for the download. I am unsure, if that might be beyond the scope of nixpkgs fetchers?

@stale
Copy link

stale bot commented Oct 18, 2020

Hello, I'm a bot and I thank you in the name of the community for your contributions.

Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:

If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list.

If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past.

If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments.

Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Oct 18, 2020
@bhipple
Copy link
Contributor

bhipple commented Oct 25, 2020

It sounds like we prefer to just keep it simple and use fetchurl for now. I'm going to close this stale PR to cleanup the backlog, but if you'd like to re-raise for discussion feel free to re-open.

@bhipple bhipple closed this Oct 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: fetch 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants