Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fetchgit: add lfs support #105998

Merged
merged 1 commit into from Jan 11, 2021
Merged

fetchgit: add lfs support #105998

merged 1 commit into from Jan 11, 2021

Conversation

jonringer
Copy link
Contributor

@jonringer jonringer commented Dec 5, 2020 •
edited

Motivation for this change

Add git-lfs support to fetchgit

motivation from https://discourse.nixos.org/t/how-to-fetch-lfs-enabled-repo-with-fetchfromgithub/5890/13

tested with:

$ cat temp.nix
let
  pkgs = import ./. { };
in
  pkgs.fetchgit {
    url = "https://gitlab.com/veloren/veloren";
    branchName = "yusdacra/override-git-lfs";
    sha256 = "sha256-aW6UWv0CL3Fd2kBxQ8tv6ta2oWtJukg4tGPggDLFS/g=";
    fetchLFS = true;
  }
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@jonringer
Copy link
Contributor Author

tried pulling a few existing packages that use fetchgit, and haven't found any issues yet.

@jonringer jonringer mentioned this pull request Dec 5, 2020
Closed
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-fetch-lfs-enabled-repo-with-fetchfromgithub/5890/15

@jonringer
Copy link
Contributor Author

Using LFS could cause a lot of downloads, not sure if we should advocate for people to also do meta.hydraPlatforms = [ ]; to prevent hydra from pulling down the world.

@jonringer
Copy link
Contributor Author 

https://github.com/NixOS/nixpkgs/pull/105998
12 packages built:
bundix cabal2nix crate2nix crystal2nix dep2nix go2nix haskellPackages.update-nix-fetchgit nix-prefetch-git nix-prefetch-scripts nix-update-source update-nix-fetchgit vgo2nix

@SuperSandro2000
Copy link
Member

Using LFS could cause a lot of downloads, not sure if we should advocate for people to also do meta.hydraPlatforms = [ ]; to prevent hydra from pulling down the world.

If you are using LFS in a public repo you should plan for this.

@jonringer
Copy link
Contributor Author

If you are using LFS in a public repo you should plan for this.

Yes, but exposing lfs within nixpkgs may encourage people to use it for packaging other projects (which is fine). Just don't want to bloat the nix cache with all of these large src pulls

@jonringer
Copy link
Contributor Author

cc @infinisil (previous deepclone work)

@jonringer
Copy link
Contributor Author

rebuilds are related to other scripts which depend on this script

12 packages built:
bundix cabal2nix crate2nix crystal2nix dep2nix go2nix haskellPackages.update-nix-fetchgit nix-prefetch-git nix-prefetch-scripts nix-update-source update-nix-fetchgit vgo2nix

@jonringer
Copy link
Contributor Author

I ran nix-build --check <pkg>.src on a dozen or so packages which use fetchgit with or without fetchSubmodules, and they all are able to be reproduced.

@jonringer jonringer merged commit 7412856 into NixOS:master Jan 11, 2021
@jonringer jonringer deleted the fetchgit-lfs branch January 11, 2021 17:41
@panicgh panicgh mentioned this pull request Feb 18, 2021
Closed
10 tasks
@FPtje FPtje mentioned this pull request Mar 9, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mic92 Mic92 Mic92 left review comments

@bhipple bhipple Awaiting requested review from bhipple

Assignees
No one assigned
Labels
6.topic: fetch 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jonringer @nixos-discourse @SuperSandro2000 @Mic92