Add vend support to go-modules #89453

c00w · 2020-06-04T02:58:22Z

Motivation for this change

Currently you have to manually patch vendor directories that distribute c code via the go package system. By adding the useVend option, this gets much simpler.

Fixes #89128

Things done

pkgs/development/tools/vend/default.nix

c00w · 2020-06-04T05:08:08Z

Had to open
nomad-software/vend#9 so that vend wouldn't run go mod tidy which isn't deterministic.

pkgs/development/go-modules/generic/default.nix

kalbasit

Would be great if we can run this by default, assuming it does not affect the current hashes as @Mic92 mentioned. Thanks for working on this!

Mic92 · 2020-06-04T15:26:59Z

What I am a bit worried about is that new versions of vend might include new files. For that reason might not want to include it for all packages.

c00w · 2020-06-05T02:52:21Z

Vend rejected my patch, so we're going to have to maintain a semi permanent fork of it anyway :( Everything else is updated.

pkgs/development/go-modules/generic/default.nix

c00w · 2020-06-11T02:04:38Z

@Mic92 Fixed formatting :)

pkgs/development/go-modules/generic/default.nix

pkgs/development/tools/vend/default.nix

c00w · 2020-06-20T01:02:00Z

@Mic92 - I think nothing else is outstanding here.

marsam · 2020-06-20T01:54:58Z

IMHO useVend could be slightly more fitting, so we could describe it as: "Whether to use vend instead of go mod vendor to vendorize the go module dependencies".
But I don't have a strong opinion about it.

zowoq · 2020-07-27T13:14:10Z

gobetween can be dropped from this PR. #93972

c00w · 2020-07-29T00:53:49Z

gobetween is dropped - working on figuring out hugo (looks like a rebase broke it).

This is used to provide alternate vendoring for go.

c00w · 2020-07-29T01:11:52Z

And hugo is fixed - should be g2g

zowoq · 2020-07-29T02:23:34Z

go-ethereum builds but the vendorSha256 needs to be updated.

diff --git a/pkgs/applications/blockchains/go-ethereum.nix b/pkgs/applications/blockchains/go-ethereum.nix
index e2b308d88e2..0edeccf4003 100644
--- a/pkgs/applications/blockchains/go-ethereum.nix
+++ b/pkgs/applications/blockchains/go-ethereum.nix
@@ -12,7 +12,7 @@ buildGoModule rec {
   };
 
   runVend = true;
-  vendorSha256 = "0r08fayxpccjyr29g38wc0zvpqgma9inyvdwcama1fyiq0n8n39d";
+  vendorSha256 = "1744df059bjksvih4653nnvb4kb1xvzdhypd0nnz36m1wrihqssv";
 
   subPackages = [
     "cmd/abidump"

Can you push the changes in nomad-software/vend@24fdebf to nomad-software/vend#10 please?

Upstream seems to be responsive so I'll give them a day or two to comment on the new changes.

c00w · 2020-07-29T23:08:05Z

Updated go-ethereum.

Re: vend - somehow I updated the pr verbally but didn't push. I'd strongly prefer we don't block on vend upstream, since they're still not convinced that the PR is needed + they've literally rejected every PR I sent them related to nix functionality. If they do merge it I can flip the repos back, but it's not great odds.

zowoq · 2020-07-29T23:31:08Z

Re: vend - somehow I updated the pr verbally but didn't push. I'd strongly prefer we don't block on vend upstream, since they're still not convinced that the PR is needed + they've literally rejected every PR I sent them related to nix functionality. If they do merge it I can flip the repos back, but it's not great odds.

If they are willing to accept the functionality but request changes we may need to update packages vendorSha256 again?

Mic92 · 2020-07-30T04:51:07Z

Re: vend - somehow I updated the pr verbally but didn't push. I'd strongly prefer we don't block on vend upstream, since they're still not convinced that the PR is needed + they've literally rejected every PR I sent them related to nix functionality. If they do merge it I can flip the repos back, but it's not great odds.

If they are willing to accept the functionality but request changes we may need to update packages vendorSha256 again?

I wonder if we just should maintain our own fork anyway. The package is quite simple and we cannot have upstream breaking our hashes if @r-ryantm makes a PR and somebody merges it without knowing the consequences. It could be simplified a lot i.e. removing some dependencies like command-line coloring which is not showed in many cases anyway. @c00w what do you think about a fork maintained in https://github.com/nix-community?

c00w · 2020-07-30T19:58:02Z

@zowoq @Mic92 In what proves my prothetic abilities - they just rejected the PR :P

I can move this to nix-community (although I'd prefer if I could get admin permissions there on the vend repo).

zowoq · 2020-07-31T02:43:26Z

@Mic92 Is this okay to merge or do you want to do the switch to nix-community in this PR?

Mic92 · 2020-07-31T04:54:51Z

@zowoq @Mic92 In what proves my prothetic abilities - they just rejected the PR :P

I can move this to nix-community (although I'd prefer if I could get admin permissions there on the vend repo).

I sent you an invite to the organisation. Once you moved the project their, you will be the maintainer and marked as such. However it also allows to easily move over maintenance in case you are lacking time.

Mic92 · 2020-07-31T04:55:29Z

@Mic92 Is this okay to merge or do you want to do the switch to nix-community in this PR?

Let's just move before. The process is just a few clicks away.

zowoq · 2020-07-31T04:56:46Z

Checking darwin:

@ofborg build aerc blockbook go-ethereum hugo mautrix-whatsapp saml2aws

Pull NixOS#89453 introduced a bug in the documentation that is preventing the hydra build for nixpkgs-unstable from finishing. I have added the additional option indroduced in that patch (runVend for go modules) and added the callout tag so that the documenation can build again.

Pull #89453 introduced a bug in the documentation that is preventing the hydra build for nixpkgs-unstable from finishing. I have added the additional option indroduced in that patch (runVend for go modules) and added the callout tag so that the documenation can build again.

Pull NixOS#89453 introduced a bug in the documentation that is preventing the hydra build for nixpkgs-unstable from finishing. I have added the additional option indroduced in that patch (runVend for go modules) and added the callout tag so that the documenation can build again.

c00w force-pushed the vend branch from 76630c0 to 6787acb Compare June 4, 2020 02:59

c00w changed the title ~~Vend~~ Add Vend to go-modules Jun 4, 2020

ofborg bot added the 6.topic: golang label Jun 4, 2020

c00w changed the title ~~Add Vend to go-modules~~ Add vend support to go-modules Jun 4, 2020

c00w force-pushed the vend branch from 6787acb to 7e21f6d Compare June 4, 2020 03:11

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jun 4, 2020

c00w mentioned this pull request Jun 4, 2020

Problematic behavior of buildGoModule in conjunction with vendorSha256 #89310

Closed

zowoq reviewed Jun 4, 2020

View reviewed changes

pkgs/development/tools/vend/default.nix Show resolved Hide resolved

c00w force-pushed the vend branch from 7e21f6d to b010eae Compare June 4, 2020 05:04

Mic92 reviewed Jun 4, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Outdated Show resolved Hide resolved

kalbasit approved these changes Jun 4, 2020

View reviewed changes

c00w force-pushed the vend branch from b010eae to d1ce321 Compare June 5, 2020 02:49

Mic92 reviewed Jun 5, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Outdated Show resolved Hide resolved

Mic92 reviewed Jun 5, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Show resolved Hide resolved

Mic92 reviewed Jun 5, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Show resolved Hide resolved

c00w force-pushed the vend branch from d1ce321 to 675caa8 Compare June 6, 2020 16:35

ofborg bot added the 8.has: documentation label Jun 6, 2020

Mic92 reviewed Jun 6, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Outdated Show resolved Hide resolved

c00w force-pushed the vend branch from 675caa8 to 3afb3c9 Compare June 11, 2020 02:04

marsam reviewed Jun 12, 2020

View reviewed changes

pkgs/development/go-modules/generic/default.nix Show resolved Hide resolved

pkgs/development/go-modules/generic/default.nix Show resolved Hide resolved

pkgs/development/tools/vend/default.nix Outdated Show resolved Hide resolved

pkgs/development/tools/vend/default.nix Show resolved Hide resolved

c00w force-pushed the vend branch from 3afb3c9 to 8d29441 Compare June 19, 2020 00:19

marsam approved these changes Jun 20, 2020

View reviewed changes

c00w force-pushed the vend branch from c8fcb13 to d880d0b Compare July 29, 2020 00:48

c00w added 2 commits July 28, 2020 21:11

vend: init at unstable-2020-06-04

148138e

This is used to provide alternate vendoring for go.

go-modules: Add useVend flag to go-modules

95be7b2

c00w force-pushed the vend branch from d880d0b to a040adc Compare July 29, 2020 01:11

c00w added 6 commits July 29, 2020 19:07

go-ethereum: use vend to simplify package

04dabea

hugo: use vend to simplify package

68248a9

blockbook: use vend to simplify package

298c6a7

saml2aws: use vend to simplify package

4561583

aerc: update to use runvend

3e92195

mautrix-whatsapp: update to use runVend

96908e4

c00w force-pushed the vend branch from a040adc to 96908e4 Compare July 29, 2020 23:08

zowoq merged commit e703f3f into NixOS:master Jul 31, 2020

matthewmazzanti mentioned this pull request Aug 1, 2020

doc: Fix missing callout in go.xml #94466

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vend support to go-modules #89453

Add vend support to go-modules #89453

c00w commented Jun 4, 2020 •

edited

c00w commented Jun 4, 2020

kalbasit left a comment

Mic92 commented Jun 4, 2020 •

edited

c00w commented Jun 5, 2020

c00w commented Jun 11, 2020

c00w commented Jun 20, 2020

marsam commented Jun 20, 2020 •

edited

zowoq commented Jul 27, 2020

c00w commented Jul 29, 2020

c00w commented Jul 29, 2020

zowoq commented Jul 29, 2020 •

edited

c00w commented Jul 29, 2020

zowoq commented Jul 29, 2020

Mic92 commented Jul 30, 2020 •

edited

c00w commented Jul 30, 2020

zowoq commented Jul 31, 2020

Mic92 commented Jul 31, 2020 •

edited

Mic92 commented Jul 31, 2020

zowoq commented Jul 31, 2020

Add vend support to go-modules #89453

Add vend support to go-modules #89453

Conversation

c00w commented Jun 4, 2020 • edited

Motivation for this change

Things done

c00w commented Jun 4, 2020

kalbasit left a comment

Choose a reason for hiding this comment

Mic92 commented Jun 4, 2020 • edited

c00w commented Jun 5, 2020

c00w commented Jun 11, 2020

c00w commented Jun 20, 2020

marsam commented Jun 20, 2020 • edited

zowoq commented Jul 27, 2020

c00w commented Jul 29, 2020

c00w commented Jul 29, 2020

zowoq commented Jul 29, 2020 • edited

c00w commented Jul 29, 2020

zowoq commented Jul 29, 2020

Mic92 commented Jul 30, 2020 • edited

c00w commented Jul 30, 2020

zowoq commented Jul 31, 2020

Mic92 commented Jul 31, 2020 • edited

Mic92 commented Jul 31, 2020

zowoq commented Jul 31, 2020

c00w commented Jun 4, 2020 •

edited

Mic92 commented Jun 4, 2020 •

edited

marsam commented Jun 20, 2020 •

edited

zowoq commented Jul 29, 2020 •

edited

Mic92 commented Jul 30, 2020 •

edited

Mic92 commented Jul 31, 2020 •

edited