Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bee: init at 0.1.0 #91825

Closed
wants to merge 3 commits into from
Closed

bee: init at 0.1.0 #91825

wants to merge 3 commits into from

Conversation

paulperegud
Copy link

@paulperegud paulperegud commented Jun 30, 2020

Motivation for this change

Bee is an official implementation of Ethereum's Swarm - distributed storage platform.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@dasj19
Copy link
Contributor

dasj19 commented Jul 12, 2020

Tested with nix-shell -p nixpkgs-review --run "nixpkgs-review pr 91825" and works fine. You can check the third box in the description.

Copy link
Contributor

@puzzlewolf puzzlewolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for your first PR 🎉

Looks good, binary seems to work, just two small changes :)

Also, it's common to add a new maintainer in a separate commit. Please split your commit into two: maintainers: add paulperegud and bee: init at 0.1.0.

};

meta = with stdenv.lib; {
homepage = https://swarm-gateways.net/bzz:/docs.swarm.eth/;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please quote the URL, see NixOS/rfcs#45

{ stdenv, buildGoModule, fetchFromGitHub }:

buildGoModule rec {
name = "bee-${version}";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name = "bee-${version}";
pname = "bee";

The name will be ${pname}-${version} automatically.

@ofborg ofborg bot requested a review from kalbasit July 14, 2020 09:42
@Lassulus
Copy link
Member

getting this error:

panic: qtls.ConnectionState not compatible with tls.ConnectionState

goroutine 1 [running]:
github.com/lucas-clemente/quic-go/internal/handshake.init.1()
	/build/source/vendor/github.com/lucas-clemente/quic-go/internal/handshake/unsafe.go:17 +0x12e

@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 91825 run on x86_64-darwin 1

1 package built:
  • bee

@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 91825 run on x86_64-linux 1

1 package built:
  • bee

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
@attila-lendvai
Copy link
Contributor

bee is at version 0.4.1 at this point. updating to that would most probably fix this issue.

i'm new to nix, but i'm willing to try to update this PR and maybe even add a service to it. how shall i proceed?

shall i create a new PR and reference this one?

or can i get write access to this PR somehow and then i just update this one?

Copy link
Contributor

@attila-lendvai attila-lendvai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm very new to all this. i've changed the ~/.cache/nixpkgs-review/pr-91825/build.nix file, but with my requested changes, it fails to build, and i don't know how to extract more information about the actual error:

$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/alendvai/.cache/nixpkgs-review/pr-91825/build.nix
builder for '/nix/store/dkxm5g3fb9pdfvmlcy1s22p8y8326q6a-bee-0.4.1.drv' failed with exit code 1; last 10 log lines:
  github.com/pelletier/go-toml
  github.com/spf13/afero/mem
  github.com/spf13/afero
  github.com/spf13/cast
  github.com/spf13/jwalterweatherman
  github.com/subosito/gotenv
  gopkg.in/ini.v1
  gopkg.in/yaml.v2
  github.com/spf13/viper
  golang.org/x/crypto/ssh/terminal
cannot build derivation '/nix/store/3s49s30yn4kgka6fishzhjb2s2lhls3b-env.drv': 1 dependencies couldn't be built
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/3s49s30yn4kgka6fishzhjb2s2lhls3b-env.drv' failed


buildGoModule rec {
pname = "bee";
version = "0.1.0";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version = "0.1.0";
version = "0.4.1";

owner = "ethersphere";
repo = "bee";
rev = "v${version}";
sha256 = "1iwwhs2zm8ml484sc0d7smmmp1x893zjafr1mnaswl8pxg0lnm2l";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sha256 = "1iwwhs2zm8ml484sc0d7smmmp1x893zjafr1mnaswl8pxg0lnm2l";
sha256 = "1bmgbav52pcb5p7cgq9756512fzfqhjybyr0dv538plkqx47mpv7";

sha256 = "1iwwhs2zm8ml484sc0d7smmmp1x893zjafr1mnaswl8pxg0lnm2l";
};

vendorSha256 = "1a9fmf5c3asvylcb0ixlwb6jk2z5hzdaa6yffg1pjvq9rx92jbjy";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
vendorSha256 = "1a9fmf5c3asvylcb0ixlwb6jk2z5hzdaa6yffg1pjvq9rx92jbjy";
vendorSha256 = "0j393va4jrg9q3wlc9mgkbpgnn2w2s3k2hcn8phzj8d5fl4n4v2h";

@paulperegud
Copy link
Author

I'm also trying to create a 4.1 build basing on this PR. Debugging things that you are seeing. BTW - I'm also pretty new to nix :-)

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Jan 7, 2021

bee is at version 0.4.1 at this point. updating to that would most probably fix this issue.

i'm new to nix, but i'm willing to try to update this PR and maybe even add a service to it. how shall i proceed?

shall i create a new PR and reference this one?

Yeah, create a new one and write in the description of it that it replaces this one.

or can i get write access to this PR somehow and then i just update this one?

That is not possible that easily.

@SuperSandro2000
Copy link
Member

i'm very new to all this. i've changed the ~/.cache/nixpkgs-review/pr-91825/build.nix file, but with my requested changes, it fails to build, and i don't know how to extract more information about the actual error:

$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/alendvai/.cache/nixpkgs-review/pr-91825/build.nix
builder for '/nix/store/dkxm5g3fb9pdfvmlcy1s22p8y8326q6a-bee-0.4.1.drv' failed with exit code 1; last 10 log lines:
  github.com/pelletier/go-toml
  github.com/spf13/afero/mem
  github.com/spf13/afero
  github.com/spf13/cast
  github.com/spf13/jwalterweatherman
  github.com/subosito/gotenv
  gopkg.in/ini.v1
  gopkg.in/yaml.v2
  github.com/spf13/viper
  golang.org/x/crypto/ssh/terminal
cannot build derivation '/nix/store/3s49s30yn4kgka6fishzhjb2s2lhls3b-env.drv': 1 dependencies couldn't be built
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/3s49s30yn4kgka6fishzhjb2s2lhls3b-env.drv' failed

take a look into the logs directory. There are the full logs.

@attila-lendvai
Copy link
Contributor

attila-lendvai commented Jan 11, 2021

so, the issue is that it builds fine, but fails with an assert at runtime:

$ ./results/bee/bin/bee version
panic: qtls.ConnectionState not compatible with tls.ConnectionState

the actual issue we are facing is discussed here: quic-go/quic-go#2614

according to this, it should work using go_1_14, but i couldn't find a way to specify the exact golang version to be used.

but on the other hand, if i check out the version from the bee repo, then i can build it using golang 1.15, which is strange to me.

$ go version
go version go1.15.5 linux/amd64

[nix-shell:~/workspace/ethereum/bee]$ git status
HEAD detached at v0.4.1

[nix-shell:~/workspace/ethereum/bee]$ ./dist/bee version
0.4.1-4b98b68

maybe it's because buildGoModule is not building with the exact same dependencies as a make binary in the checked out sources? it may be related to what's discussed here (long): #84826

FTR, this is the default.nix that i tried:

{ stdenv, buildGoModule, fetchFromGitHub }:

buildGoModule rec {
  pname = "bee";
  version = "0.4.1";

  src = fetchFromGitHub {
    owner = "ethersphere";
    repo = "bee";
    rev = "v${version}";
    sha256 = "1bmgbav52pcb5p7cgq9756512fzfqhjybyr0dv538plkqx47mpv7";
  };

  vendorSha256 = "0j393va4jrg9q3wlc9mgkbpgnn2w2s3k2hcn8phzj8d5fl4n4v2h";

  subPackages = [ "cmd/bee" ];  

  meta = with stdenv.lib; {
    homepage = "https://swarm-gateways.net/bzz:/docs.swarm.eth/";
    description = "Ethereum Swarm Bee";
    license = with licenses; [ bsd3 ];
    maintainers = with maintainers; [ paulperegud ];
  };
}

@SuperSandro2000
Copy link
Member

You can change the verison of go used by replacing buildGoModule wit heg buildGo114Module.

@attila-lendvai
Copy link
Contributor

attila-lendvai commented Jan 13, 2021

i have a reasonable default.nix for bee now that i think should work, but it doesn't compile, because of missing header files:

github.com/karalabe/usb
# github.com/karalabe/usb
vendor/github.com/karalabe/usb/hid_enabled.go:22:10: fatal error: ./hidapi/hidapi/hidapi.h: No such file or directory
   22 | #include "./hidapi/hidapi/hidapi.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
github.com/ethereum/go-ethereum/crypto/secp256k1
# github.com/ethereum/go-ethereum/crypto/secp256k1
vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:10: fatal error: libsecp256k1/include/secp256k1.h: No such file or directory
   42 | #include "libsecp256k1/include/secp256k1.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

my default.nix:

{ stdenv, buildGoModule, fetchFromGitHub, hidapi, secp256k1 }:

buildGoModule rec {
#buildGo114Module rec {
  pname = "bee";
  version = "0.4.1";

  #nativeBuildInputs = [ hidapi secp256k1 ];
  buildInputs = [ hidapi secp256k1 ];

  src = fetchFromGitHub {
    owner = "ethersphere";
    repo = "bee";
    rev = "v${version}";
    sha256 = "1bmgbav52pcb5p7cgq9756512fzfqhjybyr0dv538plkqx47mpv7";
  };

  vendorSha256 = "0j393va4jrg9q3wlc9mgkbpgnn2w2s3k2hcn8phzj8d5fl4n4v2h";

  subPackages = [ "cmd/bee" ];

  meta = with stdenv.lib; {
    homepage = "https://swarm-gateways.net/bzz:/docs.swarm.eth/";
    description = "Ethereum Swarm Bee";
    license = with licenses; [ bsd3 ];
    maintainers = with maintainers; [ paulperegud ];
  };
}

as you can see i have specified the dependencies with buildInputs, yet the .h files are not available. i've been searching and reading, but to no avail. AFAIU there are no separate -dev packages in the Nix universe.

this is also not a very useful error message for a newcomer like me:

$ nix-shell pkgs/applications/networking/bee/default.nix 
error: cannot auto-call a function that has an argument without a default value ('stdenv')

EDIT: i think i found the right spell, but it didn't help me get any further:

nix-shell -E 'with import <nixpkgs> { }; callPackage ./pkgs/applications/networking/bee/default.nix { }'

@SuperSandro2000 any hints on what to read/understand, or any examples to look at? i even searched github for buildGoModule and tried to find something similar. (sorry if i missed your answer on IRC)

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Jan 13, 2021

AFAIU there are no separate -dev packages in the Nix universe.

The dev dependency is has .dev if there is one. buildInputs should be smart enough to choose that.

The hidapi header file is this /nix/store/hz6mghlv18dzfhn22dgll32jnzfly6sr-hidapi-0.10.1/include/hidapi/hidapi.h

./hidapi/hidapi/hidapi.h does not exist and may be a local header file. I think runVend = true; vendors those files.

Same with libsecp256k1/include/secp256k1.h. The file in nix is /nix/store/yx1mgkm2x0ssfm20g10w4dd639gslksr-secp256k1-2020-08-16/include/secp256k1.h.

You either need to change the includes or patch the includes.

this is also not a very useful error message for a newcomer like me:

Nix 3.0 should improve this by a bit.

nix-shell pkgs/applications/networking/bee/default.nix

You can't do this. Go into the root of your nixpkgs clone and run nix-build -A bee.

nix-shell -E 'with import { }; callPackage ./pkgs/applications/networking/bee/default.nix { }'

That opens a nix-shell in that environment. You need to run some function like genericPhase or so. Don't remember the actual name right now.

sorry if i missed your answer on IRC

You should get a message if you missed something if I mentioned you and I normally don't write that much on IRC.

@attila-lendvai
Copy link
Contributor

@SuperSandro2000 thanks a whole lot!

i can now compile a bee and bee-unstable package. my branch has extensive changes on top of these two commits, so i'll open another PR that supersedes it once bee-clef and the bee service part is also packaged properly.

@attila-lendvai attila-lendvai mentioned this pull request Jan 14, 2021
10 tasks
@attila-lendvai
Copy link
Contributor

@paulperegud i'm making good progress with the packaging and created another (draft) PR: #109393

let me know if you wish to be included in the maintainers metadata. i've added you for now, but if you ask i can leave you out of this.

@SuperSandro2000 i'd appreciate any/all hints/reviews on the new PR.

@paulperegud
Copy link
Author

@attila-lendvai Thank you! So I'm closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants