Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

smallstep: 0.13.3 -> 0.14.6 #93711

Merged
merged 2 commits into from Dec 26, 2020
Merged

smallstep: 0.13.3 -> 0.14.6 #93711

merged 2 commits into from Dec 26, 2020

Conversation

arianvp
Copy link
Member

@arianvp arianvp commented Jul 23, 2020

Uses the same workaround as yubikey-agent. Namely build with
go-piv/piv-go#75 to make piv-go build properly

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@arianvp
Copy link
Member Author

arianvp commented Jul 23, 2020

@GrahamcOfBorg build step-cli step-ca

@philandstuff
Copy link
Contributor

Out of interest, why isn’t the piv-go patch needed for step-cli? piv-go is mentioned in the step-cli go.sum but for all I know this could be stale and not actually used.

@arianvp
Copy link
Member Author

arianvp commented Jul 23, 2020

Good question. I don't know. Let's have a look before merging. It compiled fine without the dependency

@flokli
Copy link
Contributor

flokli commented Jul 23, 2020

Probably best to regenerate the go.sum file and ask the smallstep maintainers about it.

@philandstuff
Copy link
Contributor

Yeah if go mod tidy removes it from go.sum then we don’t need to worry about it

@flokli
Copy link
Contributor

flokli commented Jul 23, 2020

@arianvp scrolling through their website, it seems they also provide pam and nss modules for user management.

Is this up somewhere too?

# pull in go-piv/piv-go#75
# once go-piv/piv-go#75 is merged and released, we should
# use the released version (and push upstream to do the same)
patches = [ ../yubikey-agent/use-piv-go-75.patch ];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting this from another package is dirty, and considering the version this patch is pointing to breaks darwin (see go-piv/piv-go#75), and piv-go might not be needed at all, let's reach out to upstream on their go.sum file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

piv-go is needed. Package doesn't build without it.

Copy link
Member Author

@arianvp arianvp Jul 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note there are two packages being bumped here. One really requires piv-go

Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the deps.nix.

@@ -1,19 +1,30 @@
{ lib, buildGoPackage, fetchFromGitHub }:
{ lib, stdenv, pkgconfig, pcsclite, buildGoModule, fetchFromGitHub, darwin }:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{ lib, stdenv, pkgconfig, pcsclite, buildGoModule, fetchFromGitHub, darwin }:
{ lib, stdenv, pkg-config, pcsclite, buildGoModule, fetchFromGitHub, PCSC }:

Please inerhit PCSC in top-level.nix.

@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 93711 run on x86_64-darwin 1

2 packages failed to build:
  • step-ca
  • step-cli

Please fix the tests with sandbox enabled.

    remote_test.go:48: unknown host smallstep.com: lookup smallstep.com: no such host
--- FAIL: TestGetPeerCertificateServerName (0.00s)
FAIL
FAIL    github.com/smallstep/cli/command/certificate    0.026s
FAIL

@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 93711 run on x86_64-linux 1

2 packages failed to build:
  • step-ca
  • step-cli

@AluisioASG
Copy link
Contributor

Is this still being worked on?

@flokli
Copy link
Contributor

flokli commented Dec 22, 2020

I'm also interested in this. go-piv/piv-go#75 is included in go-piv/piv-go since v1.6.0, and https://github.com/smallstep/certificates/blob/master/go.mod points to v1.6.0, so this should just work without any patches and the latest release (0.15.6).

@arianvp, do you want to update this?

@arianvp
Copy link
Member Author

arianvp commented Dec 22, 2020 via email

@flokli
Copy link
Contributor

flokli commented Dec 22, 2020

I pushed a new version addressing all the comments.

For step-cli, when just using 0.15.3, downloading all go dependencies fails:

Build Log
these derivations will be built:
  /nix/store/jj0j1vz9gqk24sl6brlabyzdh6zhkhym-step-cli-0.15.3-go-modules.drv
  /nix/store/x3ygbhsf4i1lw5dg6scjpqcmks92bakm-step-cli-0.15.3.drv
building '/nix/store/jj0j1vz9gqk24sl6brlabyzdh6zhkhym-step-cli-0.15.3-go-modules.drv'...
unpacking sources
unpacking source archive /nix/store/hawljm6w47nlha8pmpb5z4zi2il4ra43-source
source root is source
patching sources
configuring
building
go: downloading golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de
go: downloading github.com/urfave/cli v1.22.2
go: downloading github.com/ThomasRooney/gexpect v0.0.0-20161231170123-5482f0350944
go: downloading gopkg.in/square/go-jose.v2 v2.5.1
go: downloading github.com/smallstep/certificates v0.15.5
go: downloading github.com/google/uuid v1.1.2
go: downloading golang.org/x/sys v0.0.0-20200828194041-157a740278f4
go: downloading github.com/pquerna/otp v1.0.0
go: downloading github.com/manifoldco/promptui v0.3.1
go: downloading github.com/smallstep/zlint v0.0.0-20180727184541-d84eaafe274f
go: downloading github.com/icrowley/fake v0.0.0-20180203215853-4178557ae428
go: downloading github.com/smallstep/assert v0.0.0-20200723003110-82e2b9b3b262
go: downloading github.com/Microsoft/go-winio v0.4.14
go: downloading github.com/kr/pty v1.1.8
go: downloading github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0
go: downloading software.sslmate.com/src/go-pkcs12 v0.0.0-20200830195227-52f69702a001
go: downloading github.com/juju/ansiterm v0.0.0-20180109212912-720a0952cc2a
go: downloading github.com/stretchr/testify v1.5.1
go: downloading github.com/boombuler/barcode v1.0.0
go: downloading github.com/samfoo/ansi v0.0.0-20160124022901-b6bd2ded7189
go: downloading github.com/mattn/go-isatty v0.0.11
go: downloading github.com/smallstep/truststore v0.9.6
go: downloading go.step.sm/crypto v0.6.1
go: downloading gopkg.in/yaml.v2 v2.2.7
go: downloading github.com/smallstep/zcrypto v0.0.0-20200203191936-fbc32cf76bce
go: downloading github.com/russross/blackfriday v1.5.2
go: downloading github.com/russross/blackfriday/v2 v2.0.1
go: downloading howett.net/plist v0.0.0-20181124034731-591f970eefbb
go: downloading github.com/creack/pty v1.1.7
go: downloading github.com/corpix/uarand v0.1.1
go: downloading github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
go: downloading github.com/pkg/errors v0.9.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/weppos/publicsuffix-go v0.4.0
go: downloading github.com/lunixbochs/vtclean v1.0.0
go: downloading golang.org/x/net v0.0.0-20200822124328-c89045814202
go: downloading github.com/smallstep/nosql v0.3.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/smallstep/certinfo v1.4.0
go: downloading github.com/rs/xid v1.2.1
go: downloading github.com/mattn/go-colorable v0.1.4
go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: downloading github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf
go: downloading github.com/go-chi/chi v4.0.2+incompatible
go: downloading google.golang.org/protobuf v1.25.0
go: downloading github.com/dgraph-io/badger v1.5.3
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/Masterminds/sprig/v3 v3.1.0
go: downloading github.com/google/certificate-transparency-go v1.1.0
go: downloading github.com/Masterminds/semver v1.4.2
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: downloading github.com/Masterminds/semver/v3 v3.1.0
go: downloading github.com/Masterminds/goutils v1.1.0
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.2
go: downloading github.com/newrelic/go-agent v2.15.0+incompatible
go: downloading cloud.google.com/go v0.65.1-0.20200904011802-3c2db50b5678
go: downloading github.com/mitchellh/copystructure v1.0.0
go: downloading github.com/dgraph-io/badger/v2 v2.0.1-rc1.0.20200413122845-09dd2e1a4195
go: downloading github.com/imdario/mergo v0.3.8
go: downloading google.golang.org/grpc v1.32.0
go: downloading google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d
go: downloading github.com/mitchellh/reflectwalk v1.0.0
go: downloading go.etcd.io/etcd v3.3.18+incompatible
go: downloading github.com/huandu/xstrings v1.3.1
go: downloading github.com/golang/mock v1.4.4
go: downloading github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2
go: downloading github.com/dgraph-io/ristretto v0.0.2-0.20200115201040-8f368f2f2ab3
go: downloading github.com/dustin/go-humanize v1.0.0
go: downloading github.com/golang/protobuf v1.4.2
go: downloading github.com/golangci/golangci-lint v1.22.2
go: downloading github.com/DataDog/zstd v1.4.1
go: downloading go.etcd.io/bbolt v1.3.2
go: downloading github.com/go-sql-driver/mysql v1.5.0
go: downloading golang.org/x/tools v0.0.0-20200903185744-af4cc2cd812e
go: downloading github.com/google/monologue v0.0.0-20191220140058-35abc9683a6c
go: downloading google.golang.org/api v0.31.0
go: downloading github.com/AndreasBriese/bbloom v0.0.0-20190306092124-e2d15f34fcf9
go: downloading github.com/golang/snappy v0.0.1
go: downloading github.com/spf13/viper v1.6.1
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading gopkg.in/ini.v1 v1.51.1
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/golangci/maligned v0.0.0-20180506175553-b1d89398deca
go: downloading github.com/ultraware/whitespace v0.0.4
go: downloading github.com/uudashr/gocognit v1.0.1
go: downloading github.com/spf13/cobra v0.0.5
go: downloading github.com/coreos/etcd v3.3.18+incompatible
go: downloading github.com/ultraware/funlen v0.0.2
go: downloading github.com/cespare/xxhash v1.1.0
go: downloading github.com/securego/gosec v0.0.0-20200106085552-9cb83e10afad
go: downloading github.com/golangci/errcheck v0.0.0-20181223084120-ef45e06d44b6
go: downloading github.com/golangci/prealloc v0.0.0-20180630174525-215b22d4de21
go: downloading github.com/golangci/misspell v0.0.0-20180809174111-950f5d19e770
go: downloading honnef.co/go/tools v0.0.1-2020.1.4
go: downloading github.com/sourcegraph/go-diff v0.5.1
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/golangci/gofmt v0.0.0-20190930125516-244bba706f1a
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/tommy-muehle/go-mnd v1.1.1
go: downloading github.com/spf13/afero v1.2.2
go: downloading github.com/stretchr/objx v0.2.0
go: downloading github.com/golangci/gocyclo v0.0.0-20180528144436-0a533e8fa43d
go: downloading github.com/fatih/color v1.8.0
go: downloading github.com/golangci/check v0.0.0-20180506172741-cfe4005ccda2
go: downloading github.com/mitchellh/mapstructure v1.1.2
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/golangci/lint-1 v0.0.0-20191013205115-297bf364a8e0
go: downloading github.com/go-lintpack/lintpack v0.5.2
go: downloading github.com/golangci/ineffassign v0.0.0-20190609212857-42439a7714cc
go: downloading github.com/golangci/go-misc v0.0.0-20180628070357-927a3d87b613
go: downloading github.com/gofrs/flock v0.0.0-20190320160742-5135e617513b
go: downloading github.com/magiconair/properties v1.8.1
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/bombsimon/wsl/v2 v2.0.0
go: downloading sourcegraph.com/sqs/pbtypes v1.0.0
go: downloading github.com/fsnotify/fsnotify v1.4.7
go: downloading github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e
go: downloading github.com/pelletier/go-toml v1.6.0
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/golangci/revgrep v0.0.0-20180812185044-276a5c0a1039
go: downloading mvdan.cc/unparam v0.0.0-20191111180625-960b1ec0f2c2
go: downloading github.com/subosito/gotenv v1.2.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.28
go: downloading golang.org/x/mod v0.3.0
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/time v0.0.0-20191024005414-555d28b269f0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go: downloading github.com/prometheus/client_golang v1.3.0
go: downloading mvdan.cc/interfacer v0.0.0-20180901003855-c20040233aed
go: downloading github.com/golangci/unconvert v0.0.0-20180507085042-28b1c447d1f4
go: downloading github.com/matoous/godox v0.0.0-20190911065817-5d6d842e92eb
go: downloading mvdan.cc/lint v0.0.0-20170908181259-adc824a0674b
go: downloading github.com/golangci/goconst v0.0.0-20180610141641-041c5f2b40f3
go: downloading github.com/prometheus/common v0.7.0
go: downloading github.com/golangci/dupl v0.0.0-20180902072040-3e9179ac440a
go: finding module for package google.golang.org/grpc/naming
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/kisielk/gotool v1.0.0
go: downloading github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f
go: downloading github.com/go-critic/go-critic v0.4.0
go: downloading go.uber.org/zap v1.13.0
go: downloading google.golang.org/grpc v1.34.0
go: downloading github.com/go-toolsmith/astequal v1.0.0
go: downloading github.com/go-toolsmith/astcast v1.0.0
go: downloading github.com/json-iterator/go v1.1.9
go: downloading github.com/mattn/go-runewidth v0.0.7
go: downloading github.com/olekukonko/tablewriter v0.0.4
go: downloading github.com/prometheus/procfs v0.0.8
go: downloading github.com/google/btree v1.0.0
go: downloading github.com/OpenPeeDeeP/depguard v1.0.1
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/gobwas/glob v0.2.3
go: downloading github.com/cespare/xxhash/v2 v2.1.1
go: downloading github.com/go-toolsmith/strparse v1.0.0
go: downloading github.com/go-toolsmith/astfmt v1.0.0
go: downloading github.com/coreos/go-semver v0.3.0
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading github.com/prometheus/client_model v0.1.0
go: downloading github.com/timakin/bodyclose v0.0.0-20190930140734-f7f2e9bca95e
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.1.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading go.uber.org/multierr v1.4.0
go: downloading github.com/nbutton23/zxcvbn-go v0.0.0-20180912185939-ae427f1e4c1d
go: downloading golang.org/x/lint v0.0.0-20200302205851-738671d3881b
go: downloading github.com/gostaticanalysis/analysisutil v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading go.uber.org/atomic v1.5.1
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.12.1
go: downloading github.com/go-toolsmith/typep v1.0.0
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading github.com/gorilla/websocket v1.4.1
go: downloading go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee
go: downloading github.com/go-toolsmith/astp v1.0.0
go: downloading github.com/BurntSushi/toml v0.3.1
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/go-toolsmith/astcopy v1.0.0
go: downloading golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: downloading google.golang.org/appengine v1.6.6
go: downloading github.com/coreos/bbolt v1.3.3
go: downloading go.opencensus.io v0.22.4
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/google/go-cmp v0.5.2
github.com/smallstep/cli/command/certificate imports
        github.com/smallstep/certinfo imports
        github.com/google/certificate-transparency-go imports
        go.etcd.io/etcd imports
        github.com/coreos/etcd/etcdmain imports
        github.com/coreos/etcd/proxy/grpcproxy imports
        google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.34.0), but does not contain package google.golang.org/grpc/naming
builder for '/nix/store/jj0j1vz9gqk24sl6brlabyzdh6zhkhym-step-cli-0.15.3-go-modules.drv' failed with exit code 1
cannot build derivation '/nix/store/x3ygbhsf4i1lw5dg6scjpqcmks92bakm-step-cli-0.15.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/x3ygbhsf4i1lw5dg6scjpqcmks92bakm-step-cli-0.15.3.drv' failed

This has been addressed upstream in smallstep/cli#394, but there's no stable release containing it.

Simply applying on top of 0.15.3 also doesn't work, so I moved this to the commit introducing the fix.

};

goDeps = ./deps.nix;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That file should be deleted.

pkgs/tools/security/step-cli/default.nix Show resolved Hide resolved
pkgs/tools/security/step-ca/default.nix Show resolved Hide resolved
@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 93711 run on x86_64-linux 1

2 packages built:
  • step-ca
  • step-cli

@SuperSandro2000
Copy link
Member

Result of nixpkgs-review pr 93711 run on x86_64-darwin 1

1 package failed to build:
  • step-ca
1 package built:
  • step-cli
--- FAIL: TestTLSALPN01Validate (0.00s)
    --- FAIL: TestTLSALPN01Validate/ok/timeout (0.00s)
panic: httptest: failed to listen on a port: listen tcp6 [::1]:0: bind: operation not permitted [recovered]
        panic: httptest: failed to listen on a port: listen tcp6 [::1]:0: bind: operation not permitted

goroutine 226 [running]:
testing.tRunner.func1.1(0x473f620, 0xc000586630)
        testing/testing.go:1072 +0x30d
testing.tRunner.func1(0xc000103380)
        testing/testing.go:1075 +0x41a
panic(0x473f620, 0xc000586630)
        runtime/panic.go:969 +0x1b9
net/http/httptest.newLocalListener(0x47ca560, 0xc00058a2c0)
        net/http/httptest/server.go:71 +0x237
net/http/httptest.NewUnstartedServer(...)
        net/http/httptest/server.go:117
github.com/smallstep/certificates/acme.newTestTLSALPNServer(0x0, 0xc000103380, 0x0)
        github.com/smallstep/certificates/acme/challenge_test.go:1622 +0x45
github.com/smallstep/certificates/acme.TestTLSALPN01Validate.func4(0xc000103380, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        github.com/smallstep/certificates/acme/challenge_test.go:1078 +0x329
github.com/smallstep/certificates/acme.TestTLSALPN01Validate.func16(0xc000103380)
        github.com/smallstep/certificates/acme/challenge_test.go:1591 +0x5a
testing.tRunner(0xc000103380, 0xc0005865c0)
        testing/testing.go:1123 +0xef
created by testing.(*T).Run
        testing/testing.go:1168 +0x2b3
FAIL    github.com/smallstep/certificates/acme  0.055s
FAIL

arianvp and others added 2 commits December 25, 2020 01:06
Co-Authored-By: Florian Klink <flokli@flokli.de>
Co-Authored-By: Florian Klink <flokli@flokli.de>
@marsam marsam merged commit b0fc889 into NixOS:master Dec 26, 2020
@flokli
Copy link
Contributor

flokli commented Dec 26, 2020

@marsam shouldn't we disable the tests on darwin, or mark it as broken there?

@AluisioASG
Copy link
Contributor

Given what the error is, it may just need that one Darwin networking flag to be set.

@marsam
Copy link
Contributor

marsam commented Dec 26, 2020

It builds on darwin, the error shown is because nixpkgs-review uses sandboxed builds and the tests fail wanting networking with loopback (allowed on sanbox).
Sandboxing is not fully functional on darwin, so I think we can safely ignore it.

@AluisioASG AluisioASG mentioned this pull request Jan 24, 2021
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants