Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gitlab: 11.4.4 -> 11.5.0 #50962

Merged
merged 12 commits into from Nov 26, 2018
Merged

gitlab: 11.4.4 -> 11.5.0 #50962

merged 12 commits into from Nov 26, 2018

Conversation

flokli
Copy link
Contributor

@flokli flokli commented Nov 23, 2018

Motivation for this change

This updates gitlab to version 11.5.0, released on Nov 22, and all sidecar processes to the versions compatible with 11.5.0.

This includes the critical security fix in 11.4.6, which is currently missing in nixpkgs.

As upgrading gitlab and it's components was a very manual task until now, I added an update.py script, automating most of the process. It get's the latest stable tag for both GitLab CE and EE, gathers data about compatible sidecar packages (gitlab-shell, gitaly, gitlab-workhorse) into a data.json (which is used by the gitlab expression), then updates bundlerenvs where necessary, and bumps the sidecar packages.

For some weird reason in how go packages are being built, meta.position for gitlab-shell points to pkgs/development/go-modules/generic/default.nix, so update-source-version isn't able to bump this automatically (but we show the version and hash to bump to on the console).

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@flokli flokli requested review from globin and Mic92 November 23, 2018 17:09
@globin
Copy link
Member

globin commented Nov 23, 2018

@GrahamcOfBorg test gitlab

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.gitlab

Partial log (click to expand)

Cannot nix-instantiate `tests.gitlab' because:
error: while evaluating 'recursiveUpdate' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:415:26, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:384:37, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:347:21, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:332:33, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/lib/attrsets.nix:347:27:
while evaluating the attribute 'gitlab' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/nixos/tests/all-tests.nix:80:3:
while evaluating 'handleTest' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/nixos/tests/all-tests.nix:17:22, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/nixos/tests/all-tests.nix:80:12:
while evaluating 'discoverTests' at /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/nixos/tests/all-tests.nix:13:19, called from /var/lib/gc-of-borg/nix-test-rs-22/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-22/nixos/tests/all-tests.nix:18:5:
access to path '/nix/store/zb46dwfzqkmvchs7qggp8509agxza6y0-grahamc-aarch64-community-22' is forbidden in restricted mode

@GrahamcOfBorg
Copy link

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.gitlab

Partial log (click to expand)

error: while evaluating 'recursiveUpdate' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:415:26, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:384:37, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:21, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:332:33, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:27:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/lists.nix:113:41, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:46:
while evaluating the attribute 'gitlab' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:80:3:
while evaluating 'handleTest' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:17:22, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:80:12:
while evaluating 'discoverTests' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:13:19, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:18:5:
access to path '/nix/store/1zfs1zc7sv7iyl5nkx6bvql2nk97044w-gleber-bastion' is forbidden in restricted mode

@flokli
Copy link
Contributor Author

flokli commented Nov 23, 2018

I think @GrahamcOfBorg is still borked.

@flokli
Copy link
Contributor Author

flokli commented Nov 23, 2018

While resurrecting my test setup, I keep getting redirected to localhost all the time (no matter what Host is being used in the HTTP request) - I think it has always been like this.

Can somebody with a more production-ready installation give this a test run?

@flokli
gitlab tests: enable recommendedProxySettings, test redirection works…
6870eaf 
… as it should

The nixos test is a bit misleading, as the given nginx configuration
would always cause gitlab to redirect to localhost, which is clearly not
what you want in a production setup.

Instead we now enable services.nginx.recommendedProxySettings,
curl against http://gitlab, and assure we get redirected to that same
hostname, too.
@flokli
Copy link
Contributor Author

flokli commented Nov 23, 2018 •
edited

Ok, this was caused by a missing services.nginx.recommendedProxySettings setting in my test setup.

As redirecting to localhost isn't desirable anyhow, I updated the nixos test to set this, and compare the redirect to not redirect to localhost.

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: gitaly, gitlab-shell, gitlab-workhorse

The following builds were skipped because they don't evaluate on x86_64-darwin: gitlab

Partial log (click to expand)

Successfully installed github-linguist-6.2.0
1 gem installed
post-installation fixup
patching script interpreter paths in /nix/store/rj6q2vra97zca98qiick5yrggqm0kf9s-ruby2.5.3-github-linguist-6.2.0
/nix/store/rj6q2vra97zca98qiick5yrggqm0kf9s-ruby2.5.3-github-linguist-6.2.0/lib/ruby/gems/2.5.0/gems/github-linguist-6.2.0/bin/git-linguist: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/q27qm9xz6hza35i76z0jmvxijhp9gb8m-ruby-2.5.3/bin/ruby"
/nix/store/rj6q2vra97zca98qiick5yrggqm0kf9s-ruby2.5.3-github-linguist-6.2.0/lib/ruby/gems/2.5.0/gems/github-linguist-6.2.0/bin/linguist: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/q27qm9xz6hza35i76z0jmvxijhp9gb8m-ruby-2.5.3/bin/ruby"
cannot build derivation '/nix/store/277dxkwwbsr10shw478sjzfv0m4c91by-gitaly-env.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/2p6i1nailjss6lg5g8ys4frmwgvj94x7-wrapped-ruby-gitaly-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2wjv2259q42pqv1m0p0nhgwjyv2ysgnh-gitaly-0.129.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/2wjv2259q42pqv1m0p0nhgwjyv2ysgnh-gitaly-0.129.0.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gitaly, gitlab, gitlab-shell, gitlab-workhorse

Partial log (click to expand)

strip is /nix/store/6dpnd5aniypn8124mmy8f88s4mq2zl07-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby/lib  /nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby/bin
patching script interpreter paths in /nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby
/nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby/bin/ruby-cd: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/30lfkb3jniq3y82273jkvncwn84jhmxd-wrapped-ruby-gitaly-env/bin/ruby"
/nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby/bin/gitaly-ruby: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/30lfkb3jniq3y82273jkvncwn84jhmxd-wrapped-ruby-gitaly-env/bin/ruby"
checking for references to /build in /nix/store/9bhd098js59ip3y1g87qk81kqszv8284-gitaly-0.129.0-ruby...
/nix/store/g22r9c2l8va4dm1jdml0bckwawpyrcrm-gitaly-0.129.0-bin
/nix/store/81zz2zpny35bwlqfyhg9l9cknh76z5lk-gitlab-11.5.0
/nix/store/d9x60nw8f8lp1yl494xi3ql0l2rzfxzz-gitlab-shell-8.4.1
/nix/store/jiblk339963xrakbs50bz624sd5wil2v-gitlab-workhorse-7.1.0

@flokli
Copy link
Contributor Author

flokli commented Nov 23, 2018

@GrahamcOfBorg test gitlab

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.gitlab

Partial log (click to expand)

Cannot nix-instantiate `tests.gitlab' because:
error: while evaluating 'recursiveUpdate' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:415:26, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:384:37, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:347:21, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:332:33, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/lib/attrsets.nix:347:27:
while evaluating the attribute 'gitlab' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/nixos/tests/all-tests.nix:80:3:
while evaluating 'handleTest' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/nixos/tests/all-tests.nix:17:22, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/nixos/tests/all-tests.nix:80:12:
while evaluating 'discoverTests' at /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/nixos/tests/all-tests.nix:13:19, called from /var/lib/gc-of-borg/nix-test-rs-10/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-10/nixos/tests/all-tests.nix:18:5:
access to path '/nix/store/lwiv21j5cmrh8x7j8dxrnbvd4prfp0g2-grahamc-aarch64-community-10' is forbidden in restricted mode

@GrahamcOfBorg
Copy link

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.gitlab

Partial log (click to expand)

Cannot nix-instantiate `tests.gitlab' because:
error: while evaluating 'recursiveUpdate' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:415:26, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:384:37, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:347:21, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:332:33, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/lib/attrsets.nix:347:27:
while evaluating the attribute 'gitlab' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/tests/all-tests.nix:80:3:
while evaluating 'handleTest' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/tests/all-tests.nix:17:22, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/tests/all-tests.nix:80:12:
while evaluating 'discoverTests' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/tests/all-tests.nix:13:19, called from /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/tests/all-tests.nix:18:5:
access to path '/nix/store/5kl28dpgkyxd4cl8pspnix3c0qrpfwi9-builder-0-gustav.ewr1.nix.ci' is forbidden in restricted mode

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gitaly, gitlab, gitlab-shell, gitlab-workhorse

Partial log (click to expand)

strip is /nix/store/rpbg8gmqxhz8g61p1plz5d2srs84pvmv-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby/lib  /nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby/bin
patching script interpreter paths in /nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby
/nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby/bin/gitaly-ruby: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/wrhy5njcaz0p0drpw0zpc4jdj0s6bg5j-wrapped-ruby-gitaly-env/bin/ruby"
/nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby/bin/ruby-cd: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/wrhy5njcaz0p0drpw0zpc4jdj0s6bg5j-wrapped-ruby-gitaly-env/bin/ruby"
checking for references to /build in /nix/store/gyrvgpisjh3h36bhhpq0wgm6s92vybzn-gitaly-0.129.0-ruby...
/nix/store/81ih53nlqf5k1k9bvpqd4lbnap0zqvf0-gitaly-0.129.0-bin
/nix/store/ybgzqmm6k2f861lli9855avs5qsdx81d-gitlab-11.5.0
/nix/store/xj9kmq8cgz68pkg1y7xzmzqs77qqv3zs-gitlab-shell-8.4.1
/nix/store/axwjsamwb0lc2zy71mvhvx81y7jgh04h-gitlab-workhorse-7.1.0

@flokli flokli merged commit 2c2ab68 into NixOS:master Nov 26, 2018
@flokli flokli deleted the gitlab-updater branch November 26, 2018 17:47
This was referenced Nov 26, 2018
Merged
Merged
@flokli flokli mentioned this pull request Dec 24, 2018
Closed
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@globin globin Awaiting requested review from globin

@Mic92 Mic92 Awaiting requested review from Mic92

Assignees
No one assigned
Labels
6.topic: nixos 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@flokli @globin @GrahamcOfBorg