Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitlab: 12.1.6 -> 12.2.3 #67770

Closed
wants to merge 10 commits into from
Closed

gitlab: 12.1.6 -> 12.2.3 #67770

wants to merge 10 commits into from

Conversation

flokli
Copy link
Contributor

@flokli flokli commented Aug 30, 2019

Motivation for this change

https://about.gitlab.com/2019/08/29/security-release-gitlab-12-dot-2-dot-3-released/

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@flokli flokli added 1.severity: security 9.needs: port to stable A PR needs a backport to the stable release. labels Aug 30, 2019
@flokli
Copy link
Contributor Author

flokli commented Aug 30, 2019

@GrahamcOfBorg test gitlab

@flokli
Copy link
Contributor Author

flokli commented Aug 30, 2019

@GrahamcOfBorg test gitlab

Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the nixosTest for this may need to be changed somewhat, seems there's an issue with redis:

nixosTests.gitlab logs
gitlab# [ 1737.295028] gitaly[672]: time="2019-08-30T15:43:02Z" level=info msg="\tfrom /nix/store/aqg0yb1d7bckrvppmi3kb4yrl0fwg6a4-gitaly-1.59.2-ruby/bin/gitaly-ruby:8:in `<main>'" supervisor.args="[bundle exec bin/ruby-cd /var/gitlab/state/home /nix/store/aqg0yb1d7bckrvppmi3kb4yrl0fwg6a4-gitaly-1.59.2-ruby/bin/gitaly-ruby 672 /tmp/gitaly-ruby840197256/socket.0]" supervisor.name=gitaly-ruby.0
gitlab# [ 1737.308437] gitaly[672]: time="2019-08-30T15:43:02Z" level=warning msg=exited error="exit status 1" supervisor.args="[bundle exec bin/ruby-cd /var/gitlab/state/home /nix/store/aqg0yb1d7bckrvppmi3kb4yrl0fwg6a4-gitaly-1.59.2-ruby/bin/gitaly-ruby 672 /tmp/gitaly-ruby840197256/socket.0]" supervisor.name=gitaly-ruby.0
gitlab# [ 1737.325162] gitaly[672]: time="2019-08-30T15:43:03Z" level=warning msg=spawned supervisor.args="[bundle exec bin/ruby-cd /var/gitlab/state/home /nix/store/aqg0yb1d7bckrvppmi3kb4yrl0fwg6a4-gitaly-1.59.2-ruby/bin/gitaly-ruby 672 /tmp/gitaly-ruby840197256/socket.0]" supervisor.name=gitaly-ruby.0 supervisor.pid=4393
gitlab# [ 1737.332037] unicorn[1721]: I, [2019-08-30T15:43:03.012863 #1958]  INFO -- : Completed 500 Internal Server Error in 93ms (ActiveRecord: 19.6ms)
gitlab# [ 1737.338976] unicorn[1721]: F, [2019-08-30T15:43:03.016814 #1958] FATAL -- :
gitlab# [ 1737.340246] unicorn[1721]: curl: (22) The requested URL returned error: 500 Internal Server Error
gitlab# F, [2019-08-30T15:43:03.016880 #1958] FATAL -- : Redis::CannotConnectError (Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)):
gitlab# [ 1737.352113] unicorn[1721]: F, [2019-08-30T15:43:03.016905 #1958] FATAL -- :
gitlab# [ 1737.357206] unicorn[1721]: F, [2019-08-30T15:43:03.016945 #1958] FATAL -- : config/initializers/zz_metrics.rb:204:in `connect'
gitlab# [ 1737.358761] unicorn[1721]: lib/gitlab/anonymous_session.rb:12:in `block in store_session_id_per_ip'
gitlab: exit status 22
(0.52 seconds)
gitlab# [ 1737.360186] unicorn[1721]: lib/gitlab/redis/wrapper.rb:19:in `block in with'
gitlab# [ 1737.363438] unicorn[1721]: lib/gitlab/redis/wrapper.rb:19:in `with'
gitlab# [ 1737.364355] unicorn[1721]: lib/gitlab/anonymous_session.rb:11:in `store_session_id_per_ip'
gitlab# [ 1737.372307] unicorn[1721]: app/controllers/sessions_controller.rb:156:in `store_unauthenticated_sessions'
gitlab# [ 1737.373695] unicorn[1721]: lib/gitlab/session.rb:11:in `with_session'
gitlab# [ 1737.374698] unicorn[1721]: app/controllers/application_controller.rb:450:in `set_session_storage'
gitlab# [ 1737.375931] unicorn[1721]: lib/gitlab/i18n.rb:55:in `with_locale'
gitlab# [ 1737.376900] unicorn[1721]: lib/gitlab/i18n.rb:61:in `with_user_locale'
gitlab# [ 1737.377908] unicorn[1721]: app/controllers/application_controller.rb:444:in `set_locale'
gitlab# [ 1737.379322] unicorn[1721]: lib/gitlab/middleware/rails_queue_duration.rb:27:in `call'
gitlab# [ 1737.381066] unicorn[1721]: lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
gitlab# [ 1737.392593] unicorn[1721]: lib/gitlab/metrics/transaction.rb:57:in `run'
gitlab# [ 1737.417526] unicorn[1721]: lib/gitlab/metrics/rack_middleware.rb:17:in `call'
gitlab# [ 1737.418740] unicorn[1721]: lib/gitlab/middleware/multipart.rb:103:in `call'
gitlab# [ 1737.422671] unicorn[1721]: lib/gitlab/request_profiler/middleware.rb:17:in `call'
gitlab# [ 1737.439977] unicorn[1721]: lib/gitlab/middleware/go.rb:20:in `call'
gitlab# [ 1737.463483] unicorn[1721]: lib/gitlab/etag_caching/middleware.rb:13:in `call'
gitlab# [ 1737.478489] unicorn[1721]: lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
gitlab# [ 1737.479683] unicorn[1721]: lib/gitlab/middleware/correlation_id.rb:15:in `call'
gitlab# [ 1737.481667] unicorn[1721]: lib/gitlab/middleware/read_only/controller.rb:40:in `call'
gitlab# [ 1737.483693] unicorn[1721]: lib/gitlab/middleware/read_only.rb:18:in `call'
gitlab# [ 1737.485630] unicorn[1721]: lib/gitlab/middleware/basic_health_check.rb:25:in `call'
gitlab# [ 1737.499078] unicorn[1721]: lib/gitlab/request_context.rb:26:in `call'
gitlab# [ 1737.520719] unicorn[1721]: lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
gitlab# [ 1737.538547] unicorn[1721]: lib/gitlab/middleware/release_env.rb:12:in `call'
gitlab# [ 1737.551596] gitlab-workhorse[674]: gitlab 127.0.0.1 - - [2019/08/30:15:43:03 +0000] "GET /users/sign_in HTTP/1.0" 500 2926 "" "curl/7.65.3" %!f(int64=306)
gitlab# [ 1738.339770] gitaly[672]: time="2019-08-30T15:43:04Z" level=info msg="PID 4381 BUNDLE_GEMFILE=/nix/store/r7kic887qaa3amg1y9s0jrf126s0g1cg-gemfile-and-lockfile/Gemfile" supervisor.args="[bundle exec bin/ruby-cd /var/gitlab/state/home /nix/store/aqg0yb1d7bckrvppmi3kb4yrl0fwg6a4-gitaly-1.59.2-ruby/bin/gitaly-ruby 672 /tmp/gitaly-ruby840197256/socket.1]" supervisor.name=gitaly-ruby.1
(1165.16 seconds)
error: action timed out after -1 seconds at /nix/store/hiwds9rprdwsh5a6jzcgcmp5aal14r0d-nixos-test-driver/lib/perl5/site_perl/Machine.pm line 247, <__ANONIO__> line 81506.
(1739.35 seconds)
action timed out after -1 seconds at /nix/store/hiwds9rprdwsh5a6jzcgcmp5aal14r0d-nixos-test-driver/lib/perl5/site_perl/Machine.pm line 247, <__ANONIO__> line 81506.
cleaning up
killing gitlab (pid 597)
(0.00 seconds)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/71l3l09pbs1xnr8xs60xr9srf0flnnsa-vm-test-run-gitlab.drv' failed with exit code 255
error: build of '/nix/store/71l3l09pbs1xnr8xs60xr9srf0flnnsa-vm-test-run-gitlab.drv' failed

@flokli
Copy link
Contributor Author

flokli commented Aug 31, 2019

@jonringer I added a PR fixing redis in #67845. Would be great if you could take a look there, too!

@flokli
Copy link
Contributor Author

flokli commented Sep 1, 2019

@GrahamcOfBorg test gitlab

@flokli
Copy link
Contributor Author

flokli commented Sep 1, 2019

@globin there still seems to be something broken in how gitaly's gitaly_server.rb tries to require gitaly:

gitlab# [  148.646198] gitaly[673]: time="2019-09-01T16:39:53Z" level=info msg="/nix/store/wyjk1ljp9a5aadiswgir3hii8wa681fq-gitaly-1.59.2-ruby/lib/gitaly_server.rb:2:in `require': cannot load such file -- gitaly (LoadError)" supervisor.args="[bundle exec 
bin/ruby-cd /var/gitlab/state/home /nix/store/wyjk1ljp9a5aadiswgir3hii8wa681fq-gitaly-1.59.2-ruby/bin/gitaly-ruby 673 /tmp/gitaly-ruby371089635/socket.0]" supervisor.name=gitaly-ruby.0

@flokli
Copy link
Contributor Author

flokli commented Sep 7, 2019

ping @globin @fpletz @lheckemann

@bgamari
Copy link
Contributor

bgamari commented Sep 10, 2019

@flokli, isn't gitaly-ruby now completely removed?

@lheckemann lheckemann added this to the 19.09 milestone Sep 10, 2019
@flokli
Copy link
Contributor Author

flokli commented Sep 10, 2019

@bgamari I didn't quite look into gitaly - but there's still a ruby folder in there and unfortunately, CHANGELOG.md is pretty sparse.

@schneefux
Copy link
Contributor

schneefux commented Sep 19, 2019

It doesn't build on my NixOS 19.09 server:

log

### Building gitlab-zip-metadata
go build -ldflags "-X main.Version=8.8.1" -tags "tracer_static tracer_static_jaeger" -o /build/source/gitlab-zip-metadata gitlab.com/gitlab-org/gitlab-workhorse/cmd/gitlab-zip-metadata
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/bin
  Gem home: /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0
Building native extensions. This could take a while...
### Building gitlab-workhorse
go build -ldflags "-X main.Version=8.8.1" -tags "tracer_static tracer_static_jaeger" -o /build/source/gitlab-workhorse gitlab.com/gitlab-org/gitlab-workhorse
Successfully installed github-linguist-6.4.1
1 gem installed
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/lib/linguist/linguist.so
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/ext/linguist/strndup.o
wrong ELF type
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/ext/linguist/linguist.o
wrong ELF type
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/ext/linguist/linguist.so
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/ext/linguist/lex.linguist_yy.o
wrong ELF type
shrinking /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/extensions/x86_64-linux/2.6.0/github-linguist-6.4.1/linguist/linguist.so
patching script interpreter paths in /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1
/nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/bin/git-linguist: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/1agnq6dzfm68n6dkk05m1j0bjc30ycw0-ruby-2.6.4/bin/ruby"
/nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1/lib/ruby/gems/2.6.0/gems/github-linguist-6.4.1/bin/linguist: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/1agnq6dzfm68n6dkk05m1j0bjc30ycw0-ruby-2.6.4/bin/ruby"
checking for references to /build/ in /nix/store/kk96k065d38fz1vq1mpq929y9cwlb9lg-ruby2.6.4-github-linguist-6.4.1...
wrong ELF type
wrong ELF type
wrong ELF type
building '/nix/store/xv1fmbbagaka6f1v1lcca7ps3sy4jhcj-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/jafmxk01zp10jx72pj13137i637lz1js-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2/lib/ruby/gems/2.6.0/bin
  Gem home: /nix/store/jafmxk01zp10jx72pj13137i637lz1js-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2/lib/ruby/gems/2.6.0
Successfully installed gitlab-gollum-rugged_adapter-0.4.4.2
1 gem installed
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/jafmxk01zp10jx72pj13137i637lz1js-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2
patching script interpreter paths in /nix/store/jafmxk01zp10jx72pj13137i637lz1js-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2
checking for references to /build/ in /nix/store/jafmxk01zp10jx72pj13137i637lz1js-ruby2.6.4-gitlab-gollum-rugged_adapter-0.4.4.2...
building '/nix/store/7n3jy9x9vghwn4cc26miwhqll7h8b2z7-ruby2.6.4-licensee-8.9.2.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2/lib/ruby/gems/2.6.0/bin
  Gem home: /nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2/lib/ruby/gems/2.6.0
Successfully installed licensee-8.9.2
1 gem installed
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2
patching script interpreter paths in /nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2
/nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2/lib/ruby/gems/2.6.0/gems/licensee-8.9.2/bin/licensee: interpreter directive changed from "/usr/bin/env ruby" to "/nix/store/1agnq6dzfm68n6dkk05m1j0bjc30ycw0-ruby-2.6.4/bin/ruby"
checking for references to /build/ in /nix/store/00mrhx2qs8j703lihs652v2njqn3rcgz-ruby2.6.4-licensee-8.9.2...
building '/nix/store/frdg1h74yc22kp4d0pk5flig61jxnnh2-source.drv'...

trying https://gitlab.com/api/v4/projects/gitlab-org%2Fgitaly/repository/archive.tar.gz?sha=v1.59.2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2664k    0 2664k    0     0  2180k      0 --:--:--  0:00:01 --:--:-- 2182k
unpacking source archive /build/archive.tar.gz?sha=v1.59.2
building '/nix/store/p922i4q7lqd8m49n56vlnxj1q2szzpll-gitaly-env.drv'...
created 330 symlinks in user environment
building '/nix/store/hm1ip909xrshl727cv0axgzpw5jkirbg-source.drv'...

trying https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/repository/archive.tar.gz?sha=v12.2.3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
installing
install flags: SHELL=/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash PREFIX=\$\(out\) VERSION=8.8.1 GOCACHE=\$\(TMPDIR\)/go-cache install
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download source from any mirror
builder for '/nix/store/hm1ip909xrshl727cv0axgzpw5jkirbg-source.drv' failed with exit code 1
cannot build derivation '/nix/store/b7m80z54355440vgwvdc9vcrwnxp6g1x-gitlab-12.2.3.drv': 1 dependencies couldn't be built
building '/nix/store/pqp172hjrlb1dzvb4fqjnz334amqvab2-sys-953cdad.drv'...
building '/nix/store/s62glnd9hsd5mxd0i4c8zjvmjjz6b0lw-wrapped-ruby-gitaly-env.drv'...
cannot build derivation '/nix/store/rw5lxxc0r862x3pai44lcq5qap0gmrb2-etc-nixos.conf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/v7gcbbryvfb8pmxpgbzpr65j1k2iycbq-gitlab-rails.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/qkvm3jgz6qfmqjr63i1r22z6s4995dkg-gitlab-rake.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/077ag2dkwdjz61589dv8fxc0n7gbs0hx-unit-gitlab-sidekiq.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/s8ydjn5hlvp447kli027mv899wzy1sil-unit-gitlab-workhorse.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/hadzbi493gj83p8jn59a6v45qgxla7r0-unit-gitlab.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3d4lrz9m134hwlpajwrdxj5vh0ci31zy-etc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3if7s6nlk3jvpfy5pjjwpgi4jvx39a0q-system-path.drv': 1 dependencies couldn't be built

@flokli
Copy link
Contributor Author

flokli commented Sep 21, 2019

We fixed gitaly - the proto folder was missing.

Also bumped from 12.2.3 to 12.2.5

@flokli
Copy link
Contributor Author

flokli commented Sep 21, 2019

@GrahamcOfBorg test gitlab

@flokli
Copy link
Contributor Author

flokli commented Sep 21, 2019

@test gitlab

@flokli
Copy link
Contributor Author

flokli commented Sep 21, 2019

Ah, it seems this still fails due to https://gitlab.com/groups/gitlab-org/-/epics/802#note_217103588 - we should update the update scripts, locations and derivations…

@bgamari
Copy link
Contributor

bgamari commented Sep 23, 2019

I have pushed a continuation of this work to #69325.

@flokli
Copy link
Contributor Author

flokli commented Sep 24, 2019

Thanks, so let's close this here.

@flokli flokli closed this Sep 24, 2019
@TredwellGit TredwellGit removed the 9.needs: port to stable A PR needs a backport to the stable release. label Aug 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants