Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitlab: 11.10.8 -> 12.0.3 #64550

Merged
merged 9 commits into from Jul 17, 2019
Merged

gitlab: 11.10.8 -> 12.0.3 #64550

merged 9 commits into from Jul 17, 2019

Conversation

bgamari
Copy link
Contributor

@bgamari bgamari commented Jul 10, 2019

This is a major version bump but things were generally straightforward
save two wrinkles:

  • it is necessary to ignore collisions in the gitlab bundler
    environment as both omniauth_oauth2_generic and
    apollo_upload_server provide a console executable.

  • grpc had to be patched since its build system expects the AR
    environment variable to contain not just the path to ar but
    also the rpc flags (see the discussion in nixpkgs [WIP] gitlab: 11.10.4 -> 11.11.3 #63056).

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@bgamari bgamari mentioned this pull request Jul 10, 2019
10 tasks
@bgamari
Copy link
Contributor Author

bgamari commented Jul 10, 2019

I have no idea why this is failing; it builds without any trouble locally.

@bgamari bgamari force-pushed the gitlab-12.0 branch 3 times, most recently from 3e80a8a to 60357af Compare July 10, 2019 03:06
name = "gitaly-env";
inherit ruby;
gemdir = ./.;
gemset =
let x = import "${gemdir}/gemset.nix";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes your failure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why in particular?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IFD.

@mmahut
Copy link
Member

mmahut commented Jul 10, 2019

The build fails for me locally with the same as the eval check from the borg.

name = "gitlab-env-${version}";
inherit ruby;
gemdir = ./rubyEnv- + "${if gitlabEnterprise then "ee" else "ce"}";
gemset =
let x = import "${gemdir}/gemset.nix";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let x = import "${gemdir}/gemset.nix";
let x = import (gemdir + "/gemset.nix");

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should pass strict eval.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for maybe offtopic, but do you know why? I don't understand it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When gemdir is used inside the string interpolation, the directory gets copied into the nix store.
If we then try to import gemset.nix from there, that's an import from derivation (IFD).

By assembling the path outside a string interpolation, we avoid that (because it stays just a path until x is used).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think the derivation is still in the "string context" of gemdir + "/gemset.nix so this is still IFD and still prohibited by strict mode.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ericson2314

All I can say is that

HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate ./pkgs/top-level/release.nix -A gitlab --option restrict-eval true --option build-timeout 1800 --argstr system thesystem --show-trace

does work with the suggested change, but doesn't with what's currently in the PR ;-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nevermind I thought we are computing something from a "real" derivation. IMO this is just working around a nix inconsistency.

This is a major version bump but things were generally straightforward
save two wrinkles:

 * it is necessary to ignore collisions in the gitlab bundler
   environment as both `omniauth_oauth2_generic` and
   `apollo_upload_server` provide a `console` executable.

 * grpc had to be patched since its build system expects the `AR`
   environment variable to contain not just the path to `ar` but
   also the `rpc` flags (see the discussion in nixpkgs NixOS#63056).
@flokli
Copy link
Contributor

flokli commented Jul 14, 2019

@GrahamcOfBorg test gitlab

@bgamari
Copy link
Contributor Author

bgamari commented Jul 15, 2019

Sorry, @flokli; I should have mentioned that after opening I realized that the gitaly derivation isn't producing any executables. I poked at it a bit but progress is slow as I'm not particularly familiar with Go.

@globin
Copy link
Member

globin commented Jul 16, 2019

Fixed gitaly, added gem groups that appear to be necessary now and cleaned up the dir/link creation/permission handling

@globin
Copy link
Member

globin commented Jul 16, 2019

@GrahamcOfBorg test gitlab

@bgamari
Copy link
Contributor Author

bgamari commented Jul 16, 2019

Thank you @globin!

@bgamari
Copy link
Contributor Author

bgamari commented Jul 16, 2019

@GrahamcOfBorg test gitlab

@globin
Copy link
Member

globin commented Jul 16, 2019

Fixed some gitlab-shell issues not caught by the nixos test

@globin
Copy link
Member

globin commented Jul 17, 2019

Has been running in production for 2 days without any complaints, so I guess this should be fine for master at least.

@globin globin merged commit 0972409 into NixOS:master Jul 17, 2019
@flokli
Copy link
Contributor

flokli commented Jul 18, 2019

Thanks!

@jslight90
Copy link
Contributor

jslight90 commented Jul 26, 2019

I am unable to build Gitaly using these changes. I receive errors that all of the hashes of the Go dependencies (specified in gitlab/gitaly/deps.nix) don't match.

I have tried checking out both master and commit a33ddd7 and receive the same errors.

@flokli flokli mentioned this pull request Jul 27, 2019
10 tasks
@flokli
Copy link
Contributor

flokli commented Jul 27, 2019

It seems the hashes of libgit2/git2go, lightstep/lightstep-tracer-go and uber/jaeger-client-go are now different 😕

I opened #65468 to fix that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants