Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ruby-modules/gem: fix path to git checkout #58940

Merged
merged 1 commit into from Apr 7, 2019
Merged

ruby-modules/gem: fix path to git checkout #58940

merged 1 commit into from Apr 7, 2019

Conversation

erictapen
Copy link
Member

Motivation for this change

I want to build the following bundlerEnv, that is not so far from my actual case (packaging Mastodon).

with import <nixpkgs> {};

bundlerEnv {
  name = "ruby-env";
  inherit (pkgs) ruby_2_6;

  gemfile = builtins.toFile "Gemfile" ''
    gem 'http_parser.rb', '~> 0.6', git: 'https://github.com/tmm1/http_parser.rb', ref: '225354b3d94d8d76774bac5bb2ac0139dd75662c'
  '';
  lockfile = builtins.toFile "Gemfile.lock" ''
    GIT
      remote: https://github.com/tmm1/http_parser.rb
      revision: 225354b3d94d8d76774bac5bb2ac0139dd75662c
      ref: 225354b3d94d8d76774bac5bb2ac0139dd75662c
      specs:
        http_parser.rb (0.6.1)
  '';
  # generated by bundix
  gemset = {
    "http_parser.rb" = {
      groups = ["default"];
      platforms = [];
      source = {
        fetchSubmodules = true;
        rev = "225354b3d94d8d76774bac5bb2ac0139dd75662c";
        sha256 = "00avimm11xnpsylgvr1rrcxr9ijsa3rygvqj8fzil0g73cypb7fx";
        type = "git";
        url = "https://github.com/tmm1/http_parser.rb";
      };
      version = "0.6.1";
    };
  };

}

Building that on master fails with an error:

$ nix-build
these derivations will be built:
  /nix/store/cgrnanx411r0a3ah4fvyki0lbrbs73x8-ruby2.5.5-http_parser.rb-0.6.1.drv
  /nix/store/5sz7fv9ipr9asvlch2x72k7b3k1a4rvr-ruby-env.drv
building '/nix/store/cgrnanx411r0a3ah4fvyki0lbrbs73x8-ruby2.5.5-http_parser.rb-0.6.1.drv'...
unpacking sources
unpacking source archive /nix/store/rfgsdvv1mxi6kj07xiijhzhz8dps65h3-http_parser.rb-225354b
source root is http_parser.rb-225354b
patching sources
configuring
no configure script, doing nothing
building
Initialized empty Git repository in /build/http_parser.rb-225354b/.git/
installing
buildFlags:
`/homeless-shelter` is not a directory.
Bundler will use `/build/bundler/home/unknown' as your home directory temporarily.
/nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1227:in `stat': No such file or directory @ rb_file_s_stat - /nix/store/rfgsdvv1mxi6kj07xiijhzhz8dps65h3-http_parser.rb-225354b/.git (Errno::ENOENT)
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1227:in `lstat'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1256:in `copy'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:418:in `block in copy_entry'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1390:in `wrap_traverse'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:415:in `copy_entry'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:393:in `block in cp_r'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1463:in `block in fu_each_src_dest'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1479:in `fu_each_src_dest0'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1461:in `fu_each_src_dest'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:392:in `cp_r'
        from /nix/store/6g99037s460cbv2drh4l7hrny09lc19q-nix-bundle-install.rb:92:in `checkout'
        from /nix/store/q3m22q8z2rsq1hjy6lsv4amavyf872lw-bundler-1.17.2/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/bundler/source/git.rb:303:in `fetch'
        from /nix/store/q3m22q8z2rsq1hjy6lsv4amavyf872lw-bundler-1.17.2/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/bundler/source/git.rb:161:in `specs'
        from /nix/store/6g99037s460cbv2drh4l7hrny09lc19q-nix-bundle-install.rb:119:in `<main>'
builder for '/nix/store/cgrnanx411r0a3ah4fvyki0lbrbs73x8-ruby2.5.5-http_parser.rb-0.6.1.drv' failed with exit code 1
cannot build derivation '/nix/store/5sz7fv9ipr9asvlch2x72k7b3k1a4rvr-ruby-env.drv': 1 dependencies couldn't be built
error: build of '/nix/store/5sz7fv9ipr9asvlch2x72k7b3k1a4rvr-ruby-env.drv' failed

As No such file or directory @ rb_file_s_stat - /nix/store/rfgsdvv1mxi6kj07xiijhzhz8dps65h3-http_parser.rb-225354b/.git (Errno::ENOENT) says, the .git directory needs to accessed but is missing. That makes me wonder, as in

nixpkgs/pkgs/development/ruby-modules/gem/default.nix

Line 146 in e4d2143

git init
there is specific code to create that .git dir.

Things done

Finally I came to the conclusion, that the wrong path was given to nix-bundle-install.rb and it is supposed to be the build dir, not the $src store path. That makes the former bundlerEnv build.

cc'ing @cstrahan as they wrote that code in b6c06e2. Also @nyarly, you touched that file recently.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@erictapen
ruby-modules/gem: fix path to git checkout
eb9293e 
In case of the gem type 'git', nix-bundle-install.rb was called with
wrong path to the git checkout.

${src} does contain the sources, but not the newly generated .git dir,
which is created in the buildPhase of pkgs/development/ruby-modules/gem/default.nix

In some rare cases, this .git dir is needed at installPhase.
@erictapen
Copy link
Member Author

Ah, what i forgot to mention: If this is actually a fix, I'd say it's worth to backport to nixos-19.03 and nixos-18.09.

nyarly
nyarly reviewed Apr 3, 2019
View reviewed changes
pkgs/development/ruby-modules/gem/default.nix
@@ -179,7 +179,7 @@ stdenv.mkDerivation ((builtins.removeAttrs attrs ["source"]) // {
'${version}' \
'${lib.escapeShellArgs buildFlags}' \
'${attrs.source.url}' \
'${src}' \
'.' \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want to use $NIX_BUILD_TOP here.

Copy link
Member Author

@erictapen erictapen Apr 4, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, didn't know about that variable. If I do "$NIX_BUILD_TOP" \ for that line, I still get this error on my test case:

$ nix-build
these derivations will be built:
  /nix/store/3hfcigzb57ijdvs6msn0ik4lzk73s9nv-ruby2.5.5-http_parser.rb-0.6.1.drv
  /nix/store/ana9rq1r399wnmvlv9kxw76zxmjdqgqg-ruby-env.drv
building '/nix/store/3hfcigzb57ijdvs6msn0ik4lzk73s9nv-ruby2.5.5-http_parser.rb-0.6.1.drv'...
unpacking sources
unpacking source archive /nix/store/rfgsdvv1mxi6kj07xiijhzhz8dps65h3-http_parser.rb-225354b
source root is http_parser.rb-225354b
patching sources
configuring
no configure script, doing nothing
building
Initialized empty Git repository in /build/http_parser.rb-225354b/.git/
installing
buildFlags:
`/homeless-shelter` is not a directory.
Bundler will use `/build/bundler/home/unknown' as your home directory temporarily.
/nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1227:in `stat': No such file or directory @ rb_file_s_stat - /build/.git (Errno::ENOENT)
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1227:in `lstat'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1256:in `copy'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:418:in `block in copy_entry'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1390:in `wrap_traverse'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:415:in `copy_entry'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:393:in `block in cp_r'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1463:in `block in fu_each_src_dest'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1479:in `fu_each_src_dest0'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1461:in `fu_each_src_dest'
        from /nix/store/rwp5fpzqssf5m9dzbgbwsfgdzw8xajra-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:392:in `cp_r'
        from /nix/store/6g99037s460cbv2drh4l7hrny09lc19q-nix-bundle-install.rb:92:in `checkout'
        from /nix/store/q3m22q8z2rsq1hjy6lsv4amavyf872lw-bundler-1.17.2/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/bundler/source/git.rb:303:in `fetch'
        from /nix/store/q3m22q8z2rsq1hjy6lsv4amavyf872lw-bundler-1.17.2/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/bundler/source/git.rb:161:in `specs'
        from /nix/store/6g99037s460cbv2drh4l7hrny09lc19q-nix-bundle-install.rb:119:in `<main>'
builder for '/nix/store/3hfcigzb57ijdvs6msn0ik4lzk73s9nv-ruby2.5.5-http_parser.rb-0.6.1.drv' failed with exit code 1
cannot build derivation '/nix/store/ana9rq1r399wnmvlv9kxw76zxmjdqgqg-ruby-env.drv': 1 dependencies couldn't be built
error: build of '/nix/store/ana9rq1r399wnmvlv9kxw76zxmjdqgqg-ruby-env.drv' failed

Is there any case where . could go wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose there's not. It seems weird to have the parameter that only receives (AFAICT) a constant value, though. Either it should be removed as a parameter (and the receiving script updated appropriately) or the variation is significant and there's something we're missing?

Thanks for making the experiment. I'm sorry for the bum steer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm quite curious about wether I missed something. Makes sense what you say about that constant parameter. Let's see wether someone else finds a problem here before I refactor that.

@manveru
Copy link
Contributor

manveru commented Apr 7, 2019

This seems like a good fix to me and doens't break anything I checked.

@nixos-discourse
Copy link

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/8

@ryantm ryantm merged commit 488ecd6 into NixOS:master Apr 7, 2019
@erictapen erictapen deleted the ruby-gem-src-fix branch April 8, 2019 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nyarly nyarly nyarly left review comments

@manveru manveru manveru approved these changes

@alyssais alyssais Awaiting requested review from alyssais

@zimbatm zimbatm Awaiting requested review from zimbatm

Assignees
No one assigned
Labels
6.topic: ruby 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@erictapen @manveru @nixos-discourse @nyarly @ryantm @GrahamcOfBorg