New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ruby: --disable-rubygems for baseruby #61684
Conversation
This works just fine, and means we don't run into an issue with RubyGems trying to install into a different Ruby's prefix when cross-compiling. See NixOS#51842 (comment).
50e5213
to
bed62e8
Compare
@alyssais Thanks for your fix. Just so that I can test properly, can you please confirm that your fix is another solution to what I suggested (i.e. removing the |
@alyssais Thanks for your fix. Just so that I can test properly, can you please confirm that your fix is another solution to what I suggested (i.e. removing the `--destdir`) as a temporary workaround?
Yes, I believe so. Except this keeps the cross-build working as well,
and so I intend this change to be permanent.
|
At a glance, this looks safe for backporting to 19.03, since its rubies are exhibiting this issue. Do you think this is wrong? |
At a glance, this looks safe for backporting to 19.03, since its rubies are exhibiting this issue. Do you think this is wrong?
I think this should be safe to backport, providing it does actually fix
the issue.
|
Hmm, having an issue. Ruby 2.6's native
The PATH order here matters. The other way around |
@samueldr : this is actually the error that I got with the destdir (bundler cannot find himself), but only with versions different from the latest one (see comment given in the description of the PR for details) Does it work with other versions for you? |
Yes, the default.nix when built only fails for 2.6 with the change; calling |
Ok, so it’s the exact reverse of the current state then |
On Sat, May 18, 2019 at 03:53:41PM -0700, Samuel Dionne-Riel wrote:
Hmm, having an issue. Ruby 2.6's native `bundler` apparently doesn't play well with that change.
[Repro](https://gist.github.com/samueldr/e9a12642e240726621a3a006c0bde324)
```
~/tmp/tmp/repro $ nix-build --keep-going -I nixpkgs=/Users/samuel/tmp/nixpkgs/nixpkgs-PR61684
these derivations will be built:
/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv
building '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' on ***@***.***'...
ruby 2.6.3p62 (2019-04-16) [x86_64-linux]
`/homeless-shelter` is not a directory.
Bundler will use `/build/bundler/home/unknown' as your home directory temporarily.
Bundler version 1.17.2
/nix/store/s7x3gaf88bbi7p647jpbrbc29249f65m-ruby-2.6.3/lib/ruby/site_ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundler (Gem::GemNotFoundException)
from /nix/store/s7x3gaf88bbi7p647jpbrbc29249f65m-ruby-2.6.3/lib/ruby/site_ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
from /nix/store/s7x3gaf88bbi7p647jpbrbc29249f65m-ruby-2.6.3/bin/bundler:23:in `<main>'
builder for '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' failed with exit code 1
error: build of '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' on ***@***.***' failed: builder for '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' failed with exit code 1
builder for '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' failed with exit code 1
error: build of '/nix/store/qkn0f4hiqkii9j11xnk4kjrw5mgjc4x9-test-with-ruby_2_6.drv' failed
```
The PATH order here matters. The other way around `bundler` comes from `ruby_2_6`. This means that if an end-user with `ruby_2_6` could have a different bundler than expected. Additionally, I think the bundler shipped with ruby should work, otherwise it seems weird to me.
As far as I can tell, this is only an issue with the `bundler'
executable, which is designed to just be a convenience alias for
`bundle', which is the command referred to in the documentation, etc.
I don't know how much of an issue it is that this doesn't work? I
wouldn't even have thought to test it. But maybe other people have come
to rely on it?
|
This should prevent problems caused by trying to install our own RubyGems over the top of the one that comes with Ruby.
I've added a new commit. Please try again.
It feels very strange that we now include a seperately-versioned
RubyGems with Ruby, but with Bundler, you get the default version, and
can use the bundler package to get the "latest" version.
I'm inclined to say that we should distribute a rubygems package for the
latest version of RubyGems, and distribute stock RubyGems (with our
patches) with Ruby. I haven't thought much about the implications of
this, but it instintively feels right to me... Thoughts?
(cc @zimbatm @manveru)
|
Oh, you're right it only fails with I'm have prepared a couple tests to see how the behaviour compares, it is descriptive of the situation, and not necessarily prescriptive; meaning that you do what you feel is right with the data. Though, I am not confident that the test tests @immae's case. The stuff required to run it: Results: My take on the results:
Additionally, I do not know what should be happening with |
To clarify: there /shouldn't/ be a difference between `bundle' and
`bundler'. But when RubyGems is installed, it (deliberately) only
installs `bundle'. I don't know why.
|
I'm have prepared a couple tests to see how the behaviour compares, it is descriptive of the situation, and not necessarily prescriptive; meaning that you do what you feel is right with the data.
Thanks very much for these. They've been very helpful.
An observation: Ruby 2.3 in 18.09 includes a `bundler' executable. In
19.03 it does not.
|
The current behaviour (as of the tip of this branch), which is that a Ruby elsewhere in the environment does not affect an application just because it happens to be written in Ruby, is desirable in almost all cases. The problem is just that, with Bundler, we actually want the exact opposite behaviour — it should use whatever Ruby is in the environment. In the cases @samueldr has found where this works, it works because of a Bundler executable bundled with Ruby that finds and executes Bundler from Ruby’s LOAD_PATH. In these cases, Ruby and Bundler both have To have this work perfectly, we would need a way to ensure that |
nice detective work yall :) I think I'm the one that installed rubygems directly into the ruby installation. The reason was that rubygems had a security vulnerability and I wanted to make sure that all the ruby users would have the latest version available. Regarding the priority, how about marking |
Nowadays Ruby upstream issues version bumps for Rubygems vulns (e.g. 2.6.2), so I think I'd like to go back to shipping the default version of RubyGems with Ruby, and making a seperate RubyGems package that could be used to get latest. Out of scope of this PR though.
In But the case I'm really thinking of is this:
If you can get one where Ruby beats Bundler because it's hash is sorted after Bundler's, it doesn't make a difference if you do
You still get Ruby's Bundler. Any ideas? |
This makes me think, my initial testing (never shared here) with nix-shell failed because I had a Just saying in case it is related or relevant to the current issue with the bundler executable being overriden. |
What's the status of this PR? |
To summarise how PATH order is determined in temporary environments:
In nix-shell, order matters:
~ $ nix-shell -p ruby_2_6 -p bundler --run 'which bundle'
/nix/store/0cqkbznpnvjw1lwkq05dh0psyykzqw3z-ruby-2.6.3/bin/bundle
~ $ nix-shell -p bundler -p ruby_2_6 --run 'which bundle'
/nix/store/zqbn9k0rk06bllxk1k2wk5xs1q1j6m5z-bundler-1.17.2/bin/bundle
So, users need to always put Bundler first.
In nix run, order doesn't matter and paths are always sorted
reverse-lexicographically:
~ $ nix run -f '<nixpkgs>' bundler ruby_2_6 -c which bundle
/nix/store/zqbn9k0rk06bllxk1k2wk5xs1q1j6m5z-bundler-1.17.2/bin/bundle
~ $ nix run -f '<nixpkgs>' ruby_2_6 bundler -c which bundle
/nix/store/zqbn9k0rk06bllxk1k2wk5xs1q1j6m5z-bundler-1.17.2/bin/bundle
~ $ env -i nix run -f /home/src/nixlib bash hello coreutils ruby_2_6 bundler -c sh -c 'echo $PATH' | tr : '\n'
/nix/store/zqbn9k0rk06bllxk1k2wk5xs1q1j6m5z-bundler-1.17.2/bin
/nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23/bin
/nix/store/k8lhqzpaaymshchz8ky3z4653h4kln9d-coreutils-8.31/bin
/nix/store/d31xwav4imnlpw7zk2ipsc6szbi8dp4k-hello-2.10/bin
/nix/store/0cqkbznpnvjw1lwkq05dh0psyykzqw3z-ruby-2.6.3/bin
It does not appear that there is anything we can do in Nixpkgs
(e.g. setting meta.priority) to alter this order. So, while this isn't
ideal, improving it is out of scope for this PR (and Nixpkgs in
general), and needs to be a change in Nix.
|
Motivation for this change
This works just fine, and means we don't run into an issue with RubyGems trying to install into a different Ruby's prefix when cross-compiling. See #51842 (comment).
Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nix-review --run "nix-review wip"
./result/bin/
)nix path-info -S
before and after)Cc: @immae @samueldr @Mic92