Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundlerEnv: improve handling of groups #51884

Merged
merged 3 commits into from Dec 12, 2018

Conversation

alyssais
Copy link
Member

@alyssais alyssais commented Dec 11, 2018

Motivation for this change

The most recent version of Bundix introduces group attribute generation. This has revealed some deficiencies with how groups are handled by bundlerEnv. These would be addressed by making the following changes:

  • Suppose I have a Gemfile like this:

    source "https://rubygems.org"
    gem "actioncable"
    gem "websocket-driver", group: :test

    The gemset.nix generated by Bundix 2.4.1 will set ActionCable's groups to [ "default" ], and websocket-driver's to [ "test" ]. This means that the generated bundlerEnv wouldn't include websocket-driver unless the test group was included, even though it's required by the default group.

    This is arguably a bug in Bundix ("default" isn't included in groups on transitive dependencies listed explicitly in other groups nix-community/bundix#39) (websocket-driver's groups should probably be [ "default" "test" ] or just [ "default" ]), but there's no reason bundlerEnv should omit dependencies even given such an input -- it won't necessarily come from Bundix, and it would be good for bundlerEnv to do the right thing.

    To fix this, filterGemset is now a recursive function, that adds dependencies of gems in the group to the filtered gemset until it stabilises on the gems that match the required groups, and all of their recursive dependencies.

  • All groups are now included by default. This wasn't really an issue until the latest minor release of Bundix (2.4), because prior to then Bundix didn't emit group attributes, and so this functionality of bundlerEnv wasn't really used. However, it is now apparent that a better default for bundlerEnv would be to include all gem groups by default, not just the default group. This matches the behavior of Bundler, and makes more sense, because the default group alone isn't necessarily useful for anything -- consider a Rails app with production, development, and test groups. It has the additional benefit of being backwards compatible with how this would have worked before the Bundix update.

  • Default gems are always included. default isn't really a group, it's more the absence of one. With Bundler, this means that a gem should be installed unconditionally, regardless of which groups are specified. It doesn't really make sense to allow these gems to be omitted from a bundlerEnv.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Suppose I have a Gemfile like this:

    source "https://rubygems.org"
    gem "actioncable"
    gem "websocket-driver", group: :test

The gemset.nix generated by Bundix 2.4.1 will set ActionCable's groups
to [ "default" ], and websocket-driver's to [ "test" ]. This means that
the generated bundlerEnv wouldn't include websocket-driver unless the
test group was included, even though it's required by the default group.

This is arguably a bug in Bundix (websocket-driver's groups should
probably be [ "default" "test" ] or just [ "default" ]), but there's no
reason bundlerEnv should omit dependencies even given such an input --
it won't necessarily come from Bundix, and it would be good for
bundlerEnv to do the right thing.

To fix this, filterGemset is now a recursive function, that adds
dependencies of gems in the group to the filtered gemset until it
stabilises on the gems that match the required groups, and all of their
recursive dependencies.
This wasn't really an issue until the latest minor release of Bundix
(2.4), because prior to then Bundix didn't emit group attributes, and so
this functionality of bundlerEnv wasn't really used. However, it is now
apparent that a better default for bundlerEnv would be to include all
gem groups by default, not just the default group. This matches the
behavior of Bundler, and makes more sense, because the default group
alone isn't necessarily useful for anything -- consider a Rails app with
production, development, and test groups. It has the additional benefit
of being backwards compatible with how this would have worked before the
Bundix update.
"default" isn't really a group, it's more the absence of one. With
Bundler, this means that a gem should be installed unconditionally,
regardless of which groups are specified. It doesn't really make sense
to allow these gems to be omitted from a bundlerEnv.
Copy link
Member

@zimbatm zimbatm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really nice, thanks!

@grahamc
Copy link
Member

grahamc commented Dec 12, 2018

lgtm on a light review

@alyssais alyssais merged commit b450083 into NixOS:master Dec 12, 2018
@alyssais alyssais deleted the bundlerEnv-groups branch December 12, 2018 23:52
# nix-repl> converge (x: x / 2) 16
# 0
converge = f: x:
if (f x) == x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alyssais
This could be optimized by introducing a let binding, i.e.

converge = f: x:
  let f_of_x = f x;
  in if f_of_x == x
    then x
    else converge f f_of_x;

That way f x doesn't have to be evaluated twice.

@alyssais alyssais mentioned this pull request Apr 17, 2019
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants