Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terraform: provide plugins for 0.10 #28729

Merged
merged 1 commit into from
Aug 30, 2017

Conversation

copumpkin
Copy link
Member

@copumpkin copumpkin commented Aug 30, 2017

Obviously very much a WIP, but this is the sort of thing I had in mind. I don't really care how we plug the plugins into it, but this way seemed to work and didn't take too much hackery.

To use, try:

terraform_0_10.withPlugins [ terraform_0_10.providers.aws ]

That should get you a version of terraform that:

  1. Uses the aws provider compiled from source by Nix
  2. Doesn't automatically download precompiled providers from their plugin repository

Feedback welcome! I've been considering making withPlugins take a function from the available plugins for that version of terraform, so you could write something like withPlugins (p: [ p.aws ]) or something.

Motivation for this change

#28537

Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@mention-bot
Copy link

@copumpkin, thanks for your PR! By analyzing the history of the files in this pull request, we identified @zimbatm, @peterhoeg and @kamilchm to be potential reviewers.

Copy link
Member

@zimbatm zimbatm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.

Even if it's more verbose, the simple approach is probably better because it allows to plug things in different ways. For example we might have a terraform2nix that generates the plugin set and then it will be terraform.withPlugins (import ./terraform2nix.nix { pkgs }).

We don't know how the terraform2nix tool will look like yet. It might be necessary to scan all the modules if third-parties start including provider version ranges. For now I'm fine with a script that generates all the terraform plugins from the latest git tags.

};

# TODO: make me stackable
withPlugins = plugins: stdenv.mkDerivation {
Copy link
Member

@zimbatm zimbatm Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could also take terraform: as the first argument. Then set the passthru in the generic builder where terraform is a reference to self.

@shlevy
Copy link
Member

shlevy commented Aug 30, 2017

I like the haskell set convention of the withPlugins taking the set itself, to cover @zimbatm's case we should just include p.callPackage or something like that that can be used to import a plugin expression with the current terraform as the base.

@copumpkin
Copy link
Member Author

Any thoughts on withPlugins vs. withProviders? The only plugins I've heard of are providers but I don't know if it's planned to be exclusively providers.

Also @shlevy, are you saying you'd prefer withPlugins to take a function like I had in my example?

@shlevy
Copy link
Member

shlevy commented Aug 30, 2017

No strong feelings on the name, yeah I think it should take a function

@zimbatm
Copy link
Member

zimbatm commented Aug 30, 2017

No strong objections either. If Haskell takes a function we could do the same for consistency's sake.

@copumpkin
Copy link
Member Author

A function it is!

@copumpkin
Copy link
Member Author

@shlevy @zimbatm any thoughts on how to handle the override* cleanly here? I want this stuff to stack naturally, but the most obvious way to do that would also cause TF to recompile whenever changing plugins and that seems shitty. I can add special handling for overrideAttrs and override to shuffle around the composition order a bit (so that withPlugins can still be overridden as usual and it composes nicely) but it feels messy.

Otherwise I think this is ready.

@copumpkin copumpkin changed the title [WIP] terraform: provide plugins for 0.10 terraform: provide plugins for 0.10 Aug 30, 2017
@fpletz fpletz added this to the 17.09 milestone Aug 30, 2017
@copumpkin
Copy link
Member Author

Also, y'all feel welcome to add other plugins!

Also add a few starter plugins/providers
@copumpkin
Copy link
Member Author

Okay, I now have AWS, Google, Azure, and K8s plugins, and I've got a not-beautiful-but-not-awful solution to the override question. You can now write this monstrosity and have it work:

$ nix-build -E "with import ./. {}; (((terraform_0_10.withPlugins (p: [ p.aws ])).withPlugins (p: [ p.azurerm p.google ])).overrideAttrs (orig: { name = orig.name + \"foo\"; })).withPlugins (p: [ p.kubernetes ])"

I'm going to merge this but I welcome improvements, new plugins, or suggestions on how to do the override crap more nicely.

@copumpkin copumpkin merged commit b291ead into NixOS:master Aug 30, 2017
@copumpkin
Copy link
Member Author

Actually, the way I homogenized the withPlugins stuff means that it won't ever auto-download anymore. I'll fix that.

@copumpkin
Copy link
Member Author

Fixed in 11753b3

@zimbatm
Copy link
Member

zimbatm commented Aug 31, 2017

LGTM & composable! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants