Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

terraform: update all providers and move nixos provider under terraform.withPlugins #47407

Merged
merged 6 commits into from Sep 30, 2018
Merged

terraform: update all providers and move nixos provider under terraform.withPlugins #47407

merged 6 commits into from Sep 30, 2018

Conversation

kalbasit
Copy link
Member

@kalbasit kalbasit commented Sep 26, 2018 •
edited

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

/cc @jgeerds @zimbatm @peterhoeg

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (14.41 MiB download, 102.57 MiB unpacked):
  /nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin
copying path '/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin' from 'https://cache.nixos.org'...
/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: terraform

Partial log (click to expand)

github.com/hashicorp/terraform/vendor/github.com/hashicorp/go-plugin
github.com/hashicorp/terraform/vendor/github.com/hashicorp/go-getter
github.com/hashicorp/terraform/moduledeps
github.com/hashicorp/terraform/config
github.com/hashicorp/terraform/config/module
github.com/hashicorp/terraform/terraform
github.com/hashicorp/terraform/plugin
github.com/hashicorp/terraform/tools/terraform-bundle
builder for '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed with exit code 51
error: build of '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed

@flokli
Copy link
Contributor

flokli commented Sep 26, 2018

@kalbasit moving "official" and "standalone" providers into the same namespace makes a lot of sense, thanks :-)

Could you also move the two remaining providers, terraform-provider-ibm and terraform-provider-libvirt in there?

The former one should be completely straightforward, for the latter one we would need to add a way to overrideAttrs, setting (propagated)buildInputs - either from update-all, or in providers/default.nix.

@kalbasit
Copy link
Member Author

@flokli I tried moving both providers but I hit two issues that I thought were out of scope for this PR:

I'm hoping to first merge this PR, and work on the support for custom attributes in a separate PR.

@flokli
Copy link
Contributor

flokli commented Sep 26, 2018

@kalbasit separate PR is fine, too - just wanted to make sure these don't slip through :-)

@kalbasit
Copy link
Member Author

@flokli It won't! I'll figure out the best way and submit a new PR sometime this week.

@kalbasit kalbasit force-pushed the nixpkgs_add-terraform-provider-nixos branch from 0ad8175 to 7dadabb Compare September 27, 2018 16:52
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (14.63 MiB download, 105.12 MiB unpacked):
  /nix/store/0vbhc7kk6r10va4p18zy0lg3wv3046sf-tzdata-2018e
  /nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin
  /nix/store/kmqiiwrvb04hqnxhpjxl2jkn3md19pp2-iana-etc-20180711
copying path '/nix/store/kmqiiwrvb04hqnxhpjxl2jkn3md19pp2-iana-etc-20180711' from 'https://cache.nixos.org'...
copying path '/nix/store/0vbhc7kk6r10va4p18zy0lg3wv3046sf-tzdata-2018e' from 'https://cache.nixos.org'...
copying path '/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin' from 'https://cache.nixos.org'...
/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: terraform

Partial log (click to expand)

github.com/hashicorp/terraform/vendor/github.com/hashicorp/hcl2/hcl/hclsyntax
github.com/hashicorp/terraform/vendor/google.golang.org/grpc/transport
github.com/hashicorp/terraform/vendor/google.golang.org/grpc
github.com/hashicorp/terraform/vendor/github.com/hashicorp/hcl2/hcl/json
github.com/hashicorp/terraform/vendor/github.com/hashicorp/hcl2/hclparse
github.com/hashicorp/terraform/vendor/google.golang.org/grpc/health/grpc_health_v1
github.com/hashicorp/terraform/vendor/google.golang.org/grpc/health
github.com/hashicorp/terraform/vendor/github.com/hashicorp/go-plugin
builder for '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed with exit code 50
error: build of '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed

@zimbatm
Copy link
Member

zimbatm commented Sep 28, 2018

@kalbasit just pushed a change to your branch to move the providers to their own terraform-providers group. I wanted to add an alias from terraform-provider-nixos and the target wasn't reachable.

For the remaining 2 plugins, since they have special requirements they could just be moved under that group with callPackage links so the generic builder doesn't have to be extended.

@kalbasit
Copy link
Member Author

@zimbatm thx for the contribution but the eval is failing, can you take another look at it?

@zimbatm @kalbasit
terraform: move providers to terraform-providers
ef64786 
Before, providers were only built indirectly. Since proviers don't
depend on terraform to build they can be moved into their own collection
of packages. This also has the advantage that they can be reached
directly using an attribute path (Eg: terraform-providers.nixos).

Co-authored-by: Wael Nasreddine <wael.nasreddine@gmail.com>
@kalbasit kalbasit force-pushed the nixpkgs_add-terraform-provider-nixos branch from acd054c to fffed44 Compare September 28, 2018 18:29
@kalbasit
Copy link
Member Author

@zimbatm never mind, I managed to fix it. The issue was that recurseIntoAttrs set up the packages as well as the functions override, overrideDerivation and the recurseForDerivations boolean. Removing them fixed the issue.

@kalbasit kalbasit force-pushed the nixpkgs_add-terraform-provider-nixos branch from fffed44 to 6ff45f3 Compare September 28, 2018 18:32
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (14.41 MiB download, 102.57 MiB unpacked):
  /nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin
copying path '/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin' from 'https://cache.nixos.org'...
/nix/store/6jb53clmhjm43ih5zy5hr433bq64rahb-terraform-0.11.8-bin

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: terraform

Partial log (click to expand)

github.com/hashicorp/terraform/vendor/github.com/aws/aws-sdk-go/aws/session
github.com/hashicorp/terraform/vendor/github.com/hashicorp/go-getter
github.com/hashicorp/terraform/config
github.com/hashicorp/terraform/config/module
github.com/hashicorp/terraform/tools/terraform-bundle
github.com/hashicorp/terraform/tools/terraform-bundle/e2etest
github.com/hashicorp/terraform/vendor/github.com/hashicorp/go-version
github.com/hashicorp/terraform/version
builder for '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed with exit code 53
error: build of '/nix/store/x1vxybzxkd4wpwvww3s5lv27sw83pb5q-terraform-0.11.8.drv' failed

@Mic92 Mic92 merged commit 5547a8b into NixOS:master Sep 30, 2018
@Mic92 Mic92 mentioned this pull request Sep 30, 2018
Closed
9 tasks
@kalbasit kalbasit deleted the nixpkgs_add-terraform-provider-nixos branch October 1, 2018 01:11
@zimbatm
Copy link
Member

zimbatm commented Oct 1, 2018

nice work @kalbasit

zimbatm pushed a commit that referenced this pull request Oct 4, 2018
@Mic92 @zimbatm
Merge pull request #47407 from kalbasit/nixpkgs_add-terraform-provide…
be38959 
…r-nixos

terraform: update all providers and move nixos provider under terraform.withPlugins
(cherry picked from commit 5547a8b)
@zimbatm
Copy link
Member

zimbatm commented Oct 4, 2018

ported to 18.09 with git cherry-pick -x -m 1 5547a8b

@kalbasit kalbasit mentioned this pull request Nov 20, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mic92 Mic92 Mic92 left review comments

Assignees
No one assigned
Labels
6.topic: nixos 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@kalbasit @GrahamcOfBorg @flokli @zimbatm @Mic92