terraform-providers: Terraform 0.13 compatibility #99198
Conversation
timstott
force-pushed
the
terraform-013-compatible-providers
branch
from
edbd38e
to
0860356
Compare
| # Place provider in Terraform 0.13 search location | ||
| TF_13_PROVIDER_PATH=go/bin/${data.provider-source-address}/${version}/$(go env GOARCH)/ | ||
| mkdir -p $TF_13_PROVIDER_PATH | ||
| ln -s go/bin/${repo}_v${version} $TF_13_PROVIDER_PATH |
There was a problem hiding this comment.
I went with the symlink approach to avoid making two copies of the binary, but it doesn't work.
I also question whether it's toDrv responsibility to make the binary available in the correct location as opposed to only building. I'm open to recommendations 🙃.
There was a problem hiding this comment.
A wrapper script that execs the providers should be a good alternative. It's the approach we are taking in https://github.com/numtide/generate-terraform-provider-shim to download providers on demand and it works fine there.
| "acme": { | ||
| "owner": "getstackhead", | ||
| "provider-source-address": "registry.terraform.io/getstackhead/acme", | ||
| "repo": "terraform-provider-acme", | ||
| "rev": "v1.5.0-patched", | ||
| "sha256": "1wdrjpd3l0xadsa3lqhsc9c57g8x2qkwb76q824sk8za1a7lapii", | ||
| "version": "1.5.0-patched" | ||
| }, |
There was a problem hiding this comment.
I have been using the acme provider to test. The remaining "provider-source-address" will be added within this PR.
There was a problem hiding this comment.
Thanks, that's great!
The question is, how complete do you want the solution to be and this depends on the energy that you have available. The current PR is already a quite good solution. It could be made better by integrating with the updateScript framework, and by allowing to get new providers from the registry.
| set -euo pipefail | ||
|
|
||
| # TODO: document usage | ||
| provider_name="$1" |
There was a problem hiding this comment.
Do you see an easy way to list and/or update all of the providers from the registry?
There was a problem hiding this comment.
I'm inclined to provide two additional scripts, one to add a provider to providers.json from it's registry URL (as opposed to exhaustively add all providers). And one script to update all known providers.
# update-all-providers
jq 'keys[]' providers.json | xargs ./update-providerI should be able to also re-use much of update-provider to create add-provider registry.terraform.io/some/provider
Any thoughts?
I will absolutely look at using update-provider with the updateScript framework ❤️
There was a problem hiding this comment.
Everything you said sounds good to me!
timstott
force-pushed
the
terraform-013-compatible-providers
branch
2 times, most recently
from
cc4a818
to
ba4d7b2
Compare
timstott
force-pushed
the
terraform-013-compatible-providers
branch
3 times, most recently
from
ecdd9ac
to
211f7bb
Compare
|
Result of 2 packages built:
|
Added provider source address to providers hosted in the same GitHub repository. Other providers have moved repository and would require a patch to the package path.
timstott
force-pushed
the
terraform-013-compatible-providers
branch
from
9429fe9
to
172e74d
Compare
| toDrv = name: data: | ||
| let | ||
| fallbackProviderSourceAddress = "nixpkgs/${data.owner}/${name}"; | ||
| providerSourceAddress = lib.attrByPath ["provider-source-address"] fallbackProviderSourceAddress data; |
There was a problem hiding this comment.
this is a little bit cleaner, the or operator is triggered if the key doesn't exist.
| providerSourceAddress = lib.attrByPath ["provider-source-address"] fallbackProviderSourceAddress data; | |
| providerSourceAddress = data.provider-source-address or fallbackProviderSourceAddress; |
There was a problem hiding this comment.
Much nicer ❤️. I've incorporated the change.
Terraform 0.13 adopts a new filesystem layout for plugins (illustrated below). Terraform 0.12 and earlier `plugins-dir/terraform-provider-aws_v3.7.0` Terraform 0.13 `plugins-dir/registry.terraform.io/hashicorp/aws/3.7.0/linux_amd64/terraform-provider-aws_v3.7.0` To support all packaged Terraform versions a shim is created at both locations. This approach was inspired by https://github.com/numtide/generate-terraform-provider-shim Terraform 0.13 provider documentation https://www.terraform.io/upgrade-guides/0-13.html#new-filesystem-layout-for-local-copies-of-providers layout terraform providers filesystem in withPlugins
timstott
force-pushed
the
terraform-013-compatible-providers
branch
from
172e74d
to
b0a0ffb
Compare
|
thanks! |
|
specifically cd1b594 broke it |
|
Sorry for the inconvenience. I will have a fix ready in 3h. EDIT: #100072 |
|
Please look into it again. let
pkgs = import (builtins.fetchTarball "https://api.github.com/repos/nixos/nixpkgs/tarball/master") { };
in
pkgs.mkShell {
buildInputs = [
(pkgs.terraform_0_13.withPlugins (p: [
p.null
p.helm
p.vultr
p.digitalocean
p.kubernetes
]))
];
}nix-shell --run "terraform init"produces: looking at plugins/registry.terraform.io, we find 2 dirs - hashicorp and nixpkgs. ls /nix/store/6bgi4zxw3xvpni361hs8l7xf9d33cy3d-terraform-0.13.5/plugins/registry.terraform.io/
# dr-xr-xr-x 3 root 3 Jan 1 1970 helm/
# dr-xr-xr-x 3 root 3 Jan 1 1970 null/Not found plugins ended up in plugins/registry.terraform.io/nixpkgs: ls /nix/store/6bgi4zxw3xvpni361hs8l7xf9d33cy3d-terraform-0.13.5/plugins/registry.terraform.io/nixpkgs
# dr-xr-xr-x 3 root 3 Jan 1 1970 digitalocean/
# dr-xr-xr-x 3 root 3 Jan 1 1970 kubernetes/
# dr-xr-xr-x 3 root 3 Jan 1 1970 vultr/ |
|
Currently you need something like... for providers without the source address set, or just: |
|
attribute 'required_providers' missing, can you please supply a full example? |
|
@voobscout the snippet below, to execute in the This is highlighting an improvement I have been meaning to make, to include the provider declaration snippet in the package description (much like on the Terraform registry - screenshot below).
|
|
That makes a lot of sense now, thank you @timstott! |
|
guys, maybe you can point me what i'm missing here: - Using previously-installed nixpkgs/kubernetes v1.11.3
- Using previously-installed hashicorp/local v2.0.0
- Finding hashicorp/kubernetes versions matching "~> 1.10, >= 1.11.1"...
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider
hashicorp/kubernetes: provider registry.terraform.io/hashicorp/kubernetes was
not found in any of the search locations
- /nix/store/cfldgk8hx5zfc3nzij52h149q6l1bnns-terraform-0.13.5/plugins
[terragrunt] 2020/11/22 18:25:40 Hit multiple errors:
exit status 1
❯ cat versions.tf
terraform {
required_providers {
helm = {
source = "nixpkgs/kubernetes"
}
}
}sorry for posting in merged pull request. |
I'm pretty sure you want that to be |
|
sorry to confuse, I had correct versions.tf, was just copy pasting from your comment before posting my comment: - Finding hashicorp/kubernetes versions matching ">= 1.11.1"...
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider
hashicorp/kubernetes: provider registry.terraform.io/hashicorp/kubernetes was
not found in any of the search locations
- /nix/store/cfldgk8hx5zfc3nzij52h149q6l1bnns-terraform-0.13.5/plugins
[terragrunt] 2020/11/22 19:58:44 Hit multiple errors:
exit status 1
❯ cat versions.tf
terraform {
required_providers {
kubernetes = {
source = "nixpkgs/kubernetes"
}
}
}
❯ cat .terraform/plugins/registry.terraform.io/nixpkgs/kubernetes/1.11.3/linux_amd64/terraform-provider-kubernetes_v1.11.3
#!/nix/store/y0sw8sxkr7kwlsndwixyf4s4snlfhjh1-bash-4.4-p23/bin/bash
exec /nix/store/cx3k25sc0bvsw0im70gwl8m6yf8g9vng-terraform-provider-kubernetes-1.11.3/bin/terraform-provider-kubernetes_v1.11.3 "$@"provider appears to be there, though i'm not sure yet why it does fail to initialize. |
|
so i did edit also eks module: ❯ cat .terraform/modules/eks/versions.tf
terraform {
required_providers {
kubernetes = {
source = "nixpkgs/kubernetes"
}
}
}and did this trick: terragrunt state replace-provider registry.terraform.io/hashicorp/kubernetes nixpkgs/kuberneteswhich is incredibly ugly. now it initiates fine: Initializing modules...
Initializing the backend...
Initializing provider plugins...
- Using previously-installed hashicorp/external v2.0.0
- terraform.io/builtin/terraform is built in to Terraform
- Using previously-installed hashicorp/template v2.2.0
- Using previously-installed hashicorp/local v2.0.0
- Using previously-installed hashicorp/random v3.0.0
- Using previously-installed nixpkgs/kubernetes v1.11.3
- Using previously-installed hashicorp/null v3.0.0
- Using previously-installed hashicorp/aws v2.65.0
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.
* hashicorp/external: version = "~> 2.0.0"
* hashicorp/local: version = "~> 2.0.0"
* hashicorp/null: version = "~> 3.0.0"
* hashicorp/random: version = "~> 3.0.0"
* hashicorp/template: version = "~> 2.2.0"
Terraform has been successfully initialized! |
|
I had a pretty hard time with this, and want to call out the replace-provider of the previous comment. I had this in my I'd then run and then I'd go to do an apply, and it fails: My shell.nix looked like this: The issue is actually a problem with the 0.12 to 0.13 migration, NOT a problem with this PR. Our providers were considered "in-house" before, and have less information than terraform wants for the automatic upgrade. This means we need to follow this: https://www.terraform.io/upgrade-guides/0-13.html#in-house-providers I needed to run: and that fixed the issue with: |
|
Note if you see an error like: it might be because you're on 20.09 and 20.09 doesn't have this PR applied. |
|
sorry to bring it here, if you have something to say from your experience please share your thoughts here: https://discourse.nixos.org/t/public-cloud-infrastructure-people-whats-you-workflow-with-terraform-terragrunt-providers/11669/4 for a couple of months trying to use nixos provided terraform and providers i faced multiple issues contributing to many people infra projects, like:
also tf 0.14 introduced a lock file to keep providers versions/hashes consistent for all contributors in repo, in case of nixos providers these hashes would be different, meaning we can't use this lock file in the same repo, me NixOS user and other engineers. again, i might using it all wrong, please share your workflow if my attitude towards nixos tf and providers are rather wrong. and yeah, just wondering what is the use case to manage/build providers in nixos, rather than using hashicorp ones? (that might be due to nixos source model, if that's the case I could not agree more). but for me, it brings much more inconvenience (maybe due to my workflow). |
|
UPDATE: Fixed the issue. I was missing a I'm seeing this in nixpkgs v21.05. In my case, I'm building my TF modules from source, and they are being dropped into the The suggested fix to run Update: If I switch my configuration to to use |
|
@anandsuresh Have you been able to figure it out? I've bumped into just now and defining |
Motivation for this change
Terraform 0.13 introduces a new filesystem layout to store providers which is incompatible with Terraform 0.12. The new filesystem layout is informed by the Terraform Registry, a repository of providers and modules.
The objectives of this work are:
Modified
provider-source-addressattribute to providers to enable the creation of the filesystem layout.providers.jsoninstead ofdata.nixto usejqonly instead ofnix+jqcombinationThings done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)nix path-info -Sbefore and after)Fixed #98652