terraform: Wrap PATH of propagatedBuildInputs #37861

adisbladis · 2018-03-26T12:24:04Z

Motivation for this change

Plugins (namely terraform-provider-libvirt) may have a run time
dependency on external binaries.

See #37852

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

cc @Mic92 @zimbatm @peterhoeg @jgeerds

Plugins (namely terraform-provider-libvirt) may have a run time dependency on external binaries.

GrahamcOfBorg · 2018-03-26T12:35:17Z

Success on x86_64-linux (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (11.78 MiB download, 83.02 MiB unpacked):
  /nix/store/svg985ny0x19lhvz8r448vsbrqd16hcw-terraform-0.11.4-bin
copying path '/nix/store/svg985ny0x19lhvz8r448vsbrqd16hcw-terraform-0.11.4-bin' from 'https://cache.nixos.org'...
/nix/store/svg985ny0x19lhvz8r448vsbrqd16hcw-terraform-0.11.4-bin

GrahamcOfBorg · 2018-03-26T12:35:21Z

Success on aarch64-linux (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (10.81 MiB download, 78.79 MiB unpacked):
  /nix/store/68vzgjqk26vhy271wr6j9hw3gw1bqqz6-terraform-0.11.4-bin
  /nix/store/vqlb5ygj34c40lr7hcsg3kgnq6pyki8p-iana-etc-20180108
copying path '/nix/store/vqlb5ygj34c40lr7hcsg3kgnq6pyki8p-iana-etc-20180108' from 'https://cache.nixos.org'...
copying path '/nix/store/68vzgjqk26vhy271wr6j9hw3gw1bqqz6-terraform-0.11.4-bin' from 'https://cache.nixos.org'...
/nix/store/68vzgjqk26vhy271wr6j9hw3gw1bqqz6-terraform-0.11.4-bin

GrahamcOfBorg · 2018-03-26T12:35:23Z

Success on x86_64-darwin (full log)

Attempted: terraform

Partial log (click to expand)

these paths will be fetched (15.29 MiB download, 120.41 MiB unpacked):
  /nix/store/gkslmhs5s0mx137xxn2f69157vfnzwqs-terraform-0.11.4-bin
copying path '/nix/store/gkslmhs5s0mx137xxn2f69157vfnzwqs-terraform-0.11.4-bin' from 'https://cache.nixos.org'...
/nix/store/gkslmhs5s0mx137xxn2f69157vfnzwqs-terraform-0.11.4-bin

zimbatm

Looks good to me.

I was thinking of a more general solution where the plugins' nix-support/setup-hook would be loaded before starting terraform but that works as well. Plus I'm not sure that setup-hook is intended for that use-case.

Mic92 · 2018-03-26T15:36:09Z

pkgs/applications/networking/cluster/terraform/default.nix

@@ -64,7 +70,8 @@ let
              buildCommand = ''
                mkdir -p $out/bin/
                makeWrapper "${terraform.bin}/bin/terraform" "$out/bin/terraform" \
-                  --set NIX_TERRAFORM_PLUGIN_DIR "${buildEnv { name = "tf-plugin-env"; paths = actualPlugins; }}/bin"
+                  --set NIX_TERRAFORM_PLUGIN_DIR "${buildEnv { name = "tf-plugin-env"; paths = actualPlugins; }}/bin" \
+                  --prefix PATH : "${lib.makeBinPath wrapperInputs}"


I have not tested this, but this looks like a nice alternative to my hack.

Mic92 · 2018-03-26T15:40:18Z

I would not abuse setup-hook for this case. This might set other build-time-only stuff we do not want to have at runtime like PYTHONHASHSEED which can be dangerous in interactive usage.

GrahamcOfBorg · 2018-03-28T00:55:36Z

Failure on x86_64-darwin (full log)

Attempted: terraform

Partial log (click to expand)

cannot build derivation '/nix/store/zy8js09gma1q7lvvnz6lv2j6cwbrfqh4-python2.7-pytest-xdist-1.22.2.drv': 10 dependencies couldn't be built
cannot build derivation '/nix/store/bjahk71667zfrjjxkpd2j48r24rj2hhy-mercurial-4.5.drv': 11 dependencies couldn't be built
cannot build derivation '/nix/store/a5six6mdihiq9n4cz6sqpsrdjdm26ic4-python2.7-hypothesis-3.45.2.drv': 13 dependencies couldn't be built
cannot build derivation '/nix/store/rkfk7d32p3h87ydv6nnfvhls72xh6gdq-python2.7-cryptography-2.1.4.drv': 21 dependencies couldn't be built
cannot build derivation '/nix/store/jlvvgi10w24g3pb1alvdciq7kdfipj53-python2.7-paramiko-2.1.1.drv': 8 dependencies couldn't be built
cannot build derivation '/nix/store/j90j2mvc1xj3cjc6jx37sd44pda7c3j4-bazaar-2.7.0.drv': 7 dependencies couldn't be built
cannot build derivation '/nix/store/jzrc0m2zh1z65piphs13mvhf8lydx1dd-go-1.9.4.drv': 17 dependencies couldn't be built
cannot build derivation '/nix/store/n6czxv2vcyrfh9w0kqb88f78gcr9fal3-govers-20150109-3b5f175.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/19qpl68wq2pfq4l35cjrra40r237hcwn-terraform-0.11.4.drv': 5 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/19qpl68wq2pfq4l35cjrra40r237hcwn-terraform-0.11.4.drv' failed

GrahamcOfBorg · 2018-03-28T01:32:03Z

Success on x86_64-linux (full log)

Attempted: terraform

Partial log (click to expand)

/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/ChrisTrenkamp/goxpath/coverage.sh: interpreter directive changed from "/bin/bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/posener/complete/test.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/ulikunitz/xz/make-docs: interpreter directive changed from "/bin/sh" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/sh"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/fsouza/go-dockerclient/travis-scripts/run-tests.bash: interpreter directive changed from "/bin/bash -ex" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash -ex"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/fsouza/go-dockerclient/travis-scripts/install.bash: interpreter directive changed from "/bin/bash -x" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash -x"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/fsouza/go-dockerclient/external/golang.org/x/sys/unix/mkerrors.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/fsouza/go-dockerclient/external/golang.org/x/sys/unix/mkall.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4/share/go/src/github.com/hashicorp/terraform/vendor/github.com/golang/protobuf/ptypes/regen.sh: interpreter directive changed from "/bin/bash -e" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash -e"
checking for references to /build in /nix/store/93a41i8m32zgyii73qyqfpfa2g12np95-terraform-0.11.4...
/nix/store/j37dvmslmkd7gdad32gk97010sgfmq9c-terraform-0.11.4-bin

GrahamcOfBorg · 2018-03-28T02:13:37Z

Failure on aarch64-linux (full log)

Attempted: terraform

Partial log (click to expand)

tests/cover/test_shrinking_limits.py .                                   [ 71%]
tests/cover/test_simple_characters.py .............                      [ 72%]
building of '/nix/store/cmdvps6qj283gnmkc71wrbay5bvb47s7-python2.7-hypothesis-3.45.2.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/pr4yqchn2cd2kgmlp90acrgk3zbs9l80-python2.7-cryptography-2.1.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/giz8r2sap24pb49399w5s83as9pydk7m-python2.7-paramiko-2.1.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/083yqs4xivsddkmb9yyzvdasf2b04k84-bazaar-2.7.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/jk90n67apangx4bcrdan4hks3g253n7p-go-1.9.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/58444z5b3pm4758qhclrcqfw1lqrrsjs-govers-20150109-3b5f175.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vxyq18lhqjf30mf2wsd1h7fxyq6brnkz-terraform-0.11.4.drv': 2 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/vxyq18lhqjf30mf2wsd1h7fxyq6brnkz-terraform-0.11.4.drv' failed

terraform: Wrap PATH of propagatedBuildInputs

cb27101

Plugins (namely terraform-provider-libvirt) may have a run time dependency on external binaries.

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Mar 26, 2018

zimbatm approved these changes Mar 26, 2018

View reviewed changes

Mic92 reviewed Mar 26, 2018

View reviewed changes

Mic92 added 3 commits March 28, 2018 00:46

terraform: wrap path of actual plugins instead

670807f

terraform: filter was actually required

e643295

terraform: filter was in the wrong order

b1756e5

Mic92 merged commit deecb4c into NixOS:master Mar 28, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

terraform: Wrap PATH of propagatedBuildInputs #37861

terraform: Wrap PATH of propagatedBuildInputs #37861

adisbladis commented Mar 26, 2018 •

edited

GrahamcOfBorg commented Mar 26, 2018

GrahamcOfBorg commented Mar 26, 2018

GrahamcOfBorg commented Mar 26, 2018

zimbatm left a comment

Mic92 Mar 26, 2018

Mic92 commented Mar 26, 2018 •

edited

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

terraform: Wrap PATH of propagatedBuildInputs #37861

terraform: Wrap PATH of propagatedBuildInputs #37861

Conversation

adisbladis commented Mar 26, 2018 • edited

Motivation for this change

Things done

GrahamcOfBorg commented Mar 26, 2018

GrahamcOfBorg commented Mar 26, 2018

GrahamcOfBorg commented Mar 26, 2018

zimbatm left a comment

Choose a reason for hiding this comment

Mic92 Mar 26, 2018

Choose a reason for hiding this comment

Mic92 commented Mar 26, 2018 • edited

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

adisbladis commented Mar 26, 2018 •

edited

Mic92 commented Mar 26, 2018 •

edited