Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DigitalOcean backend: fix authToken, add IPv6 support. #629

Closed
wants to merge 2 commits into from
Closed

DigitalOcean backend: fix authToken, add IPv6 support. #629

wants to merge 2 commits into from

Conversation

dhess
Copy link
Contributor

@dhess dhess commented Mar 20, 2017

Currently, the DigitalOcean backend ignores the
digitalOcean.authToken config variable completely.

Additionally, the DigitalOcean backend requires that the
DIGITAL_OCEAN_AUTH_TOKEN environment variable is always set for any
NixOps operation on a DigitalOcean target (create, deploy, etc.),
because it does not persist the auth token value in the NixOps state
database.

This change modifies the DigitalOcean backend so that it treats auth
tokens like the EC2 backend treats AWS access keys; namely:

  • Use the digitalOcean.authToken value, if specified in the
    deployment configuration. If both this value and the
    DIGITAL_OCEAN_AUTH_TOKEN environment variable are set, the backend
    prefers the digitalOcean.authToken value; this behavior is
    consistent with the EC2 backend and its preference of the
    ec2.accessKeyId value over the AWS_ACCESS_KEY_ID environment
    variable.

  • When a DigitalOcean droplet is created, if neither auth token value
    is set, the backend aises an exception, rather than attempting to
    create the droplet anyway and failing with an error from the
    DigitalOcean API.

  • Upon successful creation of a droplet, persist the DigitalOcean auth
    token used to create it in the NixOps state database. This happens
    regardless of whether the token was provided via the
    digitalOcean.authToken deployment configuration, or the
    DIGITAL_OCEAN_AUTH_TOKEN environment variable. Based on a review
    of the code in the EC2 backend, I believe this behavior is
    consistent with the EC2 backend, though I have not used the EC2
    backend myself to verify this.

  • If the user does not set digitalOcean.authToken and uses the
    environment variable instead, the environment variable is only ever
    used to create the droplet; from that point on, NixOps will use the
    token value that was persisted in the NixOps state database. Again,
    based on a review of the code in the EC2 backend, I believe this
    behavior is consistent with the EC2 backend.

Currently, the DigitalOcean backend ignores the
`digitalOcean.authToken` config variable completely.

Additionally, the DigitalOcean backend requires that the
`DIGITAL_OCEAN_AUTH_TOKEN` environment variable is always set for any
NixOps operation on a DigitalOcean target (`create`, `deploy`, etc.),
because it does not persist the auth token value in the NixOps state
database.

This change modifies the DigitalOcean backend so that it treats auth
tokens like the EC2 backend treats AWS access keys; namely:

- Use the `digitalOcean.authToken` value, if specified in the
  deployment configuration. If both this value and the
  `DIGITAL_OCEAN_AUTH_TOKEN` environment variable are set, the backend
  prefers the `digitalOcean.authToken` value; this behavior is
  consistent with the EC2 backend and its preference of the
  `ec2.accessKeyId` value over the `AWS_ACCESS_KEY_ID` environment
  variable.

- When a DigitalOcean droplet is created, if neither auth token value
  is set, the backend aises an exception, rather than attempting to
  create the droplet anyway and failing with an error from the
  DigitalOcean API.

- Upon successful creation of a droplet, persist the DigitalOcean auth
  token used to create it in the NixOps state database. This happens
  regardless of whether the token was provided via the
  `digitalOcean.authToken` deployment configuration, or the
  `DIGITAL_OCEAN_AUTH_TOKEN` environment variable. Based on a review
  of the code in the EC2 backend, I believe this behavior is
  consistent with the EC2 backend, though I have not used the EC2
  backend myself to verify this.

- If the user does not set `digitalOcean.authToken` and uses the
  environment variable instead, the environment variable is only ever
  used to create the droplet; from that point on, NixOps will use the
  token value that was persisted in the NixOps state database. Again,
  based on a review of the code in the EC2 backend, I believe this
  behavior is consistent with the EC2 backend.
@dhess
Copy link
Contributor Author

dhess commented Mar 20, 2017

Could someone who uses the NixOps EC2 backend please confirm my belief that the EC2 backend persists AWS access keys to the NixOps database? I don't use it, myself, but unless I missed something in the EC2 code, I don't see how it could behave otherwise.

@dhess
Copy link
Contributor Author

dhess commented Mar 20, 2017

Also, I want to go on record as saying that I don't think it's a very good idea to persist AWS access keys or DigitalOcean API tokens to the NixOps state database! But that's a decision for the NixOps team to make -- this change only intends to make the DigitalOcean backend behave like the EC2 one already does with respect to those keys/tokens.

@dhess
Copy link
Contributor Author

dhess commented Mar 21, 2017

I've added support for IPv6 in a separate commit, but the diffs overlap, so I'll wait until the authToken fix is accepted before submitting the IPv6 enhancement.

edit: ugh, looks like GitHub has automatically merged the IPv6 commit into the authToken commit. :\

@dhess
Copy link
Contributor Author

dhess commented Mar 21, 2017

I've edited the title of this PR to reflect that GitHub has merged the 2 commits. I'm happy to resubmit them one-by-one if that's preferable.

@dhess dhess changed the title DigitalOcean backend: fix authToken issue (#628). DigitalOcean backend: fix authToken, add IPv6 support. Mar 21, 2017
@domenkozar
Copy link
Member

cc @teh

@teh
Copy link
Contributor

teh commented Mar 21, 2017

Thanks! I'm not 100% sure what to do here. ec2 stores the acces key but not the secret. And the DO token is both access key and secret rolled into one. I think I'd prefer to keep it out of the state file, but I also understand the argument for keeping it in.

@dhess what's your use case for carrying around the DO auth token in the state file?

@dhess
Copy link
Contributor Author

dhess commented Mar 21, 2017

As I mentioned above, I would prefer not to keep the token in the state file. I think it's a bad idea. But in this patch, I was only using the configuration mechanisms that NixOps supports out-of-the-box. Unless I'm missing something, once you set a property on a MachineState from the declarative configuration, it gets persisted to the state file.

You make a good point about how AWS separates the authentication into separate ID and key pieces, and that NixOps doesn't persist the AWS key in the NixOps database. That is because the key never needs to be set in the NixOps declarative configuration, only the ID. Luckily boto or whatever will handle loading the key for you, given just the ID. I hadn't considered that wrinkle. Unfortunately, the DigitalOcean Python module does not support loading a key from a .credentials file or similar; nor does the DigitalOcean API separate the two concepts of key and ID, as you pointed out.

Anyway, here is how I think the DigitalOcean backend should work: you set the digitalOcean.authToken in your declarative deployment configuration. You can set its value inline, if you choose; or you can use builtins.readFile to read the token from a file that you keep in an encrypted store (this is what I do). The DigitalOcean backend evaluates digitalOcean.authToken from the declarative configuration every time it performs an operation on a DigitalOcean target, and it never persists the value of this variable to the state file.

Unfortunately, I did not see any examples of this behavior in the existing backends. There appears to be no way to set a backend property with attr_property that is not persisted to the state database.

Is there were a way to create a property that is always loaded from the declarative configuration and never stored in the database? If not, it would be useful for cases like this.

@dhess
Copy link
Contributor Author

dhess commented Mar 21, 2017

At any rate, now that you've pointed out that the EC2 backend does not persist your EC2 secret to the state database, I think this patch is probably a bad idea and I am going to close it, unless someone objects.

@dhess
Copy link
Contributor Author

dhess commented Mar 21, 2017

OK, I'm closing this PR. It's a bad idea. Better to wait for a proper fix than to dump DigitalOcean auth tokens into the NixOps state database.

@dhess dhess closed this Mar 21, 2017
@teh
Copy link
Contributor

teh commented Mar 21, 2017

Is there were a way to create a property that is always loaded from the declarative configuration and never stored in the database? If not, it would be useful for cases like this.

I can't see anything obvious but then I'm struggling a bit with nixops internals :/

Thanks for doing the work in any case!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants