Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refresh instance metadata on boot #104456

Merged
merged 4 commits into from Nov 22, 2020

Conversation

endgame
Copy link
Contributor

@endgame endgame commented Nov 21, 2020

Motivation for this change

Re-fetch instance meta data/user data on each boot. This means that we can stop instances, change user data, reboot the instance, and expect the new user data to actually be applied.

NOTE I tried to cargo-cult the docbook but I must have screwed it up. Help please? Never mind, my test instance ran out of memory trying to rebuild docs.

Things done
  • Built an EC2 AMI at commit f2ca527 (i.e., everything but the docbook changes), launched that AMI with no user data, stopped the instance, set up some user data and rebooted the instance, and checked that the user data was being applied.

Ping @grahamc

Copy link
Member

@grahamc grahamc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this doesn't change anything about how we support NixOps, right? If there was userdata for a nixops-managed host, this has the same semantics -- just it'd fetch newer metadata on boot?

The metadata fetcher scripts run each time an instance starts, and it
is not safe to assume that responses from the instance metadata
service (IMDS) will be as they were on first boot.

Example: an EC2 instance can have its user data changed while
the instance is stopped. When the instance is restarted, we want to
see the new user data applied.
@endgame endgame force-pushed the refresh-instance-metadata-on-boot branch from 11b70c5 to 4187098 Compare November 22, 2020 01:06
@endgame
Copy link
Contributor Author

endgame commented Nov 22, 2020

I suppose this doesn't change anything about how we support NixOps, right? If there was userdata for a nixops-managed host, this has the same semantics -- just it'd fetch newer metadata on boot?

Nixops will not change the ssh hostkeys even on userdata change, because the hostkey will still exists:

key="$(sed 's/|/\n/g; s/SSH_HOST_DSA_KEY://; t; d' $userData)"

I think it will still run the same amazon-init service and reapply configuration.nix userdata:

if sed '/^\(#\|SSH_HOST_.*\)/d' < "$userData" | grep -q '\S'; then

Should any of this discussion be replicated in the release notes? Also, do you have any idea what might be going wrong with the docbook stuff?

@endgame endgame force-pushed the refresh-instance-metadata-on-boot branch 2 times, most recently from 6ef3a5c to 1b653b2 Compare November 22, 2020 02:13
@endgame endgame force-pushed the refresh-instance-metadata-on-boot branch from 1b653b2 to 6fd871d Compare November 22, 2020 02:23
@grahamc grahamc merged commit 1ee1134 into NixOS:master Nov 22, 2020
@endgame endgame deleted the refresh-instance-metadata-on-boot branch September 29, 2021 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants