Google compute image #26214

zimbatm · 2017-05-29T17:46:39Z

Motivation for this change

This makes NixOS a first-class citizen on the Google Cloud Platform, allowing to integrate with the dynamic features of the platform.

Things done

Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

mention-bot · 2017-05-29T17:46:41Z

@zimbatm, thanks for your PR! By analyzing the history of the files in this pull request, we identified @rbvermaa, @8573 and @copumpkin to be potential reviewers.

Fuuzetsu · 2017-05-29T17:55:21Z

pkgs/tools/virtualization/google-compute-engine/default.nix

+
+    pushd google_compute_engine
+
+    patchPath /bin/systemctl /var/run/current-system/sw/bin/systemctl


don't understand motivation behind patchPath vs patchShebangs with proper links into store &c.

Probably executables are referenced not just in shebangs. But I would actually also prefer explicit links to nix store.

Fuuzetsu · 2017-05-29T17:55:48Z

LGTM except for patchPath stuff

Mic92 · 2017-05-29T19:17:15Z

nixos/modules/virtualisation/google-compute-image.nix

+    wants = [ "local-fs.target" "network-online.target" "network.target"];
+    wantedBy = [ "multi-user.target" ];
+    serviceConfig = {
+      ExecStart = "/run/current-system/sw/bin/true";


ExecStart = "${coreutils}/bin/true";

Mic92 · 2017-05-29T19:19:15Z

nixos/modules/virtualisation/google-compute-image.nix

@@ -191,56 +191,6 @@ in
    };
  };

-  # TODO: remove this
-  systemd.services.fetch-ssh-keys =
-    { description = "Fetch host keys and authorized_keys for root user";


How does this works now?

The google-accounts-daemon also scans for the SSH keys, but creates individual accounts per SSH key owner (with passwordless sudo). The root account isn't directly accessible through SSH anymore.

Mic92 · 2017-05-29T19:23:44Z

using ./nixos/maintainers/scripts/gce/create-gce.sh to build a gce image fails with:

Adding rules for package /nix/store/p7sm83r55gj4bl55k4pz794dsg6mxrny-extra-udev-rules                                                  
Copying /nix/store/p7sm83r55gj4bl55k4pz794dsg6mxrny-extra-udev-rules/etc/udev/rules.d/99-local.rules to /nix/store/llwyl34sl8ncrpxa7f3l49q8bhd98bns-udev-rules/99-local.rules                                                                                                  
Adding rules for package /nix/store/0pg85agkcjsz82b36kf4pzd87mm39iic-extra-hwdb-file                                                   
Checking that all programs called by relative paths in udev rules exist in /nix/store/rsilqxrxl6vcgd77ry4jd68a5jiwjnaw-systemd-232/lib/udev... OK                                                                                                                              
Checking that all programs called by absolute paths in udev rules exist... FAIL                                                        
/var/run/current-system/sw/bin/sh is called in udev rules but not installed by udev                                                    
building path(s) ‘/nix/store/708w74szmavanm8mg69wbyjzcgn2bjaq-initrd’                                                                  
building path(s) ‘/nix/store/nnq72hiq4kxryj9qi6izw1k2lz0c4csx-unit-google-startup-scripts.service’                                     
building path(s) ‘/nix/store/ic90njrpva2840xf3yjf2k3fayj4v038-unit-script’                                                             
building path(s) ‘/nix/store/ivv5qyznk4rc9k6qp1v3ii5qg50bbw9m-unit-script’                                                             
building path(s) ‘/nix/store/knq7r3b04lc6jq88gi92a47grxq3niaj-unit-script’                                                             
building path(s) ‘/nix/store/ckg4g12gsps10awy3g29mv1c3l654xhk-unit-serial-getty-hvc0.service-disabled’                                 
building path(s) ‘/nix/store/bbqib7f8j403ksr8fxpsg2simv7wv3l1-unit-serial-getty-ttyS0.service-disabled’                                
builder for ‘/nix/store/fafg91y4w6lm66jlq5r02q0ws8yiz3ay-udev-rules.drv’ failed with exit code 1                                       
cannot build derivation ‘/nix/store/1ga1lv36g7x9mfzwpam79499d511ylaa-etc.drv’: 1 dependencies couldn't be built                        
building path(s) ‘/nix/store/4cla2f0v95j02rw32abqib4j25vk7zzv-vm-run-stage2’                                                           
cannot build derivation ‘/nix/store/9r34z9lgrhxrqqdn7v4fcpbma89cym6l-nixos-system-unnamed-17.09.git.c1ef0ed.drv’: 1 dependencies couldn't be built                                                                                                                             
cannot build derivation ‘/nix/store/c8dxf2010n9hs45zzivkyc4qh4cxp9by-google-compute-image.drv’: 1 dependencies couldn't be built       
error: build of ‘/nix/store/c8dxf2010n9hs45zzivkyc4qh4cxp9by-google-compute-image.drv’ failed

rbvermaa · 2017-05-29T19:54:17Z

How does the google compute engine tool behave when you chose not to allow mutableUsers? Is it possible to disable anything related to creating users in the tool? If so, can we implement that depending on the mutableUsers config option?

FRidh · 2017-05-30T07:04:57Z

pkgs/tools/virtualization/google-compute-engine/default.nix

+
+  patches = [ ./0001-allow-nologin-other-paths.patch ];
+
+  patchPhase = ''


use postPatch, otherwise your patches is ignored because you define a new patchPhase.

zimbatm · 2017-05-30T08:15:27Z

How does the google compute engine tool behave when you chose not to allow mutableUsers? Is it possible to disable anything related to creating users in the tool? If so, can we implement that depending on the mutableUsers config option?

Good idea. I could not start the google-accounts-daemon is mutableUsers = false. In that case the nixos configuration would be provisioned through the metadata startup-script and would have to contain the SSH keys.

There is still work to do on this PR but thanks everyone for your feedback. I will let you know when it's ready for another round of review.

copumpkin · 2017-05-30T12:25:40Z

pkgs/tools/virtualization/google-compute-engine/default.nix

+    cp -r google_config/udev/*.rules $out/lib/udev/rules.d
+  '';
+
+  propagatedBuildInputs = with pythonPackages; [ boto setuptools ];


Curious why this would depend on boto 😮

Apparently just for one thing: setup boto to use the Google Cloud Storage as a backend (GCS has a S3-compatible API) ^_^ https://github.com/GoogleCloudPlatform/compute-image-packages#instance-setup

rbvermaa · 2017-05-30T12:35:13Z

Good idea. I could not start the google-accounts-daemon is mutableUsers = false. In that case the nixos configuration would be provisioned through the metadata startup-script and would have to contain the SSH keys.

What do you mean with 'through the metadata startup-script'? I am mostly interested in if this is backwards compatible with nixops.

zimbatm · 2017-05-30T19:42:53Z

What do you mean with 'through the metadata startup-script'? I am mostly interested in if this is backwards compatible with nixops.

Okay we need to have a bit of a conversation because right now it breaks backward-compatibility. Potentially I could leave the fetch-ssh-keys service and provide a migration path for future nixops release. Do you know how SSH keys are being provisioned with nixops?

zimbatm · 2017-05-30T20:17:37Z

Fixed all the things and left the fetch-ssh-keys daemon for nixops back-compat as both can live side-by-side.

I don't have a clear idea of what to do with the mutableUsers option yet as this module is not really meant to be designed to be configurable in the first place.

copumpkin · 2017-05-30T20:33:58Z

Potentially I could leave the fetch-ssh-keys service and provide a migration path for future nixops release. Do you know how SSH keys are being provisioned with nixops?

For EC2 at least, we left two options in place, so that the userdata could be used in two different ways for NixOps and for non-NixOps use cases. It would be nice to converge Google and EC2 behavior, so you can also configure Google Compute machines via userdata.

8573 · 2017-07-08T22:43:58Z

What's the status of this?

zimbatm · 2017-07-11T13:12:37Z

@8573 it's missing a corner-case where mutableUsers = true. Otherwise it's pretty stable. Feel free to push onto the branch, I will deal with it on Friday otherwise.

8573 · 2017-07-12T04:15:47Z

Okay; I don't understand GCE enough to help, so I'll wait. Thanks!

This adds a few google-specific services to setup the machine. Accounts are now dynamically created using the google-accounts-daemon, which allows to click on the "SSH" button in the console and have it working. The NixOS image now supports the userdata startup and shutdown scripts. Misc: * add all the google services from https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/google_compute_engine_init/systemd * add udev rules for disk labels * synched sysctl rules with https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_config/sysctl/11-gce-network-security.conf

zimbatm · 2017-07-15T18:43:54Z

Alright, the account service is now disabled if users.mutableUsers = false

zimbatm · 2017-07-19T08:48:59Z

I think the solution is reasonable right now. Let me know if you hit any issues.

zimbatm requested a review from Fuuzetsu May 29, 2017 17:47

Fuuzetsu reviewed May 29, 2017

View reviewed changes

Mic92 reviewed May 29, 2017

View reviewed changes

FRidh requested changes May 30, 2017

View reviewed changes

copumpkin reviewed May 30, 2017

View reviewed changes

zimbatm mentioned this pull request May 30, 2017

allow other paths for nologin GoogleCloudPlatform/compute-image-packages#406

Merged

zimbatm force-pushed the google-compute-image branch 3 times, most recently from b40fe2f to 0895e6d Compare May 30, 2017 20:15

google-compute-engine: init at 20170523

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

812346c

zimbatm force-pushed the google-compute-image branch from 0895e6d to b4ba2cb Compare July 15, 2017 17:17

zimbatm force-pushed the google-compute-image branch from b4ba2cb to c93d68b Compare July 15, 2017 18:36

zimbatm merged commit 14f53e5 into NixOS:master Jul 19, 2017

zimbatm deleted the google-compute-image branch July 19, 2017 08:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Google compute image #26214

Google compute image #26214

zimbatm commented May 29, 2017 •

edited

Loading

mention-bot commented May 29, 2017

Fuuzetsu May 29, 2017

Mic92 May 29, 2017 •

edited

Loading

Fuuzetsu commented May 29, 2017

Mic92 May 29, 2017

Mic92 May 29, 2017

zimbatm May 29, 2017

Mic92 commented May 29, 2017

rbvermaa commented May 29, 2017

FRidh May 30, 2017

zimbatm commented May 30, 2017

copumpkin May 30, 2017

zimbatm May 30, 2017

rbvermaa commented May 30, 2017

zimbatm commented May 30, 2017

zimbatm commented May 30, 2017

copumpkin commented May 30, 2017

8573 commented Jul 8, 2017

zimbatm commented Jul 11, 2017 •

edited

Loading

8573 commented Jul 12, 2017

zimbatm commented Jul 15, 2017

zimbatm commented Jul 19, 2017


		pushd google_compute_engine

		patchPath /bin/systemctl /var/run/current-system/sw/bin/systemctl


		patches = [ ./0001-allow-nologin-other-paths.patch ];

		patchPhase = ''

Google compute image #26214

Google compute image #26214

Conversation

zimbatm commented May 29, 2017 • edited Loading

Motivation for this change

Things done

mention-bot commented May 29, 2017

Fuuzetsu May 29, 2017

Choose a reason for hiding this comment

Mic92 May 29, 2017 • edited Loading

Choose a reason for hiding this comment

Fuuzetsu commented May 29, 2017

Mic92 May 29, 2017

Choose a reason for hiding this comment

Mic92 May 29, 2017

Choose a reason for hiding this comment

zimbatm May 29, 2017

Choose a reason for hiding this comment

Mic92 commented May 29, 2017

rbvermaa commented May 29, 2017

FRidh May 30, 2017

Choose a reason for hiding this comment

zimbatm commented May 30, 2017

copumpkin May 30, 2017

Choose a reason for hiding this comment

zimbatm May 30, 2017

Choose a reason for hiding this comment

rbvermaa commented May 30, 2017

zimbatm commented May 30, 2017

zimbatm commented May 30, 2017

copumpkin commented May 30, 2017

8573 commented Jul 8, 2017

zimbatm commented Jul 11, 2017 • edited Loading

8573 commented Jul 12, 2017

zimbatm commented Jul 15, 2017

zimbatm commented Jul 19, 2017

zimbatm commented May 29, 2017 •

edited

Loading

Mic92 May 29, 2017 •

edited

Loading

zimbatm commented Jul 11, 2017 •

edited

Loading