Flake support #68897

edolstra · 2019-09-16T13:11:11Z

Motivation for this change

This PR:

Adds a flake.nix file to Nixpkgs, making Nixpkgs usable from other flakes.
Adds flake support to nixos-rebuild, enabling hermetic NixOS builds.
Adds flake support to nixos-container.

A typical NixOS system configuration flake looks like this:

{
  edition = 201909;

  outputs = { self, nixpkgs, dwarffs, hydra }: rec {

    nixosConfigurations.my-server = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ dwarffs.nixosModules.dwarffs
          nixpkgs.nixosModules.notDetected
          hydra.nixosModules.hydra
          { system.configurationRevision = self.rev;
            /* typical configuration.nix stuff follows */
          }
        ];
    };

  };
}

which can be installed by doing

# nixos-rebuild switch --flake /path/to/flake --config my-server

(--config defaults to the host name so it can usually be omitted).

The option system.configurationRevision tracks the revision of the top-level flake. This allows the entire configuration to be reproduced reliably. In a running system, it can be queried using nixos-version:

$ nixos-version --json | jq -r .configurationRevision
2c475bc41c13e7a9ae5c16b6c10d8d328cd07ee7

Note: NIX_PATH is unavailable in flake-based builds. In particular this means that

imports = [ <nixpkgs/nixos/modules/installer/scan/not-detected.nix> ];

in hardware-configuration.nix no longer works. Instead you have to use

modules = [ nixpkgs.nixosModules.notDetected ];

nixos-container also supports flakes. For example, the following command creates a container that serves the NixOS homepage:

$ nixos-container create homepage --flake nixos-homepage 
$ nixos-container start homepage 
$ curl http://$(nixos-container show-ip homepage)

TODO: add flake support to nixos-install, add tests.

Things done

Notify maintainers

cc @

flokli · 2019-09-16T15:09:18Z

flake.nix

+
+  outputs = { self }:
+    let
+      pkgs = import ./. { system = "x86_64-linux"; };


does this mean we currently can't use the nixpkgs flake from any non-x86 system?

Not from the nix command line. You can call it from other flakes with a non-x86_64-linux system argument though.

Can't we use builtins.currentSystem?

Nope, not in pure evaluation mode.

$ nix eval "(builtins.currentSystem)" error: attribute 'currentSystem' missing, at (string):1:2 $ nix eval --impure "(builtins.currentSystem)" "x86_64-linux"

arianvp · 2019-09-17T17:56:37Z

I myself had a slightly different take on nixos-rebuild in the presence of flakes which I am using here: https://github.com/arianvp/nixos-stuff/blob/master/deployments.nix#L15-L46 (Though I'm still implementing the flake part, but the idea of calling lib.nixosSystem inside your package definition is the same to what I am doing).

The idea is that you ou can do:

nix run github:arianvp/nixos-stuff:deployments.arianvp-me -c deploy [switch|boot|test] [target*]

or in two steps:

nix build github:arianvp/nixos-stuff:deployments.arianvp-me
./result/deploy switch

# or deploy remotely
./result/deploy switch  root@arianvp.me

E.g. a nixos flake returns a nixos-rebuild-like command that can be called instead of having nixos-rebuild call nix build. Kind of an inversion of control thing.

Would such a thing be more ergonomic? It would also mean you don't need nixos-rebuild on your computer to be able to deploy a nixos derivation. (Because nixos-rebuild doesn't seem to be exposed as a package in nixpkgs. so I do this so I can deploy nixos servers from my non-nixos machines)

Of course this is a bit of a backwards incompatible change, but we could also keep the old nixos-rebuild for non-flake deployments, and use this "design pattern" for flake-like deployments.

What do you think?

edolstra · 2019-09-19T17:10:34Z

@arianvp Well, you already don't need nixos-rebuild since you can build the top-level system derivation and then run its switch-to-configuration script, e.g.

$ nix build .:nixosConfigurations.vyr.config.system.build.toplevel
$ ./result/bin/switch-to-configuration test

which is just what nixos-rebuild does.

Edit: or indeed

$ nix run .:nixosConfigurations.vyr.config.system.build.toplevel -c switch-to-configuration test

as you suggested.

edolstra · 2019-09-19T17:48:52Z

Update: added flake support to nixos-container.

arianvp · 2019-09-19T18:06:58Z

Well and you have to nix-env --set it right? Would there be a way to do that in one step?

flake.nix

matthewbauer · 2019-09-26T22:12:57Z

flake.nix

+      legacyPackages = pkgs;
+
+      nixosModules = {
+        notDetected = ./nixos/modules/installer/scan/not-detected.nix;


Why include this?

Note: NIX_PATH is unavailable in flake-based builds.

Should we add other modules as well? For example, I use the following imports on my server:

<nixpkgs/nixos/modules/profiles/minimal.nix> <nixpkgs/nixos/modules/virtualisation/container-config.nix> <nixpkgs/nixos/modules/installer/cd-dvd/channel.nix>

nixos/modules/installer/tools/nixos-rebuild.sh

nixos-discourse · 2019-11-11T22:40:08Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/building-nixos-system-with-nix-build-and-a-channel-specifier/4747/2

Dirty trees no longer set 'rev', so we need to handle this.

And also --refresh and --no-net.

nixos/modules/installer/tools/nixos-version.sh

yorickvP

Would be happy to see this merged. However, I'm afraid that the FIXME's in here will never get fixed once this is merged. We depend on the nixos-rebuild re'execing currently, so that fixme would be a blocker for moving to flakes for us, personally.

yorickvP · 2020-02-06T17:29:49Z

nixos/modules/installer/tools/nixos-rebuild.sh

 # Re-execute nixos-rebuild from the Nixpkgs tree.
-if [ -z "$_NIXOS_REBUILD_REEXEC" -a -n "$canRun" -a -z "$fast" ]; then
+# FIXME: get nixos-rebuild from $flake.


This is important for my workflow

The re-exec is fundamentally broken anyway, since it tries to build nixos-rebuild with the old Nix. (The code to upgrade Nix happens later.) We could do nix build $flake#nixosConfigurations.$flakeAttr.config.system.build.nixos-rebuild but I don't think it's useful currently.

nixos/modules/installer/tools/nixos-rebuild.sh

yorickvP · 2020-02-06T17:30:17Z

nixos/modules/installer/tools/nixos-rebuild.sh

@@ -296,7 +347,8 @@ prebuiltNix() {

 remotePATH=

-if [ -n "$buildNix" ]; then
+# FIXME: get nix from the flake.


FIXME: get nix from the flake.

yorickvP · 2020-02-06T17:31:05Z

pkgs/tools/virtualization/nixos-container/nixos-container.pl

-         (defined $configFile && $configFile ne "")) {
-        writeNixOSConfig $nixosConfigFile;
+    # Unless overriden on the command line, rebuild the flake recorded
+    # in the container config file. FIXME: read the container config


fixme: read the container config in a more sensible way

pkgs/tools/virtualization/nixos-container/nixos-container.pl

yorickvP

^

yorickvP · 2020-02-06T17:42:23Z

nixos/modules/installer/tools/nixos-rebuild.sh

+        if [[ -z $flake ]]; then
+            pathToConfig="$(nixBuild '<nixpkgs/nixos>' -A vm -k "${extraBuildFlags[@]}")"
+        else
+            echo "TODO: not implemented" >&2


todo: not implemented

Not supporting nixos vm builds will ensure limited flake adoption.

I haven't used build-vm in years.

I use it almost all the time, to do integration tests of modules. It's fairly prominent in the documentation and one of nixos's selling points.

Sure, but nixos-rebuild build-vm is just a wrapper around nix build. It doesn't do anything you couldn't otherwise do.

I've used build-vm to test the many many PRs that I've reviewed, merged, and tested. It's a very very useful tool for me.

Do you mean people can just do?

nix run vm -f '<nixpkgs/nixos>' --arg configuration "$PWD/configuration.nix" -c run-nixos-vm

yorickvP · 2020-02-06T17:42:31Z

nixos/modules/installer/tools/nixos-rebuild.sh

+        if [[ -z $flake ]]; then
+            pathToConfig="$(nixBuild '<nixpkgs/nixos>' -A vmWithBootLoader -k "${extraBuildFlags[@]}")"
+        else
+            echo "TODO: not implemented" >&2


todo: not implemented

alyssais · 2020-02-06T18:02:31Z

I feel very conflicted about making this change now, when not only have we still not decided on the format for flake.nix, but the RFC seems to have been stalled for months…

nixos/modules/installer/tools/nixos-rebuild.sh

"master" is not a valid SHA-1 commit hash, and it's not even necessarily the branch used. 'nixos-version --revision' now returns an error if the commit hash is not known.

edolstra · 2020-02-10T14:55:43Z

Well and you have to nix-env --set it right? Would there be a way to do that in one step?

Yes, you can do nix build --profile /nix/var/nix/profiles/system $flake#nixosConfigurations.$hostname.config.system.build.toplevel to do the build and the profile update in one step.

FRidh

I would like to know how we deal on the stable with changes to e.g. the Flake format. Can it be relied on being stable during the lifetime of 20.03, or is it consider experimental? Probably needs a release note either way, clarifying this.

flake.nix

This doesn't work in pure mode.

edolstra · 2020-02-10T15:38:21Z

I've added a note that it's experimental (so it shouldn't be relied on).

arianvp · 2020-02-13T21:00:21Z

Nixpkgs.lib.nixosSystem already exists as pkgs.nixos
It's confusing to have both. Can we remove one of the two?

Iirc nixpkgs.nixos is explicitly mentioned in our documentation too

I already noted this in November. But no reply. Could we still address this?

edolstra · 2020-02-13T23:18:40Z

Ideally we would remove pkgs.nixos, since it's not a package, but I don't want to make incompatible changes since this is all experimental.

edolstra added 0.kind: enhancement 6.topic: nixos 9.needs: documentation 6.topic: reproducible builds 9.needs: community feedback 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Sep 16, 2019

ofborg bot added 8.has: documentation 8.has: module (update) labels Sep 16, 2019

flokli reviewed Sep 16, 2019

View reviewed changes

ofborg bot added the 2.status: merge conflict label Sep 19, 2019

edolstra force-pushed the master branch from b40b0e8 to c5a7074 Compare September 19, 2019 17:19

ofborg bot added 10.rebuild-linux: 1-10 and removed 2.status: merge conflict 10.rebuild-linux: 0 labels Sep 19, 2019

edolstra force-pushed the master branch from 71a1b42 to 0a4dd6e Compare September 23, 2019 12:36

Mic92 reviewed Sep 23, 2019

View reviewed changes

flake.nix Outdated Show resolved Hide resolved

matthewbauer reviewed Sep 26, 2019

View reviewed changes

edolstra force-pushed the master branch 2 times, most recently from 1028bb3 to 4a4e146 Compare October 15, 2019 11:52

Mic92 reviewed Oct 19, 2019

View reviewed changes

nixos/modules/installer/tools/nixos-rebuild.sh Outdated Show resolved Hide resolved

edolstra force-pushed the master branch from d4a07f0 to 153ca08 Compare October 23, 2019 19:31

ofborg bot added the 2.status: merge conflict label Oct 23, 2019

edolstra force-pushed the master branch from 153ca08 to 015c9ec Compare November 6, 2019 11:31

ofborg bot removed the 2.status: merge conflict label Nov 6, 2019

edolstra added 2 commits February 5, 2020 23:15

flake.nix: Handle missing rev attribute

60330b6

Dirty trees no longer set 'rev', so we need to handle this.

nixos-rebuild: Propagate various flake lock file flags

c089308

And also --refresh and --no-net.

edolstra changed the title ~~[WIP] Flake support~~ Flake support Feb 5, 2020

edolstra force-pushed the master branch from a6ca3f4 to c089308 Compare February 5, 2020 22:15

ofborg bot removed the 2.status: merge conflict label Feb 5, 2020

samueldr reviewed Feb 5, 2020

View reviewed changes

nixos/modules/installer/tools/nixos-version.sh Show resolved Hide resolved

yorickvP requested changes Feb 6, 2020

View reviewed changes

pasqui23 reviewed Feb 7, 2020

View reviewed changes

nixos/modules/installer/tools/nixos-rebuild.sh Outdated Show resolved Hide resolved

edolstra added 5 commits February 10, 2020 15:45

nixos-rebuild: Make 'edit' work with with flakes

4089dbf

nixos-rebuild: Remove --keep-going flag

f9392f0

nixos-version --json: Use builtins.toJSON

b98ea45

nixos.revision: Use null instead of "master"

c05cc61

"master" is not a valid SHA-1 commit hash, and it's not even necessarily the branch used. 'nixos-version --revision' now returns an error if the commit hash is not known.

nixos-rebuild: Remove TODOs

8f86624

FRidh reviewed Feb 10, 2020

View reviewed changes

flake.nix Show resolved Hide resolved

edolstra added 3 commits February 10, 2020 16:20

lib/tests/misc.nix: Fix dependency on currentSystem

a274da6

This doesn't work in pure mode.

lib/tests/misc.nix: Don't make a copy on Nixpkgs

5c389f8

Fix 'nix flake check'

64e5f4d

edolstra requested review from infinisil and nbp as code owners February 10, 2020 15:27

edolstra added 2 commits February 10, 2020 16:32

Doh

fb05afd

flake.nix: Add note

a1fa9e6

edolstra merged commit 0e6ceb8 into NixOS:master Feb 10, 2020

edolstra mentioned this pull request Feb 20, 2020

[RFC 0049] Flakes NixOS/rfcs#49

Closed

worldofpeace mentioned this pull request Sep 9, 2020

nixos-rebuild + flakes to support build-vm #97487

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flake support #68897

Flake support #68897

edolstra commented Sep 16, 2019 •

edited

flokli Sep 16, 2019

edolstra Sep 19, 2019

matthewbauer Sep 26, 2019

jD91mZM2 Jan 15, 2020

arianvp commented Sep 17, 2019 •

edited

edolstra commented Sep 19, 2019 •

edited

edolstra commented Sep 19, 2019

arianvp commented Sep 19, 2019

matthewbauer Sep 26, 2019

jtojnar Oct 17, 2019 •

edited

nixos-discourse commented Nov 11, 2019

yorickvP left a comment

yorickvP Feb 6, 2020

edolstra Feb 10, 2020

yorickvP Feb 6, 2020

yorickvP Feb 6, 2020

yorickvP left a comment

yorickvP Feb 6, 2020

yorickvP Feb 10, 2020

edolstra Feb 10, 2020

yorickvP Feb 10, 2020

edolstra Feb 10, 2020

worldofpeace Feb 10, 2020 •

edited

worldofpeace Feb 11, 2020

yorickvP Feb 6, 2020

alyssais commented Feb 6, 2020

edolstra commented Feb 10, 2020

FRidh left a comment

edolstra commented Feb 10, 2020

arianvp commented Feb 13, 2020 •

edited

edolstra commented Feb 13, 2020

Flake support #68897

Flake support #68897

Conversation

edolstra commented Sep 16, 2019 • edited

Motivation for this change

Things done

Notify maintainers

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arianvp commented Sep 17, 2019 • edited

edolstra commented Sep 19, 2019 • edited

edolstra commented Sep 19, 2019

arianvp commented Sep 19, 2019

Choose a reason for hiding this comment

jtojnar Oct 17, 2019 • edited

Choose a reason for hiding this comment

nixos-discourse commented Nov 11, 2019

yorickvP left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yorickvP left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

worldofpeace Feb 10, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alyssais commented Feb 6, 2020

edolstra commented Feb 10, 2020

FRidh left a comment

Choose a reason for hiding this comment

edolstra commented Feb 10, 2020

arianvp commented Feb 13, 2020 • edited

edolstra commented Feb 13, 2020

edolstra commented Sep 16, 2019 •

edited

arianvp commented Sep 17, 2019 •

edited

edolstra commented Sep 19, 2019 •

edited

jtojnar Oct 17, 2019 •

edited

worldofpeace Feb 10, 2020 •

edited

arianvp commented Feb 13, 2020 •

edited