I run a system with thin lvm. To boot my system, initrd doesn't need all of these tools, only thin-check (which I currently add using preLVMCommands). I'm not sure the others are really necessary to boot.

Also I think adding thin lvm tools should be made conditional on actually using thin lvm instead of adding them "just in case".

IMO the best solution is a new NixOS options boot.initrd.useThinLVM (or whatever better name) that activates the kernel module and adds the necessary tool(s) to initrd.

@xeji I just managed to bring my system back up after migration to thin volumes using this hack:

  boot.initrd.extraUtilsCommands = ''
    copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/pdata_tools
    copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/thin_check
  '';
  boot.initrd.preLVMCommands = ''
    mkdir -p /etc/lvm
    echo "global/thin_check_executable = \"$(which thin_check)\"" >> /etc/lvm/lvm.conf
  '';

just adding thin_check symlink and pdata_tools binary doesn't cut it.

All other binaries are just symlinks to pdata_tools like thin_check is, so they add nothing to the size of initrd, but allow to support cached volumes as well as repairing all of them, so I think we should keep them around.

As for adding another option, I though about that, but after I found out that size of initrd doesn't change much, decided to push it without option first. I guess I'll add smth like enableLVMThinCache, to highlight support for both cache and thin volumes. Another option would be to add a list of features like boot.initrd.enableLVMFeatures = [ "thin" "cache" ] that would also add appropriate kernel modules to initrd as well. In that case we could default it to smth like [ "basic" ] and allow user to specify empty list to avoid adding LVM stuff altogether if it's not needed.

What do you think?

All other binaries are just symlinks to pdata_tools like thin_check is, so they add nothing to the size of initrd,

Right, I didn't realize that, so adding them all is fine.

I like the idea to have one option to activate both the kernel module and necessary tools. My personal preference would be to have simple boolean options, sth like: boot.initrd.lvmThin.enable and boot.initrd.lvmCache.enable rather than a feature list (but that's not a strong opinion).

Let's wait a little and see what other people think about the 1MB increase in initrd size first.

This makes me wonder why stuff like mdadm is included by default.

Yep, mdadm could also be optional. (1MB), although making it optional will break some existing configs...

I think we should make initrd fully configurable, defaulting to current state. This way if one cares one can strip it down to only what is needed for one's config. We'll have to carve things out of stage1 as well though.

mdadm could be also not included based on the stateVersion to avoid breakages of configuration.

We could do that, and I would, but there are strong objections to such uses of stateVersion, see e.g. #39329 (comment) .

I only see limited values in making it optional but disabled by default. Most people will keep it then also they do not ever need it. Is it not possible to detect if mdadm or lvm are actually needed?

It is possible to add such detection functionality to nixos-generate-config, for example, and explicitly disable everything not needed in hardware-configuration.nix. I'm not sure this would help existing users though, but for new installs this would bring smaller initrd.

Any updates on this pull request, please?

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the
   related code and @ mention them in a comment.
2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
   irc.freenode.net.

I have tried the release-2020.09, nixpkgs.git hash c6fa7bb today.

The boot works, though I see following (transcribed from screen shot so end lines are cut off):

Verifying passphrase for /dev/disk/by-uuid/... - success
Starting device mapper and LVM...
    /nix/store/eee...eee-thin-provisioning-tools-0.9.0/bin/cache_check: execvp fail...
    WARNING: Check is skipped, please install recommended missing binary /nix/store/eee...eee-thin-provisioning-tools-0.9.0/bin/cache_check!
    4 logical volume(s) in volume group "vg" now active
...

The cache check is skipped but it doesn't seem to be critical for the boot to progress.

@SnVIZQ do you have services.lvm.boot.thin.enable (from here) set in your system configuration?

@ajs124, I didn't have it there, so I've added it. No difference though. Rechecked it is enabled:

$ nixos-option services.lvm.boot.thin.enable
Value:
true

Default:
false

Type:
"boolean"

Example:
true

Description:
"Whether to enable support for booting from ThinLVs."

Declared by:
[ "/var/nixos/nixpkgs/nixos/modules/tasks/lvm.nix" ]

Defined by:
[ "/etc/nixos/configuration.nix" ]

I've also replaced thin_check with cache_check:

$ git diff
diff --git a/nixos/modules/tasks/lvm.nix b/nixos/modules/tasks/lvm.nix
index 2c3cc4c5467..db0076bdfda 100644
--- a/nixos/modules/tasks/lvm.nix
+++ b/nixos/modules/tasks/lvm.nix
@@ -43,12 +43,12 @@ in {

         extraUtilsCommands = ''
           copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/pdata_tools
-          copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/thin_check
+          copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/cache_check
         '';
       };

       environment.etc."lvm/lvm.conf".text = ''
-        global/thin_check_executable = "${pkgs.thin-provisioning-tools}/bin/thin_check"
+        global/thin_check_executable = "${pkgs.thin-provisioning-tools}/bin/cache_check"
       '';
     })
     (mkIf (cfg.dmeventd.enable || cfg.boot.thin.enable) {
@@ -56,7 +56,7 @@ in {
           mkdir -p /etc/lvm
           cat << EOF >> /etc/lvm/lvm.conf
           ${optionalString cfg.boot.thin.enable ''
-            global/thin_check_executable = "$(command -v thin_check)"
+            global/thin_check_executable = "$(command -v cache_check)"
           ''}
           ${optionalString cfg.dmeventd.enable ''
             dmeventd/executable = "$(command -v false)"

... to see if would help, but nope... Still the same:

/nix/store/eee...eee-thin-provisioning-tools-0.9.0/bin/cache_check: execvp failed: No such file or directory

This looks to me like some lvm binary, which is trying to execute cache_check, is not properly adjusted. The old hack which replaces strings within binaries using perl voodoo:

  boot.initrd.extraUtilsCommands = ''
    # Put thin-provisioning-tools into extra-utils and patch lvm accordingly.
    # NOTE: this works only because thin-provisioning-tools string, including
    # version, is longer than extra-utils string. The difference is zeroed. If
    # it would be vice versa there is a chance it would not work because the
    # stuff after the full path to the tool would be overwritten. Although there
    # seem to be some other, documentation, string just behind the full path
    # name which might not be that important... Anyways, not spending time
    # to figure out how to avoid the patching in case it is not possible doing
    # the proper way.
    for BIN in ${pkgs.thin-provisioning-tools}/bin/*; do
      copy_bin_and_libs $BIN
      SRC="(?<all>/[a-zA-Z0-9/]+/[0-9a-z]{32}-[0-9a-z-.]+(?<exe>/bin/$(basename $BIN)))"
      REP="\"$out\" . \$+{exe} . \"\\x0\" x (length(\$+{all}) - length(\"$out\" . \$+{exe}))"
      PRP="s,$SRC,$REP,ge"
      ${pkgs.perl}/bin/perl -p -i -e "$PRP" $out/bin/lvm
    done
  '';
  boot.initrd.extraUtilsCommandsTest = ''
    # The thin-provisioning-tools use pdata_tools binary as a link target of
    # supported utils so it is enough to check only one, the others should
    # "just" work...
    $out/bin/pdata_tools cache_check -V
  '';

works fine though.

I marked this as stale due to inactivity. → More info

@ajs124 could you take a look at this?

@SnVIZQ do you also have dm-cache mappings?

@SnVIZQ do you also have dm-cache mappings?

@ajs124 I'm not sure what exactly you mean by "dm-cache mappings", but my gut feeling is the answer is yes. The basic topology si like this: There's LUKS on low level - on SSDs and HDDs, on top of that is LVM, where HDDs use cache residing on SSD - the standard lvmcache(7) approach.

That's what I meant, yes. LVM2 manages device mapper things for you. We currently only support booting from thin pools or lvmthin(7), but not lvmcache(7).

Can you try if this works:

diff --git a/nixos/modules/tasks/lvm.nix b/nixos/modules/tasks/lvm.nix
index 98a0e2ddef9..a4311c6baa9 100644
--- a/nixos/modules/tasks/lvm.nix
+++ b/nixos/modules/tasks/lvm.nix
@@ -48,11 +48,13 @@ in {
         extraUtilsCommands = ''
           copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/pdata_tools
           copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/thin_check
+          copy_bin_and_libs ${pkgs.thin-provisioning-tools}/bin/cache_check
         '';
       };
 
       environment.etc."lvm/lvm.conf".text = ''
         global/thin_check_executable = "${pkgs.thin-provisioning-tools}/bin/thin_check"
+        global/cache_check_executable = "${pkgs.thin-provisioning-tools}/bin/cache_check"
       '';
     })
     (mkIf (cfg.dmeventd.enable || cfg.boot.thin.enable) {
@@ -61,6 +63,7 @@ in {
           cat << EOF >> /etc/lvm/lvm.conf
           ${optionalString cfg.boot.thin.enable ''
             global/thin_check_executable = "$(command -v thin_check)"
+            global/cache_check_executable = "$(command -v cache_check)"
           ''}
           ${optionalString cfg.dmeventd.enable ''
             dmeventd/executable = "$(command -v false)"

I have applied your patch (addition of cache_check stuff to lvm.nix) on top of b1ee3bc commit (release-21.05 branch from 2021-08-04_10-54-24), removed the "old hack" (extension of extraUtilsCommands{,Test} from my older post above), set the services.lvm.boot.thin.enable to true, rebuilt and activated this configuration and I can confirm it booted just fine.

So I'd vote for having the patch applied because it solves my issue. :-)

We could do that, although the option kind of has a misleading name, then.

Then again, cache_check is just a symlink to pdata_tools, like thin_check, so this won't increase the initrd size. And the cache_* tools are part of thin-provisioning-tools.

I'll open a PR.