Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixos/zfs: backport import and encryption handling improvements #42920

Closed
wants to merge 5 commits into from
Closed

nixos/zfs: backport import and encryption handling improvements #42920

wants to merge 5 commits into from

Conversation

woffs
Copy link
Contributor

@woffs woffs commented Jul 3, 2018 •
edited

Motivation for this change

backport #42269 and #41846

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

woffs and others added 5 commits July 3, 2018 17:06
@woffs
nixos/zfs: load encryption keys for all pools
f49fb68 
fixes NixOS#41838

At the moment it works fine for "file://" keys, but does not work for
dataPools with "prompt" keys, because the passphrase cannot be entered
(yet).

(cherry picked from commit a254d9d)
@Mic92 @woffs
nixos/zfs: improve encryption option description
aee1e99 
(cherry picked from commit 2f303d2)
@Mic92 @woffs
nixos/zfs: grammar nit
fb9ba58 
(cherry picked from commit 78e6c0d)
@Mic92 @woffs
nixos/zfs: inline zfs command for data pools service
4d3c698 
(cherry picked from commit 81a8baf)
@Baughn @woffs
zfs: Improve import handling
158103d 
(cherry picked from commit cfd8c4e)
@woffs
Copy link
Contributor Author

woffs commented Jul 3, 2018

@GrahamcOfBorg test installer.zfsroot

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.installer.zfsroot

Partial log (click to expand)

while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/release.nix:29:25, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:199:52:
while evaluating 'hydraJob' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/customisation.nix:162:14, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/release.nix:29:28:
while evaluating the attribute 'drvPath' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/customisation.nix:179:13:
while evaluating the attribute 'drvPath' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/customisation.nix:146:13:
while evaluating the attribute 'buildCommand' of the derivation 'vm-test-run-installer-zfs-root' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute 'testScript' of the derivation 'nixos-test-driver-installer-zfs-root' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating 'isFunction' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/trivial.nix:161:16, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/lib/testing.nix:100:12:
while evaluating 'testScriptFun' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/tests/installer.nix:66:19, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/tests/installer.nix:262:20:
Non-EFI boot methods are only supported on i686 / x86_64

@Baughn
Copy link
Contributor

Baughn commented Jul 3, 2018

FYI: I'm going to try building unit tests for these. In theory that should be doable. Probably don't want to wait up for me, but I'll try to report back tonight.

@woffs
Copy link
Contributor Author

woffs commented Jul 3, 2018

@Baughn there is a zfs test, but it is not linked into release.nix, and I don't know its state. @adisbladis

@Baughn
Copy link
Contributor

Baughn commented Jul 3, 2018

@woffs The test does run, at least; I have it executed as part of my system build scripts, just in case.

What I mean is I'll try to expand it to test the multi-vdev / slow import case that I was trying to fix. And probably encryption as well, because why not.

@woffs
Copy link
Contributor Author

woffs commented Jul 3, 2018

@Baughn great. Thanks for your work!

@adisbladis
Copy link
Member

@woffs The ZFS tests are fine and should pass. I intended to add them to some release but simply forgot.

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: tests.installer.zfsroot

Partial log (click to expand)

building '/nix/store/h1hi97jk1qxdy3krhmxm9zxvprx9iq5z-system-units.drv'...
cannot build derivation '/nix/store/4b293zx4aw6npmqa4whp0v4rsrisbff4-etc.drv': 1 dependencies couldn't be built
building '/nix/store/lc118i24ssgmps8nylrwchg9v321lwsw-users-groups.json.drv'...
cannot build derivation '/nix/store/8fayzs89ghkjwbkiqz26dbrgl4mf1b8z-nixos-system-machine-18.03.git.158103d.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/2n1lvv4v4a903rkfyfcj1d1yi3svc3n0-closure-info.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/xxlxpfdjvfl38aglc0jl0b9gw4bfbybj-run-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/rgbwhdnwb671x5b995f2w5qcvmgsgxlg-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/jzrxlcbvs06rmvpkzz94i1alfc3aar1y-nixos-test-driver-installer-zfs-root.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/x1466c771ad31kvgfqbc7h9b9k2fph2l-vm-test-run-installer-zfs-root.drv': 1 dependencies couldn't be built
error: build of '/nix/store/x1466c771ad31kvgfqbc7h9b9k2fph2l-vm-test-run-installer-zfs-root.drv' failed

@woffs
Copy link
Contributor Author

woffs commented Jul 3, 2018 •
edited

Just a timeout. Sorry for submitting a kernel build job here. I have built and tested this locally, it works (in my case, 24 disks, 1 encrypted data pool).

@woffs
Copy link
Contributor Author

woffs commented Jul 6, 2018

@GrahamcOfBorg eval

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: zfs

Partial log (click to expand)

/nix/store/v77x3sriq5lz2r84dwz16bsxqcnch5rq-zfs-user-0.7.9

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: zfs

Partial log (click to expand)

copying path '/nix/store/xzp79br9qn8ag9ica7ckikqk65dv8lg6-systemd-237-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/0f24sb6s0j1528kiqi711wcqr2mv4b6f-libmicrohttpd-0.9.58' from 'https://cache.nixos.org'...
copying path '/nix/store/a13rjb77sz2p3jlfzvzh115h09zsjjsw-util-linux-2.31.1-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/gxq2lm3m8yjz8s3n1nfn51cr092dg28v-systemd-237' from 'https://cache.nixos.org'...
copying path '/nix/store/998hblgi8wp5vj3pvdz2mh52bdfk4943-util-linux-2.31.1-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/v9mpcjmw7idf7q8lx4wi20cdghm23rkn-lvm2-2.02.177' from 'https://cache.nixos.org'...
copying path '/nix/store/gy4agsa7jhmnx5xd17w77256rndfmvlw-nfs-utils-2.1.1' from 'https://cache.nixos.org'...
copying path '/nix/store/pplcs5xhnp41lpqris3dbmmhhvswxvys-zfs-user-0.7.9-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/a4dhavydiryk3v37f10xs25qh22vm963-zfs-user-0.7.9' from 'https://cache.nixos.org'...
/nix/store/a4dhavydiryk3v37f10xs25qh22vm963-zfs-user-0.7.9

@samueldr
Copy link
Member

samueldr commented Oct 6, 2018

Hi!

Closing since 18.03 is in maintenance mode, the relevant changes are part of 18.09.

Sorry this couldn't be taken care of earlier!

@samueldr samueldr closed this Oct 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@woffs @GrahamcOfBorg @Baughn @adisbladis @samueldr @Mic92