Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zfs: fix zfs legacy mount #111375

Merged
merged 1 commit into from Feb 1, 2021
Merged

Conversation

Philipp-M
Copy link
Member

@Philipp-M Philipp-M commented Jan 31, 2021

Closes #110376

See related discussion

Things done

Apply the following patch from ZFS upstream: openzfs/zfs@03f036c

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Copy link
Member

@hmenke hmenke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried to build this but something is going wrong.

hash mismatch in fixed-output derivation '/nix/store/qg2803fwr0fvg6k2cdvb58vm4ggkh2f6-03f036cbccdd8699f5fe8540ef317595a35bceb8.patch.drv':
  specified: sha256-grrxQdmoUgpFpzYtSP7c2tLhQn+gmFwl06+VCqlHVyw=                                                 
     got:    sha256-wVe7tlUaTnIMP6ugBeG3LkaS0wTG/14OaFaRvUcZfco=

@Philipp-M
Copy link
Member Author

Philipp-M commented Jan 31, 2021

I just tried to build this but something is going wrong.

hash mismatch in fixed-output derivation '/nix/store/qg2803fwr0fvg6k2cdvb58vm4ggkh2f6-03f036cbccdd8699f5fe8540ef317595a35bceb8.patch.drv':
  specified: sha256-grrxQdmoUgpFpzYtSP7c2tLhQn+gmFwl06+VCqlHVyw=                                                 
     got:    sha256-wVe7tlUaTnIMP6ugBeG3LkaS0wTG/14OaFaRvUcZfco=

Hmm weird, just tried it again (nixos-rebuild), and it works for me...
I have also run

nix-prefetch-url https://github.com/openzfs/zfs/commit/03f036cbccdd8699f5fe8540ef317595a35bceb8.patch

which outputs this:

0b2p8ylhm5dgscjmr650gx1f3lnsvkz4hb9nlx2hllm8v50z3fl2
Could it be that the patch probably changes depending on where (physical location) this url is accessed?
I have already wondered, why I had to change the hash from the original issue, which seems to be different as well to this hash mismatch.
Anything I can try to reproduce this?

@hmenke
Copy link
Member

hmenke commented Jan 31, 2021

I simply ran

nix-shell -p nixpkgs-review --run "nixpkgs-review pr --no-shell --post-result 111375"

EDIT: I checked the store path and indeed, the patch header seems to be missing. Instead of a patch I only got a diff, so this might be a problem on GitHub's side. I'm deleting the store paths right now and then retry.

@hmenke
Copy link
Member

hmenke commented Jan 31, 2021

I checked out your branch and tried to build a single derivation but it still fails.

$ git log -1
commit 5794ba8b652b862f19b0662acb7dabe37c0535ef (HEAD -> fix-zfs-path-sanitizer-patch, refs/nixpkgs-review/1)
Author: Philipp Mildenberger <philipp.mildenberger@koeln.de>
Date:   Sun Jan 31 02:00:06 2021 +0100

    zfs: fix zfs legacy mount
    
    Closes #110376
$ nix-build -A zfs
these 2 derivations will be built:
  /nix/store/qg2803fwr0fvg6k2cdvb58vm4ggkh2f6-03f036cbccdd8699f5fe8540ef317595a35bceb8.patch.drv
  /nix/store/xnmc86mn26idv3gjhyc2gjn52vfn5hhh-zfs-user-2.0.1.drv
building '/nix/store/qg2803fwr0fvg6k2cdvb58vm4ggkh2f6-03f036cbccdd8699f5fe8540ef317595a35bceb8.patch.drv'...

trying https://github.com/openzfs/zfs/commit/03f036cbccdd8699f5fe8540ef317595a35bceb8.patch
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3594  100  3594    0     0  18151      0 --:--:-- --:--:-- --:--:-- 18060
error: --- Error --- nix-daemon
hash mismatch in fixed-output derivation '/nix/store/qg2803fwr0fvg6k2cdvb58vm4ggkh2f6-03f036cbccdd8699f5fe8540ef317595a35bceb8.patch.drv':
  specified: sha256-grrxQdmoUgpFpzYtSP7c2tLhQn+gmFwl06+VCqlHVyw=
     got:    sha256-wVe7tlUaTnIMP6ugBeG3LkaS0wTG/14OaFaRvUcZfco=
error: --- Error -------------------------------------------------------------------------------------- nix-build
1 dependencies of derivation '/nix/store/xnmc86mn26idv3gjhyc2gjn52vfn5hhh-zfs-user-2.0.1.drv' failed to build

Maybe somebody else can reproduce. Here is the output of nix-shell -p nix-info --run "nix-info -m":

  • system: "x86_64-linux"
  • host os: Linux 5.9.16, NixOS, 20.09.2787.6e7f25001fe (Nightingale)
  • multi-user?: yes
  • sandbox: yes
  • version: nix-env (Nix) 2.4pre20201201_5a6ddb3
  • channels(root): ""
  • nixpkgs: /etc/channels/nixos

@Philipp-M
Copy link
Member Author

Maybe it helps to compare the patches itself, this is the content of the url when I access it:
https://pastebin.com/4xSsBQQh

Is there a more reproducable way to access patches in github?

@jonringer
Copy link
Contributor

nix-prefetch-url https://github.com/openzfs/zfs/commit/03f036cbccdd8699f5fe8540ef317595a35bceb8.patch

which outputs this:

0b2p8ylhm5dgscjmr650gx1f3lnsvkz4hb9nlx2hllm8v50z3fl2
Could it be that the patch probably changes depending on where (physical location) this url is accessed?

no, this is because fetchpatch will do some normalization before saving it to the store path, thus it changes the sha.

I have already wondered, why I had to change the hash from the original issue, which seems to be different as well to this hash mismatch.
Anything I can try to reproduce this?

change the sha to something invalid, I usually just change something to an "a". and run the build again, the expression will then be evaluated, instead of referencing an existing store path.

nix-prefetch-url https://github.com/openzfs/zfs/commit/03f036cbccdd8699f5fe8540ef317595a35bceb8.patch

which outputs this:

0b2p8ylhm5dgscjmr650gx1f3lnsvkz4hb9nlx2hllm8v50z3fl2

nix-prefetch-url will add the contents to the nix store. which means that you will have a fixed-output-derivation that is different from what fetchpatch uses. That's why it works for you locally, as your nix-prefetch-url store contents is being referenced, but that was done locally so it can't be reproduced by @hmenke

@Philipp-M
Copy link
Member Author

Thanks, something new learned again.
I could now reproduce the same sha256sum.

@ofborg ofborg bot requested a review from hmenke January 31, 2021 16:12
@jonringer
Copy link
Contributor

there was an attempt to make the fixed-output-derivations a little more ergonomic in NixOS/rfcs#84 . However, once you make a content addressed derivation more than just it's content.... then you have issues when you want to have alternate sources, and a few other issues.

For now, the rule of thumb is just to make the sha invalid, and take what nix says it wanted. As there's also other issues like if the content is packed or unpack (fetchFromGithub will unpack the source) which will also not align with nix-prefetch-url.

@hmenke
Copy link
Member

hmenke commented Jan 31, 2021

Result of nixpkgs-review pr 111375 1

1 package marked as broken and skipped:
  • bareos
3 packages failed to build:
  • libvmi
  • linuxPackages-libre.zfs (linuxPackages-libre.zfsUnstable)
  • samba4Full
59 packages built:
  • ceph
  • ceph-client
  • collectd
  • collectd-data
  • docker-machine-kvm
  • docker-machine-kvm2
  • easysnap
  • gnome3.gnome-boxes
  • libceph
  • libguestfs
  • libguestfs-with-appliance
  • libvirt
  • libvirt-glib
  • libvirt_5_9_0
  • linuxPackages.zfs (linuxPackages.zfsUnstable)
  • linuxPackages_4_14.zfs (linuxPackages_4_14.zfsUnstable)
  • linuxPackages_4_19.zfs (linuxPackages_4_19.zfsUnstable)
  • linuxPackages_4_4.zfs (linuxPackages_4_4.zfsUnstable)
  • linuxPackages_4_9.zfs (linuxPackages_4_9.zfsUnstable)
  • linuxPackages_5_10.zfs (linuxPackages_5_10.zfsUnstable)
  • linuxPackages_hardened.zfs (linuxPackages_hardened.zfsUnstable)
  • linuxPackages_latest-libre.zfs (linuxPackages_latest-libre.zfsUnstable)
  • linuxPackages_latest_hardened.zfs (linuxPackages_latest_hardened.zfsUnstable)
  • linuxPackages_latest_xen_dom0.zfs (linuxPackages_latest_xen_dom0.zfsUnstable)
  • linuxPackages_latest_xen_dom0_hardened.zfs (linuxPackages_latest_xen_dom0_hardened.zfsUnstable)
  • linuxPackages_lqx.zfs (linuxPackages_lqx.zfsUnstable)
  • linuxPackages_testing_bcachefs.zfs (linuxPackages_testing_bcachefs.zfsUnstable)
  • linuxPackages_xen_dom0.zfs (linuxPackages_xen_dom0.zfsUnstable)
  • linuxPackages_xen_dom0_hardened.zfs (linuxPackages_xen_dom0_hardened.zfsUnstable)
  • linuxPackages_zen.zfs (linuxPackages_zen.zfsUnstable)
  • minikube
  • minishift
  • nixops
  • nixopsUnstable
  • nixops_1_6_1
  • ocamlPackages.ocaml_libvirt
  • perl530Packages.SysVirt
  • perl532Packages.SysVirt
  • python37Packages.guestfs
  • python37Packages.libvirt
  • python38Packages.guestfs
  • python38Packages.libvirt
  • python39Packages.guestfs
  • python39Packages.libvirt
  • qemu_full
  • rubyPackages.ruby-libvirt
  • rubyPackages_2_5.ruby-libvirt
  • rubyPackages_2_7.ruby-libvirt
  • sanoid
  • terraform-full
  • terraform-provider-libvirt (terraform-providers.libvirt)
  • vagrant
  • virtmanager (virt-manager)
  • virtmanager-qt (virt-manager-qt)
  • virt-top
  • virtviewer (virt-viewer)
  • virtlyst
  • zfs (zfsUnstable)
  • zfstools

@Mic92 Mic92 merged commit 9b935b0 into NixOS:master Feb 1, 2021
@Mic92
Copy link
Member

Mic92 commented Feb 1, 2021

do we need a backport?

@hmenke
Copy link
Member

hmenke commented Feb 1, 2021

@Mic92 Yes, #111529

@Philipp-M Philipp-M deleted the fix-zfs-path-sanitizer-patch branch February 2, 2021 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ZFS legacy mounting not working after recent version update (zfs: 2.0.0 -> 2.0.1)
6 participants