Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs improvements #28621

Merged
merged 6 commits into from
Aug 30, 2017
Merged

ipfs improvements #28621

merged 6 commits into from
Aug 30, 2017

Conversation

elitak
Copy link
Contributor

@elitak elitak commented Aug 27, 2017

Motivation for this change

Ipfs service needs more configurability.

Things done

See commit messages.

  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@mention-bot
Copy link

@elitak, thanks for your PR! By analyzing the history of the files in this pull request, we identified @kamilchm, @mguentner and @zimbatm to be potential reviewers.

@globin
Copy link
Member

globin commented Aug 27, 2017

cc @mguentner


buildGoPackage rec {
name = "ipfs-${version}";
version = "0.4.10";
rev = "4679f806bd00c0a5299c22c82d1fbfdbad928e6d";
version = "0.4.11-pre";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason to package an unreleased version? We generally try to use releases with patches to fix specific issues if needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no pressing need for any new features, so I'll remove this commit.


postFixup = ''
wrapProgram $bin/bin/ipfs \
--prefix PATH : ${fuse}/bin
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure that this will work? The fuse tools need SUID root to work IIRC which isn't possible in the Nix store.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true, but it's still better than "fuse not found" that will always be printed without this. I'll add a note in the autoMount option description that mentions the need to run the service as root for it to work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a good idea to run IPFS as root just because of the SUID problem.
Wouldn't it be better to use security.wrappers instead?

-> nixos/modules/security/wrappers/default.nix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably; I didn't know about those, thanks. I will try it out and update this PR today.

@elitak elitak force-pushed the ipfs branch 2 times, most recently from e0cad26 to fc6dd88 Compare August 27, 2017 23:15
@mguentner
Copy link
Contributor

mguentner commented Aug 28, 2017

LGTM, besides the issue with fuse that @fpletz mentioned

We already have quite some repetition in the service module. This should be replaced with one abstract ipfs service from which the concrete services derive their character (systemd.services.ipfs-offline systemd.services.ipfs-norouting systemd.services.ipfs etc.)

Edit: ⏫ at some point, not necessarily in this PR

@elitak
Copy link
Contributor Author

elitak commented Aug 28, 2017

The service is now execing /run/wrappers/bin/fusermount (which has the setuid root bit set), but it's still having some kind of privilege error. I'm having a real hard time figuring out what it is. Can someone have a look? I refactored as suggested.

mkdir -p $(${ipfs}/bin/ipfs --local config Mounts.IPNS)
ipfs --local config Mounts.FuseAllowOther --json true
mkdir -p $(ipfs --local config Mounts.IPFS)
mkdir -p $(ipfs --local config Mounts.IPNS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in preStart as normal users like ipfs are not allowed to create the directories /ipfs /ipns.
However ipfs is not initialized in preStart, so $(ipfs --local config Mounts.IPFS) is not possible. Maybe make the mount points configurable in nix and then use cfg.ipfsMountPoint and set pass that to ipfs accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Come to think of it, that stuff shouldn't be part of the -init service anyway. I'll move it to the preStart of the other 3 daemon services.

@mguentner
Copy link
Contributor

Please see #28746 for an updated IPFS test. The current test is mostly useless \o/

Remove the # to test the autoMount feature: https://github.com/NixOS/nixpkgs/pull/28746/files#diff-5b094f3f9b39431503bfe48522a2adfdR27

please also check out this updated patch. However fusermount within ipfs still fails.

diff --git a/nixos/modules/services/network-filesystems/ipfs.nix b/nixos/modules/services/network-filesystems/ipfs.nix
index 34bb6770ef..31aa7d475a 100644
--- a/nixos/modules/services/network-filesystems/ipfs.nix
+++ b/nixos/modules/services/network-filesystems/ipfs.nix
@@ -41,8 +41,8 @@ let
       ipfs --local config Addresses.Gateway ${cfg.gatewayAddress}
     '' + optionalString cfg.autoMount ''
       ipfs --local config Mounts.FuseAllowOther --json true
-      mkdir -p $(ipfs --local config Mounts.IPFS)
-      mkdir -p $(ipfs --local config Mounts.IPNS)
+      ipfs --local config Mounts.IPFS ${cfg.ipfsMountDir}
+      ipfs --local config Mounts.IPNS ${cfg.ipnsMountDir}
     '' + concatStringsSep "\n" (collect
           isString
           (mapAttrsRecursive
@@ -110,6 +110,18 @@ in {
         description = "Whether IPFS should try to mount /ipfs and /ipns at startup.";
       };
 
+      ipfsMountDir = mkOption {
+        type = types.str;
+        default = "/ipfs";
+        description = "Where to mount the IPFS namespace to";
+      };
+
+      ipnsMountDir = mkOption {
+        type = types.str;
+        default = "/ipns";
+        description = "Where to mount the IPNS namespace to";
+      };
+
       gatewayAddress = mkOption {
         type = types.str;
         default = "/ip4/127.0.0.1/tcp/8080";
@@ -203,6 +215,9 @@ in {
 
       preStart = ''
         install -m 0755 -o ${cfg.user} -g ${cfg.group} -d ${cfg.dataDir}
+      '' + optionalString cfg.autoMount ''
+        install -m 0755 -o ${cfg.user} -g ${cfg.group} -d ${cfg.ipfsMountDir}
+        install -m 0755 -o ${cfg.user} -g ${cfg.group} -d ${cfg.ipnsMountDir}
       '';
       script = ''
         if [[ ! -f ${cfg.dataDir}/config ]]; then

@elitak
Copy link
Contributor Author

elitak commented Aug 30, 2017

I realize the autoMount is still broken for non-root users. I've straced both invocations and can't figure out why, even though the setuid root wrapper is being used. I'm just going to remove it if nobody else wants to help me figure out how to get it to work. I'll incorporate the 2 mountpoint options you suggest but leave them commented out, for the time being.

@fpletz
Copy link
Member

fpletz commented Aug 30, 2017

We'll merge this in the current state to get it into 17.09. You can always PR fixes to enable the autoMount option. Thanks!

@fpletz fpletz merged commit a22e1d4 into NixOS:master Aug 30, 2017
@elitak
Copy link
Contributor Author

elitak commented Aug 30, 2017

Okay, thanks.

@mguentner, please mention me if you manage to figure out why the wrapped fusermount isn't working.

@mguentner
Copy link
Contributor

Sure thing. I tried it until I figured out that unless I actually start debugging IPFS, I won't find out.
Maybe insert some debug output into the go(o)? code^^

@mguentner
Copy link
Contributor

FYI, from the strace(1) man page:

Programs that use the setuid bit do not have effective user ID privileges while being traced.

@elitak
Copy link
Contributor Author

elitak commented Aug 31, 2017

Oh I hadn't thought of the setuid/strace interaction, but it makes sense.

The problem with adding debug code is that I have to redownload the entire set of dependencies for all of ipfs every time, because the way the build works is the hash is computed over the entire set (and the fuse mount code is a dep) and there's no opportunity during the build for patching, unless I'm overlooking that feature in gx. On top of that, for each change of a dep, I need to patch the code, then replace the old ipfs hash with the newly computed one. It's totally maladapted to Nix.

You can see what I mean here: elitak@04b0200

That sed command needs to be adjusted to find and replace the new hash of the patched dep each time I make a change. Am I missing an obviously better way?

Not only does this make debugging extremely tedious, but it's pretty important we have a simpler way to patch things build with gx downstream for compatibility purposes.

@elitak
Copy link
Contributor Author

elitak commented Aug 31, 2017

After writing all that, I realize I should just be debugging from a nix-shell environment, probably. Anyway, I'm just venting so as to explain why I likely won't be trying to fix the fuse mount permissions problem for a while.

@elitak
Copy link
Contributor Author

elitak commented Sep 16, 2017

Turns out I needed just 1 line in /etc/fuse.conf, which is not ideal, but probably clean enough for now. I'm still not sure why the setuid wrapper isn't working, since needing user_allow_other is evidence that fusermount is not running as root.

@mguentner can you try #29133 and the tests you wrote for it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants