Add spruned application and nixos module #50382
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Failure on x86_64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
|
Failure on aarch64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
|
Success on x86_64-darwin (full log) Attempted: python The following builds were skipped because they don't evaluate on x86_64-darwin: spruned Partial log (click to expand)
|
|
missed a few things, fixing |
ryantm
requested changes
Nov 15, 2018
infinisil
requested changes
Nov 15, 2018
|
|
||
| serviceConfig = { | ||
| Restart = "on-abort"; | ||
| ExecStart = "${pkgs.spruned}/bin/spruned ${cliArgs}"; |
There was a problem hiding this comment.
Like this, the service runs as root. Try using the DynamicUser and StateDirectory systemd settings (man systemd.exec) to prevent that.
There was a problem hiding this comment.
so I tried running it as non-root by first using user services and then assigned users, but spruned calls out to ping and doesn't work without it. I couldn't figure out how to make a systemd service that uses ping and isn't root. I looked into the CapabilityBoundingSet CAP_NET_RAW thing but it didn't seem to work.
There was a problem hiding this comment.
Use /run/wrappers/bin/ping instead of the ping in iputils. And really try to use DynamicUser, I think it should work.
There was a problem hiding this comment.
ok I'll try to take another pass at this and patch spruned to use /run/wrappers/bin/ping and switch to DynamicUser. Why DynamicUser instead of a user service?
There was a problem hiding this comment.
DynamicUser lets you can run an un-elevated service without assigning it a system user id.
There was a problem hiding this comment.
I'm getting this when I try to run with DynamicUser:
Nov 17 12:42:17 monad systemd[1]: Started spruned service.
Nov 17 12:42:17 monad spruned[17804]: Failed to import the site module
Nov 17 12:42:17 monad spruned[17804]: Traceback (most recent call last):
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/site.py", line 544, in <module>
Nov 17 12:42:17 monad spruned[17804]: main()
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/site.py", line 530, in main
Nov 17 12:42:17 monad spruned[17804]: known_paths = addusersitepackages(known_paths)
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/site.py", line 282, in addusersitepackages
Nov 17 12:42:17 monad spruned[17804]: user_site = getusersitepackages()
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/site.py", line 258, in getusersitepackages
Nov 17 12:42:17 monad spruned[17804]: user_base = getuserbase() # this will also set USER_BASE
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/site.py", line 248, in getuserbase
Nov 17 12:42:17 monad spruned[17804]: USER_BASE = get_config_var('userbase')
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/sysconfig.py", line 601, in get_config_var
Nov 17 12:42:17 monad spruned[17804]: return get_config_vars().get(name)
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/sysconfig.py", line 558, in get_config_vars
Nov 17 12:42:17 monad spruned[17804]: _CONFIG_VARS['userbase'] = _getuserbase()
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/sysconfig.py", line 205, in _getuserbase
Nov 17 12:42:17 monad spruned[17804]: return joinuser("~", ".local")
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/sysconfig.py", line 184, in joinuser
Nov 17 12:42:17 monad spruned[17804]: return os.path.expanduser(os.path.join(*args))
Nov 17 12:42:17 monad spruned[17804]: File "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/lib/python3.6/posixpath.py", line 249, in expanduser
Nov 17 12:42:17 monad spruned[17804]: userhome = pwd.getpwuid(os.getuid()).pw_dir
Nov 17 12:42:17 monad spruned[17804]: KeyError: 'getpwuid(): uid not found: 61833'
Nov 17 12:42:17 monad systemd[1]: spruned.service: Main process exited, code=exited, status=1/FAILURE
Nov 17 12:42:17 monad systemd[1]: spruned.service: Failed with result 'exit-code'.
|
range-diff v1 -> v2 1: f4f6217cfb1 = 1: f4f6217cfb1 altcoins: remove haskellPackages from input arguments
2: 31b93d6b70e = 2: 31b93d6b70e python: pycoin: init at 0.80
3: c6018ab3f51 = 3: c6018ab3f51 python: apply-default: init at 0.1.1
4: f8b62675675 = 4: f8b62675675 python: jsonrpcserver: init at 3.5.6
5: a769c143ff6 ! 5: 9232550f85b spruned: init at 0.0.4b4
@@ -14,7 +14,7 @@
bitcoin = libsForQt5.callPackage ./bitcoin.nix { miniupnpc = miniupnpc_2; withGui = true; };
bitcoind = callPackage ./bitcoin.nix { miniupnpc = miniupnpc_2; withGui = false; };
clightning = callPackage ./clightning.nix { };
-+ spruned = callPackage ./spruned { pythonPackages = python3.pkgs; };
++ spruned = callPackage ./spruned { };
bitcoin-abc = libsForQt5.callPackage ./bitcoin-abc.nix { boost = boost165; withGui = true; };
bitcoind-abc = callPackage ./bitcoin-abc.nix { boost = boost165; withGui = false; };
@@ -24,12 +24,14 @@
--- /dev/null
+++ b/pkgs/applications/altcoins/spruned/default.nix
@@
-+{ stdenv, pythonPackages }:
++{ stdenv, python3 }:
+
++let
++ pythonPackages = python3.pkgs;
++in
+with stdenv.lib;
+pythonPackages.buildPythonApplication rec {
+ pname = "spruned";
-+ name = "${pname}-${version}";
+ version = "0.0.4b4";
+
+ src = pythonPackages.fetchPypi {
@@ -58,7 +60,8 @@
+ --replace 'aiohttp==3.0.0b0' aiohttp \
+ --replace 'daemonize==2.4.7' daemonize \
+ --replace 'async-timeout==2.0.1' async-timeout \
-+ --replace 'jsonrpcserver==3.5.3' jsonrpcserver
++ --replace 'jsonrpcserver==3.5.3' jsonrpcserver \
++ --replace 'plyvel==0.9.0' plyvel
+ '';
+
+ doCheck = false;
6: 4cba35865e1 ! 6: 3ff5cc8aa37 nixos/spruned: init module
@@ -37,16 +37,10 @@
+{
+ options = {
+ services.spruned = {
-+ enable = mkOption {
-+ type = types.bool;
-+ default = false;
-+ description = ''
-+ Whether or not to enable the spruned lightweight Bitcoin pseudonode
-+ daemon service.
-+ '';
-+ };
++ enable = mkEnableOption "The spruned lightweight Bitcoin pseudonode daemon service";
+ dataDir = mkOption {
-+ type = types.string;
++ type = types.str;
++ default = "/var/lib/spruned";
+ description = ''
+ Directory to store cached block data and spruned logs.
+ '';
@@ -59,7 +53,7 @@
+ '';
+ };
+ extraArguments = mkOption {
-+ type = types.string;
++ type = types.separatedString " ";
+ example = "--mempoolsize 20 --debug";
+ default = "";
+ description = '' |
|
Success on aarch64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
ryantm
reviewed
Nov 15, 2018
| { | ||
| options = { | ||
| services.spruned = { | ||
| enable = mkEnableOption "The spruned lightweight Bitcoin pseudonode daemon service"; |
There was a problem hiding this comment.
You might be sad that you used a capital "T" here.
description = "Whether to enable ${name}."; https://github.com/NixOS/nixpkgs/blob/master/lib/options.nix#L68
ryantm
approved these changes
Nov 15, 2018
|
Success on x86_64-darwin (full log) Attempted: python The following builds were skipped because they don't evaluate on x86_64-darwin: spruned Partial log (click to expand)
|
infinisil
requested changes
Nov 16, 2018
|
|
||
| serviceConfig = { | ||
| Restart = "on-abort"; | ||
| ExecStart = "${pkgs.spruned}/bin/spruned ${cliArgs}"; |
There was a problem hiding this comment.
Use /run/wrappers/bin/ping instead of the ping in iputils. And really try to use DynamicUser, I think it should work.
It doesn't seem to be used Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This is needed by jsonrpcserver Signed-off-by: William Casarin <jb55@jb55.com>
There is a newer version (4.0.0), but this one is required for spruned. Signed-off-by: William Casarin <jb55@jb55.com>
spruned is a lightweight Bitcoin pseudonode designed to fetch any block or transaction. Signed-off-by: William Casarin <jb55@jb55.com>
This adds a nixos module for spruned, the lightweight Bitcoin pseudonode Needs root because of ping. Signed-off-by: William Casarin <jb55@jb55.com>
|
DynamicUser didn't seem to work, but it seems to work ok as a user service. v2 -> v3: 1: f4f6217cfb1 = 1: 8fdb7dbb5a7 altcoins: remove haskellPackages from input arguments
2: 31b93d6b70e = 2: e664d19caff python: pycoin: init at 0.80
3: c6018ab3f51 = 3: 03ee97f0d43 python: apply-default: init at 0.1.1
4: f8b62675675 = 4: 7715ba1d5c9 python: jsonrpcserver: init at 3.5.6
5: 9232550f85b ! 5: e34041c3b76 spruned: init at 0.0.4b4
@@ -55,6 +55,9 @@
+ patchPhase = ''
+ sed -i 's,import spruned,,;s,spruned.__version__,"${version}",' setup.py
+
++ substituteInPlace spruned/application/tools.py \
++ --replace "subprocess.call(['ping" "subprocess.call(['/run/wrappers/bin/ping"
++
+ substituteInPlace requirements.txt \
+ --replace 'sqlalchemy==1.2.6' 'sqlalchemy==1.2.*' \
+ --replace 'aiohttp==3.0.0b0' aiohttp \
6: 3ff5cc8aa37 ! 6: 8311ab29521 nixos/spruned: init module
@@ -64,14 +64,11 @@
+ };
+
+ config = mkIf cfg.enable {
-+ systemd.services.spruned = {
++ systemd.user.services.spruned = {
+ description = "spruned service";
+ after = [ "local-fs.target" "network.target" ];
+ wantedBy = [ "multi-user.target" ];
+
-+ # needs ping
-+ path = [ pkgs.iputils ];
-+
+ serviceConfig = {
+ Restart = "on-abort";
+ ExecStart = "${pkgs.spruned}/bin/spruned ${cliArgs}"; |
|
Success on x86_64-darwin (full log) Attempted: python The following builds were skipped because they don't evaluate on x86_64-darwin: spruned Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: python, spruned Partial log (click to expand)
|
|
Closing this for now until I have time to fix some issues with it |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
spruned is a lightweight Bitcoin pseudonode with RPC that can fetch any block or transaction
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)nix path-info -Sbefore and after)