Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Second attempt at adding PeerTube package #106492

Closed
wants to merge 2 commits into from

Conversation

stevenroose
Copy link
Contributor

Closes #46987.

So there was an attempt at packaging PeerTube here: #84189. I squashed that attempt into a single commit, just in order to preserve its content.

I'm totally new to making Nix packages (been using NixOS as a user for a while), so I've been trying to learn things over the past weekend. It seems that #84189 is hand-rolling the building of PeerTube as a Yarn package instead of using the yarn tooling like mkYarnPackage.

However, it seems that the yarn tooling is unable to build the yarn package anyway.

I'm looking into what's causing this by making modifications to the yarn tooling.

@expipiplus1
Copy link
Contributor

Thank you for this effort, looking forward to it!

@stevenroose
Copy link
Contributor Author

@expipiplus1 I think I officially gave up. The yarn tooling here in Nix is really bad. (Well, I'm not saying yarn itself or any JS dev practices are good to start with.) PeerTube has (1) a "resolutions" section in their package.json using which they force some deps to use subdependencies not compliant with their semver version (aweful practice I didn't know existed..) and (2) has some special dependencies that need bindings or some external sources that the yarn2nix tooling also can't handle and (3) some yarn dependency URL is also not working with yarn2nix.

(3) is easy to fix by manually patching the yarn.lock and yarn.nix; (1) I tried to fix by also manually patching the yarn.lock file but I that ultimately resulted in breakage because of wrong dep versions and (2) is something that @immae fixed in his version of this package, so it's doable but it's annoying.

I think ultimately the problem is the yarn2nix tooling being really bad. (It has things like big JS scripts to do a simple sed and manual URL pattern matching and only supports very default yarn workflows. I spent a few evenings trying to see if I could get the yarn2nix tooling to work with the PeerTube setup, but I'm not a JS dev and I don't really understand the yarn workflows.)

I might in the future take a second look but using a simple docker container to run the JS stuff. It's very sad, but it's probably better. I very welcome anyone with yarn knowledge or from the yarn2nix project (@moretea ?) to take a look or give it a shot.

@expipiplus1
Copy link
Contributor

but I'm not a JS dev and I don't really understand the yarn workflows.)

I feel this :)

@stevenroose stevenroose changed the title WIP: Second attempt at adding PeerTube package (mkYarnPackage fails) Second attempt at adding PeerTube package Jan 31, 2021
@stevenroose
Copy link
Contributor Author

K, I have a working instance using this code. Many thanks to @immae to actually do most of the work. I think this is ready for review. Not sure how to flag that or so..

Copy link
Contributor

@matthiasbeyer matthiasbeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote some small patches, I'll submit them shortly to @stevenroose - all of them are debatable of course.

I approve, save the split of the maintainer adding into a seperate commit!

Awesome work! Thanks a lot!


Ninja edit: stevenroose#1 adds some style fixes.

maintainers/maintainer-list.nix Show resolved Hide resolved
@stevenroose
Copy link
Contributor Author

I think I might have some more questions at first. Mostly this one:

The peertube module forces the version of postgres to be 12. This means that if anothre module also enables postgres but without version and peertube gets disabled, the postgres version will change from 12 to the default and the data will be invalid. (I had this happen during testing because master is on postgres 13.)

That doesn't seem to be ideal. Should I have a postgresqlPackage option? Or should I just not set the version and leave that to the user?

EDIT: I actually just realized the documentation doesn't mention any version. So I will just remove it and leave that to the user. I'll add a line of documentation as a suggestion somewhere.

@stevenroose
Copy link
Contributor Author

I also applied Matthias' style suggestions and rebased.

Copy link
Contributor

@immae immae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ofborg asked me to review, so I do approve. But note that some elements of this code were written by myself so it’s not exactly fair to approve :)

@@ -649,6 +650,7 @@ in
#mailman = 316; # removed 2019-08-30
zigbee2mqtt = 317;
shadow = 318;
peertube = 319;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't we used DynamicUser instead (combined with systemd's StateDirectory)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I don't know what DynamicUser is in this case, how would that work? One thing that I know I have to do is use the tmpfiles.rules to set the datadirs of the project as being owned by the peertube user. (The actual datadir is only set in the config file, not using the module.) So I think it makes sense to have a username here that one could access using a variable in order to set certain permissions. (As long as we don't auto-generate the entire PeerTube config file from within the module. But it's really complete, I would not advise that.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DynamicUser can be used to dynamically allocate an ephemeral user/group when the service starts. That is usually possible if the users data does not need to be accessed by other users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in this case it would be hard, because the user needs to externally set the permission of some directories to be owned accessible by that user. I don't intend to do the full PeerTube configuration in the nixos module.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why systemd provides

RuntimeDirectory=, StateDirectory=, CacheDirectory=, LogsDirectory= or ConfigurationDirectory=

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But those directories are only used in the PeerTube's internal config file, NixOS never knows about them at the module level.

nixos/modules/services/web-apps/peertube.nix Outdated Show resolved Hide resolved
nixos/modules/services/web-apps/peertube.nix Outdated Show resolved Hide resolved
@stevenroose
Copy link
Contributor Author

@adisbladis I added a commit with some of your review addressed. I also added postfix. I tried this on my running staging instance. I can squash after review.

After I get most review, I'm gonna try get my production deployment in place using this.

@stevenroose
Copy link
Contributor Author

Also, I don't understand why the grahamcofborg CI job is failing. The error is quite cryptic and doesn't mention a line number of any of the files changed in this PR...

@stevenroose
Copy link
Contributor Author

Can someone point me to what is needed for this to be able to get merged. I don't understand the missing CI.

Would it help if I split this up into two separate PRs: pkg and module?

Copy link
Contributor

@mohe2015 mohe2015 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than that well done! I used this in a testing environment and youtube-dl integration and even bidirectional federation with a mastodon instance worked out of the box!

Edit: I don't think splitting this up would help as it's quite hard to get anything new and big into NixOS right now (not enough reviews and mergers)


environment.NODE_CONFIG_DIR = "${cfg.runtimeDir}/config";
environment.NODE_ENV = "production";
environment.HOME = cfg.package;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
environment.HOME = cfg.package;
environment.HOME = cfg.package;
environment.NODE_EXTRA_CA_CERTS = "/etc/ssl/certs/ca-certificates.crt";

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an proposal but this make it work for me out of the box with self-signed certificates added to my local trust store.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, where does that magic string "/etc/ssl/certs/ca-certificates.crt" come from? Could we perhaps add this conditional on some nixos variable? Or is "/etc/ssl/certs/ca-certificates.crt" the standard location always for certs in NixOS?

Copy link
Contributor

@mohe2015 mohe2015 Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/ca.nix#L85 it's called the canonical location for NixOS so should be fine.

Copy link
Contributor

@mohe2015 mohe2015 Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also many other packages/modules reference it or the old /etc/ssl/certs/ca-bundle.crt one.

home = cfg.runtimeDir;
useDefaultShell = true;
# todo: fix this. needed for postgres authentication
password = "peertube";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need to be fixed I think. Does peertube not support unix socket authentication for postgresql as I think this is the approach other services take?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, it might do, but I didn't work that out yet. Yeah that's why I suggested maybe just getting the pkg in might be easier, since modules are always a bit more opinionated..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I will look into this later.

Copy link
Contributor

@mohe2015 mohe2015 Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevenroose see stevenroose#2. Feel free to squash the commits after merging. You can keep me as co-author if you like.

@Izorkin
Copy link
Contributor

Izorkin commented Apr 5, 2021

@stevenroose small update to service configuration:

From 6c64e41ccf3856dd4639711c0f7c0c9ae9746572 Mon Sep 17 00:00:00 2001
From: Izorkin <izorkin@elven.pw>
Date: Mon, 5 Apr 2021 15:03:48 +0300
Subject: [PATCH] nixos/peertube: update service configuration

---
 nixos/modules/misc/ids.nix                   |   2 -
 nixos/modules/services/web-apps/peertube.nix | 128 ++++++++-----------
 2 files changed, 56 insertions(+), 74 deletions(-)

diff --git a/nixos/modules/misc/ids.nix b/nixos/modules/misc/ids.nix
index 443f2e1e168..1fd56adfe10 100644
--- a/nixos/modules/misc/ids.nix
+++ b/nixos/modules/misc/ids.nix
@@ -347,7 +347,6 @@ in
       #mailman = 316;  # removed 2019-08-30
       zigbee2mqtt = 317;
       # shadow = 318; # unused
-      peertube = 319;
 
       # When adding a uid, make sure it doesn't match an existing gid. And don't use uids above 399!
 
@@ -650,7 +649,6 @@ in
       #mailman = 316;  # removed 2019-08-30
       zigbee2mqtt = 317;
       shadow = 318;
-      peertube = 319;
 
       # When adding a gid, make sure it doesn't match an existing
       # uid. Users and groups with the same name should have equal
diff --git a/nixos/modules/services/web-apps/peertube.nix b/nixos/modules/services/web-apps/peertube.nix
index 38c3f21a549..9db927687cc 100644
--- a/nixos/modules/services/web-apps/peertube.nix
+++ b/nixos/modules/services/web-apps/peertube.nix
@@ -1,25 +1,21 @@
 { lib, pkgs, config, ... }:
 
 let
-  name = "peertube";
   cfg = config.services.peertube;
 
-  uid = config.ids.uids.peertube;
-  gid = config.ids.gids.peertube;
-in
-{
+in {
   options.services.peertube = {
     enable = lib.mkEnableOption "Enable Peertube’s service";
 
     user = lib.mkOption {
       type = lib.types.str;
-      default = name;
+      default = "peertube";
       description = "User account under which Peertube runs";
     };
 
     group = lib.mkOption {
       type = lib.types.str;
-      default = name;
+      default = "peertube";
       description = "Group under which Peertube runs";
     };
 
@@ -68,7 +64,7 @@ in
 
     runtimeDir = lib.mkOption {
       type = lib.types.path;
-      default = "/var/lib/${name}";
+      default = "/var/lib/peertube";
       description = "The directory where Peertube stores its runtime data.";
     };
 
@@ -82,44 +78,10 @@ in
   };
 
   config = lib.mkIf cfg.enable {
-    users.users = lib.optionalAttrs (cfg.user == name) {
-      "${name}" = {
-        inherit uid;
-        group = cfg.group;
-        description = "Peertube user";
-        home = cfg.runtimeDir;
-        useDefaultShell = true;
-        # todo: fix this. needed for postgres authentication
-        password = "peertube";
-      };
-    };
-    users.groups = lib.optionalAttrs (cfg.group == name) {
-      "${name}" = {
-        inherit gid;
-      };
-    };
-
-    services.postgresql = lib.mkIf cfg.database.createLocally {
-      enable = true;
-      ensureUsers = [ { name = cfg.database.user; }];
-      # The database is created as a startup script of the peertube service.
-      authentication = ''
-        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 trust
-        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 md5
-      '';
-    };
-
-    services.postfix = lib.mkIf cfg.smtp.createLocally {
-      enable = true;
-    };
-
-    services.redis = lib.mkIf cfg.redis.createLocally {
-      enable = true;
-    };
-
     # Make sure the runtimeDir exists with the desired permissions.
     systemd.tmpfiles.rules = [
-      "d \"${cfg.runtimeDir}\" - ${cfg.user} ${cfg.group} - -"
+      "d '${cfg.runtimeDir}' 0750 ${cfg.user} ${cfg.group} - -"
+      "d '/var/www/peertube' 0750 ${cfg.user} ${cfg.group} - -"
     ];
 
     systemd.services.peertube = {
@@ -141,46 +103,68 @@ in
       '';
 
       serviceConfig = {
+        Type = "simple";
+        ExecStartPre = let script = pkgs.writeScript "peertube-pre-start.sh" ''
+          #!/bin/sh
+          sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;" ${cfg.database.name}
+          sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION IF NOT EXISTS unaccent;" ${cfg.database.name}
+        '';
+        in "+${script}";
+        Restart = "always";
+        RestartSec = 20;
+        TimeoutSec = 60;
+        WorkingDirectory = cfg.package;
+        # User and group
         User = cfg.user;
         Group = cfg.group;
-        WorkingDirectory = cfg.package;
+        # State directory and mode
         StateDirectory = "peertube";
         StateDirectoryMode = "0750";
-        PrivateTmp = true;
+        # Capabilities
+        CapabilityBoundingSet = "~CAP_SYS_ADMIN";
+        # Sandboxing
         ProtectHome = true;
-        ProtectControlGroups = true;
         ProtectSystem = "full";
-        Restart = "always";
-        Type = "simple";
-        TimeoutSec = 60;
-        CapabilityBoundingSet = "~CAP_SYS_ADMIN";
-        ExecStartPre = let script = pkgs.writeScript "peertube-pre-start.sh" ''
-          #!/bin/sh
-          set -e
+        PrivateTmp = true;
+        ProtectControlGroups = true;
+      };
 
-          if ! [ -e "${cfg.runtimeDir}/.first_run" ]; then
-            set -v
-            if [ -e "${cfg.runtimeDir}/.first_run_partial" ]; then
-              echo "Warn: first run was interrupted"
-            fi
-            touch "${cfg.runtimeDir}/.first_run_partial"
+      unitConfig.RequiresMountsFor = cfg.runtimeDir;
+    };
 
-            echo "Running PeerTube's PostgreSQL initialization..."
-            echo "PeerTube is known to work with PostgreSQL v12, if any error occurs, please check your version."
+    services.postfix = lib.mkIf cfg.smtp.createLocally {
+      enable = true;
+    };
 
-            sudo -u postgres "${config.services.postgresql.package}/bin/createdb" -O ${cfg.database.user} -E UTF8 -T template0 ${cfg.database.name}
-            sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION pg_trgm;" ${cfg.database.name}
-            sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION unaccent;" ${cfg.database.name}
+    services.redis = lib.mkIf cfg.redis.createLocally {
+      enable = true;
+    };
 
-            touch "${cfg.runtimeDir}/.first_run"
-            rm "${cfg.runtimeDir}/.first_run_partial"
-          fi
-        '';
-        in "+${script}";
+    services.postgresql = lib.mkIf cfg.database.createLocally {
+      enable = true;
+      ensureUsers = [{
+        name = cfg.database.user;
+        ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; };
+      }];
+      ensureDatabases = [ cfg.database.name ];
+      authentication = ''
+        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 trust
+        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 md5
+      '';
+    };
+
+    users.users = lib.optionalAttrs (cfg.user == "peertube") {
+     peertube  = {
+        isSystemUser = true;
+        home = cfg.runtimeDir;
+        group = cfg.group;
+        # todo: fix this. needed for postgres authentication
+        password = "peertube";
       };
+    };
 
-      unitConfig.RequiresMountsFor = cfg.runtimeDir;
+    users.groups = lib.optionalAttrs (cfg.group == "peertube") {
+      peertube = { };
     };
   };
 }
-
-- 
2.31.1

@Izorkin
Copy link
Contributor

Izorkin commented Apr 5, 2021

Peertube connects via unix socket.
Now using 2 config files - config/default.yaml + 'config/production.yaml' https://docs.joinpeertube.org/install-any-os?id=peertube-configuration
Temp patch:

diff --git a/nixos/modules/services/web-apps/peertube.nix b/nixos/modules/services/web-apps/peertube.nix
index 9db927687cc..da7592b5c32 100644
--- a/nixos/modules/services/web-apps/peertube.nix
+++ b/nixos/modules/services/web-apps/peertube.nix
@@ -3,6 +3,51 @@
 let
   cfg = config.services.peertube;

+  settingsFormat = pkgs.formats.yaml {};
+  configFile = pkgs.writeText  "production.yaml" ''
+    listen:
+      hostname: 'localhost'
+      port: 9000
+
+    webserver:
+      https: true
+      hostname: '${cfg.hostname}'
+      port: 443
+
+    database:
+      hostname: '/run/postgresql'
+      port: 5432
+      ssl: false
+      suffix: '_prod'
+      username: 'peertube'
+      password: 'peertube'
+      pool:
+        max: 5
+
+    redis:
+      hostname: 'localhost'
+      port: 6379
+      auth: null
+      db: 0
+
+    storage:
+      tmp: '${cfg.runtimeDir}/storage/tmp/'
+      avatars: '${cfg.runtimeDir}/storage/avatars/'
+      videos: '${cfg.runtimeDir}/storage/videos/'
+      streaming_playlists: '${cfg.runtimeDir}/storage/streaming-playlists/'
+      redundancy: '${cfg.runtimeDir}/storage/redundancy/'
+      logs: '${cfg.runtimeDir}/storage/logs/'
+      previews: '${cfg.runtimeDir}/storage/previews/'
+      thumbnails: '${cfg.runtimeDir}/storage/thumbnails/'
+      torrents: '${cfg.runtimeDir}/storage/torrents/'
+      captions: '${cfg.runtimeDir}/storage/captions/'
+      cache: '${cfg.runtimeDir}/storage/cache/'
+      plugins: '${cfg.runtimeDir}/storage/plugins/'
+      client_overrides: '${cfg.runtimeDir}/storage/client-overrides/'
+
+    ${cfg.extraConfig}
+  '';
+
 in {
   options.services.peertube = {
     enable = lib.mkEnableOption "Enable Peertube’s service";
@@ -19,11 +64,14 @@ in {
       description = "Group under which Peertube runs";
     };

-    configFile = lib.mkOption {
-      type = lib.types.path;
-      description = ''
-        The configuration file path for Peertube.
-      '';
+    hostname = lib.mkOption {
+      type = lib.types.str;
+      default = "example.com";
+    };
+
+    extraConfig = lib.mkOption {
+      type = lib.types.lines;
+      default = "";
     };

     database = {
@@ -81,7 +129,7 @@ in {
     # Make sure the runtimeDir exists with the desired permissions.
     systemd.tmpfiles.rules = [
       "d '${cfg.runtimeDir}' 0750 ${cfg.user} ${cfg.group} - -"
-      "d '/var/www/peertube' 0750 ${cfg.user} ${cfg.group} - -"
+      "d '${cfg.runtimeDir}/storage' 0750 ${cfg.user} ${cfg.group} - -"
     ];

     systemd.services.peertube = {
@@ -98,7 +146,8 @@ in {

       script = ''
         install -m 0750 -d ${cfg.runtimeDir}/config
-        ln -sf ${cfg.configFile} ${cfg.runtimeDir}/config/production.yaml
+        ln -sf ${cfg.package}/config/default.yaml ${cfg.runtimeDir}/config/default.yaml
+        ln -sf ${configFile} ${cfg.runtimeDir}/config/production.yaml
         exec npm start
       '';

@@ -158,8 +207,6 @@ in {
         isSystemUser = true;
         home = cfg.runtimeDir;
         group = cfg.group;
-        # todo: fix this. needed for postgres authentication
-        password = "peertube";
       };
     };

@mohe2015
Copy link
Contributor

mohe2015 commented Apr 5, 2021

@Izorkin stevenroose#2 contains some of these changes. Also please use
```diff
for highlighting

@Izorkin
Copy link
Contributor

Izorkin commented Apr 5, 2021

@mohe2015 Can you make my changes to your PR?
And don't need delete database.user. This user is required to work with external PostgreSQL.

@Izorkin
Copy link
Contributor

Izorkin commented Apr 5, 2021

Add variant to work with external PostgreSQL

diff --git a/nixos/modules/services/web-apps/peertube.nix b/nixos/modules/services/web-apps/peertube.nix
index da7592b5c32..00547758f3c 100644
--- a/nixos/modules/services/web-apps/peertube.nix
+++ b/nixos/modules/services/web-apps/peertube.nix
@@ -2,6 +2,8 @@
 
 let
   cfg = config.services.peertube;
+  # We only want to create a database if we're actually going to connect to it.
+  databaseActuallyCreateLocally = cfg.database.createLocally && cfg.database.host == "/run/postgresql";
 
   settingsFormat = pkgs.formats.yaml {};
   configFile = pkgs.writeText  "production.yaml" ''
@@ -15,14 +17,10 @@ let
       port: 443
 
     database:
-      hostname: '/run/postgresql'
-      port: 5432
-      ssl: false
-      suffix: '_prod'
-      username: 'peertube'
-      password: 'peertube'
-      pool:
-        max: 5
+      hostname: '${cfg.database.host}'
+      port: '${toString cfg.database.port}'
+      name: '${cfg.database.name}'
+      username: '${cfg.database.user}'
 
     redis:
       hostname: 'localhost'
@@ -81,9 +79,22 @@ in {
         default = true;
       };
 
+      host = lib.mkOption {
+        type = lib.types.str;
+        default = "/run/postgresql";
+        example = "192.168.15.47";
+        description = "Database host address or unix socket.";
+      };
+
+      port = lib.mkOption {
+        type = lib.types.int;
+        default = 5432;
+        description = "Database host port.";
+      };
+
       name = lib.mkOption {
         type = lib.types.str;
-        default = "peertube_prod";
+        default = "peertube";
         description = "Database name.";
       };
 
@@ -92,6 +103,16 @@ in {
         default = "peertube";
         description = "Database user.";
       };
+
+      passwordFile = lib.mkOption {
+        type = lib.types.nullOr lib.types.path;
+        default = null;
+        example = "/run/keys/peertube-db-password";
+        description = ''
+          A file containing the password corresponding to
+          <option>database.user</option>.
+        '';
+      };
     };
 
     smtp = {
@@ -129,14 +150,17 @@ in {
     # Make sure the runtimeDir exists with the desired permissions.
     systemd.tmpfiles.rules = [
       "d '${cfg.runtimeDir}' 0750 ${cfg.user} ${cfg.group} - -"
+      "d '${cfg.runtimeDir}/config' 0700 ${cfg.user} ${cfg.group} - -"
       "d '${cfg.runtimeDir}/storage' 0750 ${cfg.user} ${cfg.group} - -"
     ];
 
     systemd.services.peertube = {
       description = "Peertube";
       wantedBy = [ "multi-user.target" ];
-      after = [ "network.target" "postgresql.service" "redis.service" ];
-      wants = [ "postgresql.service" "redis.service" ];
+      after = [ "network.target" "redis.service" ]
+        ++ (if databaseActuallyCreateLocally then [ "postgresql.service" ] else []);
+      wants = [ "redis.service" ]
+        ++ (if databaseActuallyCreateLocally then [ "postgresql.service" ] else []);
 
       environment.NODE_CONFIG_DIR = "${cfg.runtimeDir}/config";
       environment.NODE_ENV = "production";
@@ -144,21 +168,16 @@ in {
 
       path = [ pkgs.nodejs pkgs.bashInteractive pkgs.ffmpeg pkgs.openssl pkgs.sudo pkgs.youtube-dl ];
 
-      script = ''
-        install -m 0750 -d ${cfg.runtimeDir}/config
-        ln -sf ${cfg.package}/config/default.yaml ${cfg.runtimeDir}/config/default.yaml
-        ln -sf ${configFile} ${cfg.runtimeDir}/config/production.yaml
-        exec npm start
-      '';
-
       serviceConfig = {
         Type = "simple";
-        ExecStartPre = let script = pkgs.writeScript "peertube-pre-start.sh" ''
+        ExecStart = let startScript = pkgs.writeScript "peertube-start.sh" ''
           #!/bin/sh
-          sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;" ${cfg.database.name}
-          sudo -u postgres "${config.services.postgresql.package}/bin/psql" -c "CREATE EXTENSION IF NOT EXISTS unaccent;" ${cfg.database.name}
+          install -m 0750 -d ${cfg.runtimeDir}/config
+          ln -sf ${cfg.package}/config/default.yaml ${cfg.runtimeDir}/config/default.yaml
+          ln -sf ${configFile} ${cfg.runtimeDir}/config/production.yaml
+          exec npm start
         '';
-        in "+${script}";
+        in "${startScript}";
         Restart = "always";
         RestartSec = 20;
         TimeoutSec = 60;
@@ -176,7 +195,23 @@ in {
         ProtectSystem = "full";
         PrivateTmp = true;
         ProtectControlGroups = true;
-      };
+      } // (lib.optionalAttrs (!databaseActuallyCreateLocally) {
+        ExecStartPre = let preStartScript = pkgs.writeScript "peertube-pre-start.sh" ''
+          #!/bin/sh
+          cat > ${cfg.runtimeDir}/config/local-production.yaml <<EOF
+          database:
+            password: '$(cat ${cfg.database.passwordFile})'
+          EOF
+        '';
+        in "+${preStartScript}";
+      }) // (lib.optionalAttrs databaseActuallyCreateLocally {
+        ExecStartPre = let preStartScript = pkgs.writeScript "peertube-pre-start.sh" ''
+          #!/bin/sh
+          sudo -u postgres ${config.services.postgresql.package}/bin/psql -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;" ${cfg.database.name}
+          sudo -u postgres ${config.services.postgresql.package}/bin/psql -c "CREATE EXTENSION IF NOT EXISTS unaccent;" ${cfg.database.name}
+        '';
+        in "+${preStartScript}";
+      });
 
       unitConfig.RequiresMountsFor = cfg.runtimeDir;
     };
@@ -189,17 +224,13 @@ in {
       enable = true;
     };
 
-    services.postgresql = lib.mkIf cfg.database.createLocally {
+    services.postgresql = lib.mkIf databaseActuallyCreateLocally {
       enable = true;
       ensureUsers = [{
         name = cfg.database.user;
         ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; };
       }];
       ensureDatabases = [ cfg.database.name ];
-      authentication = ''
-        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 trust
-        host ${cfg.database.name} ${cfg.database.user} 127.0.0.1/32 md5
-      '';
     };
 
     users.users = lib.optionalAttrs (cfg.user == "peertube") {

@mohe2015
Copy link
Contributor

mohe2015 commented Apr 7, 2021

@Izorkin I applied the parts where I agreed but I still need to do some testing. Take care with the "+" before the systemd startup commands as they give root privileges to the process.

@Izorkin
Copy link
Contributor

Izorkin commented Apr 7, 2021

@Izorkin
Copy link
Contributor

Izorkin commented Apr 8, 2021

@stevenroose @mohe2015 small update - Izorkin@44434c4

@Izorkin
Copy link
Contributor

Izorkin commented Apr 8, 2021

@Izorkin I applied the parts where I agreed but I still need to do some testing. Take care with the "+" before the systemd startup commands as they give root privileges to the process.

fixed in Izorkin@44434c4

@stevenroose @mohe2015 can create separate PR? Here you just add a package, and for the service - create a new PR?

@stevenroose
Copy link
Contributor Author

I took the pkg part out here: #119319 (also bumped to v3.1.0)

Now let me look at all the suggestions for the module. I'm not very familiar with best practices for modules etc, though. So I might end up accepting most changes..

@mohe2015
Copy link
Contributor

mohe2015 commented Apr 13, 2021

I took the pkg part out here: #119319 (also bumped to v3.1.0)

Now let me look at all the suggestions for the module. I'm not very familiar with best practices for modules etc, though. So I might end up accepting most changes..

If you want to continue this you should really look at #119110 and #119262.

And note to @Izorkin and me: We should add @stevenroose as a co-author (author is not recommended AFAIK). so Co-authored-by: Steven Roose <steven@stevenroose.org> in the third line of the commit message

@stevenroose
Copy link
Contributor Author

So I guess I can close this in favor of #119319 for the pkg and #119110 for the module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Packaging PeerTube
9 participants