[WIP] nixos/mautrix-telegram: init module #60151

Vskilet · 2019-04-24T09:26:08Z

Motivation for this change

On the same pattern that mautrix-whatsapp, I suggest this service for mautrix-telegram.
What do you think about it @Ma27 :-) ?

Things done

Ma27

First of all thanks a lot for upstreaming NixOS modules for Matrix bridges!

Aside from the comments below I have another concern about this: both mautrix-whatsapp and mautrix-telegram are more or less equal to configure (well, except for the alembic part here), so I'm still wondering if we can create a single module where you can deploy several bridges, similar to the approach taken for services.prometheus.exporters where you can deploy multiple prometheus exporters using the same abstration in the NixOS module.

nixos/modules/services/misc/mautrix-telegram.nix

Ma27 · 2019-04-24T10:16:21Z

nixos/modules/services/misc/mautrix-telegram.nix

+        DynamicUser = true;
+        StateDirectory = "mautrix-telegram";
+        ExecStart = ''
+          ${pkgs.mautrix-telegram}/bin/mautrix-telegram -c "${configFile}/config.yaml"


I'm afraid that this won't work right away. As described in the upstream docs you need to configure a database first using alembic.

To get this working locally, I had to patch the mautrix-telegram package accordingly: #59519

I had an error locally to generate the database. I will try with your pull request.

I try to merge your pull request but I've a problem during the generation of the database :

Traceback (most recent call last): File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/bin/.alembic-wrapped", line 11, in <module> sys.exit(main()) File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/config.py", line 527, in main CommandLine(prog=prog).main(argv=argv) File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/config.py", line 521, in main self.run_cmd(cfg, options) File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/config.py", line 501, in run_cmd **dict((k, getattr(options, k, None)) for k in kwarg) File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/command.py", line 276, in upgrade script.run_env() File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/script/base.py", line 475, in run_env util.load_python_file(self.dir, "env.py") File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 90, in load_python_file module = load_module_py(module_id, path) File "/nix/store/2z280p610w8yn9ypsngk2xzpgi54hl6b-python3.7-alembic-1.0.7/lib/python3.7/site-packages/alembic/util/compat.py", line 156, in load_module_py spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "alembic/env.py", line 10, in <module> from mautrix_telegram.base import Base ModuleNotFoundError: No module named 'mautrix_telegram.base'

Have you used mautrix-telegram.alembic or python3Packages.alembic? As described in the PR you need a alembic with mautrix-telegram available which is why I added the passthru expression.

I just checked it locally whether it creates a sqlite db successfully in my mautrix-telegram checkout.

I merged just your commit on my branch and use a nix-shell to generated it so I think it should be the right packet, isn't ?

It's unfortunately not documented in the bridge's wiki since this is usually not a problem when setting up the bridge in a git checkout of mautrix-telegram.

I simply did a ln -sf ${pkgs.mautrix-telegram}/alembic alembic with in a preStart hook of the systemd service back then. Unless you have any better ideas that should be the way to go :)

I tried your suggestion Ma27 but I've this error :
ln: failed to create symbolic link 'alembic': Read-only file system

Can you please share your local diff with me then?

@pacien wrote:

Have you tried setting the working directory to the application's source root? It might work as script_location is defined relatively in alembic.ini.

Repeating myself here. The following works:

preStart = '' cd ${pkgs.mautrix-telegram} ${pkgs.mautrix-telegram.alembic}/bin/alembic -x config=${configFile}/config.yaml upgrade head '';

Also note that preStart should belong to systemd.services.mautrix-telegram instead of being nested in serviceConfig.

Sorry, I've just seen the last answer while reading unread GitHub notifications. In fact that's even better than my solution 👍

Vskilet · 2019-04-24T12:17:49Z

[...] if we can create a single module where you can deploy several bridges [...]

So should I delete all my previous PR and retry in just one ?
And I should add a default configuration if we do that, isn't it ?

Ma27 · 2019-04-24T12:47:13Z

So should I delete all my previous PR and retry in just one ?

No, I'm not even sure if we actually want to do that. Let's focus on getting mautrix-whatsapp functional for now to have a POC in the nixpkgs master and discuss here if we want to unify the modules.

infinisil · 2019-04-26T18:11:10Z

nixos/modules/services/misc/mautrix-telegram.nix

+    enable = mkEnableOption "Mautrix-telegram, a bridge between Matrix and Telegram";
+
+    configOptions = mkOption {
+      type = types.attrs;


(feel free to ignore) Another candidate for NixOS/rfcs#42

pacien · 2019-06-05T15:20:56Z

It would be nice to have a way to supply secret API keys and tokens (telegram.api_id, telegram.api_hash and telegram.bot_token) outside of the world-readable nix store. This can be achieved by using an environment file if mautrix-telegram supports configuration overrides through environment variables.

Ma27 · 2019-06-05T17:40:47Z

This can be achieved by using an environment file if mautrix-telegram supports configuration overrides through environment variables.

IIRC this is not possible. You have a single config file for everything and we'd have to patch the bridge accordingly. This is more or less the same problem as in the corresponding mautrix-whatsapp PR (#59211).

pacien · 2019-06-06T20:31:27Z

@Ma27 wrote:

IIRC this is not possible. You have a single config file for everything and we'd have to patch the bridge accordingly.

Just submitted a pull request to upstream for that: mautrix/telegram#332

pacien · 2019-06-06T22:57:23Z

nixos/modules/services/misc/mautrix-telegram.nix

+        DynamicUser = true;
+        StateDirectory = "mautrix-telegram";
+        ExecStart = ''
+          ${pkgs.mautrix-telegram}/bin/mautrix-telegram -c "${configFile}/config.yaml"


@pacien wrote:

Have you tried setting the working directory to the application's source root? It might work as script_location is defined relatively in alembic.ini.

Repeating myself here. The following works:

preStart = '' cd ${pkgs.mautrix-telegram} ${pkgs.mautrix-telegram.alembic}/bin/alembic -x config=${configFile}/config.yaml upgrade head '';

Also note that preStart should belong to systemd.services.mautrix-telegram instead of being nested in serviceConfig.

pacien · 2019-06-06T23:06:55Z

nixos/modules/services/misc/mautrix-telegram.nix

+  options.services.mautrix-telegram = {
+    enable = mkEnableOption "Mautrix-telegram, a bridge between Matrix and Telegram";
+
+    configOptions = mkOption {


An example and a default value would be nice here.

In particular, we could set appservice.database to sqlite:////var/lib/mautrix-telegram/mautrix-telegram.db by default to use an SQLite database placed in the StateDirectory of the service.

I can do that but the exemple will be long !

pacien · 2019-06-06T23:08:21Z

nixos/modules/services/misc/mautrix-telegram.nix

+
+    services.matrix-synapse.app_service_config_files = [ "${configFile}/registration.yaml" ];
+
+  };


If you intend to maintain this module, you can set meta.maintainers.

I never view a module with a maintainers ... OK I'll do it

pacien · 2019-06-07T14:18:18Z

mautrix/telegram#332 has been merged in upstream.

Vskilet · 2019-06-25T08:50:53Z

Ok guys, @pacien in #63589 suggest something better than this and it anticipates the next version of mautrix-telegram to avoid that api token can be worldwide readable.

Vskilet requested a review from infinisil as a code owner April 24, 2019 09:26

Vskilet force-pushed the mautrix-telegram branch from 581185e to 27ed81f Compare April 24, 2019 09:27

ofborg bot added 6.topic: nixos 8.has: module (update) labels Apr 24, 2019

Ma27 requested changes Apr 24, 2019

View reviewed changes

nixos/mautrix-telegram: init module

9cb1f05

Vskilet force-pushed the mautrix-telegram branch from 27ed81f to 9cb1f05 Compare April 24, 2019 12:37

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Apr 24, 2019

Vskilet changed the title ~~nixos/mautrix-telegram: init module~~ [WIP] nixos/mautrix-telegram: init module Apr 26, 2019

infinisil reviewed Apr 26, 2019

View reviewed changes

mautrix-telegram: creation database on start

ea3ec8f

ofborg bot added 10.rebuild-linux: 0 and removed 10.rebuild-linux: 1-10 labels May 28, 2019

pacien suggested changes Jun 6, 2019

View reviewed changes

Ma27 mentioned this pull request Jun 8, 2019

[WIP] matrix-appservice-irc: init at 0.12.0 #62864

Closed

12 tasks

pacien mentioned this pull request Jun 20, 2019

nixos/mautrix-telegram: add module #63589

Merged

16 tasks

Vskilet closed this Jun 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] nixos/mautrix-telegram: init module #60151

[WIP] nixos/mautrix-telegram: init module #60151

Vskilet commented Apr 24, 2019 •

edited

Loading

Ma27 left a comment

Ma27 Apr 24, 2019

Vskilet Apr 24, 2019

Vskilet Apr 26, 2019

Ma27 Apr 26, 2019

Vskilet Apr 26, 2019

Ma27 Jun 5, 2019

Vskilet Jun 6, 2019

Ma27 Jun 6, 2019

pacien Jun 6, 2019

Ma27 Jun 7, 2019

Vskilet commented Apr 24, 2019 •

edited

Loading

Ma27 commented Apr 24, 2019

infinisil Apr 26, 2019

pacien commented Jun 5, 2019

Ma27 commented Jun 5, 2019

pacien commented Jun 6, 2019 •

edited

Loading

pacien Jun 6, 2019

pacien Jun 6, 2019

Vskilet Jun 14, 2019

pacien Jun 6, 2019

Vskilet Jun 14, 2019

pacien commented Jun 7, 2019

Vskilet commented Jun 25, 2019


		services.matrix-synapse.app_service_config_files = [ "${configFile}/registration.yaml" ];

		};

[WIP] nixos/mautrix-telegram: init module #60151

[WIP] nixos/mautrix-telegram: init module #60151

Conversation

Vskilet commented Apr 24, 2019 • edited Loading

Motivation for this change

Things done

Ma27 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vskilet commented Apr 24, 2019 • edited Loading

Ma27 commented Apr 24, 2019

Choose a reason for hiding this comment

pacien commented Jun 5, 2019

Ma27 commented Jun 5, 2019

pacien commented Jun 6, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pacien commented Jun 7, 2019

Vskilet commented Jun 25, 2019

Vskilet commented Apr 24, 2019 •

edited

Loading

Vskilet commented Apr 24, 2019 •

edited

Loading

pacien commented Jun 6, 2019 •

edited

Loading