Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sourcehut: include module #65109

Closed
wants to merge 8 commits into from
Closed

Conversation

eadwu
Copy link
Member

@eadwu eadwu commented Jul 19, 2019

In case I ever finish this or if someone wants to see some sort of thing as reference.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@eadwu eadwu force-pushed the sourcehut/include-module branch 3 times, most recently from 88e4c6d to 17cf6fe Compare August 10, 2019 01:23
@eadwu eadwu mentioned this pull request Aug 10, 2019
10 tasks
@eadwu eadwu force-pushed the sourcehut/include-module branch 3 times, most recently from 729bf36 to c01ec1c Compare August 17, 2019 01:25
# from other sites in your network.
#
# Use the srht-webhook-keygen command to generate a key.
webhooks.private-key = mkDefault null;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Serious security problem here.
Any idea? I thought of just appending this part from the content of a secure file, creating a third, non-store config for the service to use.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC services e.g. ssh generate keys on service start if they're not present...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appending won't fix the problem, it'll still end up in the store. If you don't want it to be in the store, the only option is to have as an external file and you're better off setting it up yourself since I can't access the settings unless I parse the file's contents.

I'm pretty sure the generated key isn't suppose to be used for ssh connections.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why wont appending fix the problem? I dont think you understood what I meant. The file we're appending from is not in the store, and neiter is the file were appending to. Only the 'base' file (the one created here, without private keys) would be in the store.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, the only configuration file for sourcehut lies in /etc/sr.ht/config.ini since the relative config.ini is out of the question unless one starts overriding the derivation.

a = 1;
${builtins.readFile path_to_external_file}

Would have the file contents of path_to_external_file in the file in the store.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This really is a sourcehut bug. Secrets dont go in config files.

@matthiasbeyer
Copy link
Contributor

Any progress here?

@eadwu
Copy link
Member Author

eadwu commented Dec 27, 2019

All the web interfaces should work fine? At least when I ran a couple of qemu vms through nixops it seemed everything worked fine locally in terms of the interface. For the lists and other integration things (like mailbox server) things I have no clue. I basically only host an instance with meta/git/paste locally.

@matthiasbeyer
Copy link
Contributor

Could you share your configuration.nix for the setup you have? I plan on trying to host an installation on a server I have, probably deployed to individual nspawn containers (not sure about how to do it right yet) and test it.

I have a mailserver running on that server, as well as postgres and redis, but no cron (yet, as I use systemd timers). I wonder how to properly set this up using containers for everything (or VMs, as said not sure which way to go, yet).

@eadwu
Copy link
Member Author

eadwu commented Feb 26, 2020

https://git.sr.ht/~eadwu/nixos-configuration/tree/master/nixops/sourcehut

Some configuration options might be missing in this PR.

Basically a port from git

TESTED
Creating a repo through the web interface
Cloning through https `hg clone --insecure`
TESTED
Creating a ticket through the web interface
Responding, resolving, blah, blah through the web interface
};

settings = mkOption {
type = with types; attrsOf (attrsOf (nullOr (either bool (either int (either float (either str path))))));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will produce a crazy type like the one in #86402. Can you add a decription with // { description = "..."; }?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there already a description for this in line 73?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think he meant an example maybe?

@pinpox
Copy link
Member

pinpox commented Oct 21, 2020

Hey, what is the status of this PR? I'm looking to setup sourcehut on my nixOS server, is there anything I can do to get this finished? Any help needed?

EDIT: I would need a bit of guidance, I'm quite new to the nix world

@happysalada
Copy link
Contributor

happysalada commented Nov 16, 2020

@pinpox I am by no means an expert, so take my advice with a grain of salt.
The first step is to test the current PR.
for that, you need to copy the files of this PR on a nixos machine, add an imports = [ ./sourcehut.nix] in the imports of your configuration.nix file. sourcehut.nix should just include the content of the sourcehut/default.nix from this PR. Then you can add inside your configuration.nix the proper configuration for the sourcehut service. I've found the configuration that @eadwu uses https://github.com/eadwu/nixos-configuration/tree/master/nixops/sourcehut for reference

Let me know if anything doesn't make sense.

@eadwu eadwu closed this Nov 17, 2020
@eadwu eadwu deleted the sourcehut/include-module branch November 17, 2020 23:34
@tgunnoe
Copy link
Contributor

tgunnoe commented Feb 11, 2021

Why was this closed?

@eadwu
Copy link
Member Author

eadwu commented Feb 11, 2021

It is extremely unlikely that I will ever finish the entire module. I don't use some of the sub-services at all (such as builds or lists) nor do I have the experience of setting up build clusters correctly.

@happysalada
Copy link
Contributor

If anybody ever picks this up again, just put a comment here, I'm interested in helping the person test the PR and merge it.

@tgunnoe
Copy link
Contributor

tgunnoe commented Feb 14, 2021

ok thanks @eadwu for the update. I'll take a hit at it as soon as I figure it well

@tomberek
Copy link
Contributor

Had to update some of the available settings, but the existing setup seems to function (https://todo.srht.tomberek.info/).

Registration + email works.
git: https://git.srht.tomberek.info/~tomberek2/thing
todo: https://todo.srht.tomberek.info/~tomberek2/something-to-track

There are some fiddly things. The current one:
git update hooks don't seem to be called with the right Arg[0]: https://git.sr.ht/~sircmpwn/git.sr.ht/commit/84afd9d7b0298b6b418684daf7feab7f74635fa1#gitsrht-update-hook/main.go-1-31 And I get errors like:

Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 4 threads
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 10.05 KiB | 3.35 MiB/s, done.
Total 11 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Unknown git hook /nix/store/jzad97p554fhb589027igf1rrfmabr1b-python3.8-gitsrht-0.61.10/bin/gitsrht-update-hook
To git.srht.tomberek.info:~tomberek2/sourcehut-nix
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git.srht.tomberek.info:~tomberek2/sourcehut-nix'

@eadwu
Copy link
Member Author

eadwu commented Feb 14, 2021

I believe the original approach in this PR was a workaround for a bug in the implementation but it was since fixed upstream. Sadly I don't remember the specifics.

@tomberek
Copy link
Contributor

This is in relation to gtsrht-update-hook. It seems os.Args[0] is the full path, not the relative path and symlink. It is expecting something like "hooks/pre-receive". The links exist on disk. Looks okay. Was thinking there might be some obscure git setting "don't resolve symlinks before calling them".

@tomberek
Copy link
Contributor

Did a mass update to the latest sources and added the 5100 API (not exactly sure if i did that right, but until I did, none of the web services were responding at all). Still have the same issue with the gitsrht-update-hook.

@tomberek
Copy link
Contributor

The arg0 was an artifact caused by Nix. The python builder wraps all the executables in bin and changes arg0 in doing so. The fix is to point the setting directly at the generated go binary instead of through the wrapper.

@tomberek
Copy link
Contributor

Putting my WIP here: https://git.srht.tomberek.info/~tomberek2/sourcehut-nix
Has the module in root, updated package code in sourcehut/, and example settings in nginx.nix.

@eadwu can we reopen this, or should we start a new PR?

@eadwu
Copy link
Member Author

eadwu commented Feb 14, 2021

A new PR, I won't be able to review it since as stated above I don't use all the services anyway.

@tomberek tomberek mentioned this pull request Feb 16, 2021
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants