Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sourcehut: init #54425

Merged
merged 10 commits into from Aug 14, 2019
Merged

sourcehut: init #54425

merged 10 commits into from Aug 14, 2019

Conversation

eadwu
Copy link
Member

@eadwu eadwu commented Jan 21, 2019

Motivation for this change

Totally didn't steal pgpy and sqlalchemy-utils from #50328.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some requested changes apply to packages that were not yet reviewed as well.

pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/sshpubkeys/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/sshpubkeys/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/todo.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/todo.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/todo.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/todo.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/meta.nix Outdated Show resolved Hide resolved
@dotlambda
Copy link
Member

Please squash the changes into the appropriate commits.

@eadwu eadwu force-pushed the init/srhtPackages branch 3 times, most recently from bfa4f2c to 3633fb7 Compare January 21, 2019 17:04
@eadwu eadwu force-pushed the init/srhtPackages branch 2 times, most recently from 06294cf to e3dc968 Compare January 21, 2019 18:55
pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved
srhtPackages = callPackage ../applications/version-management/sr-ht {
# https://github.com/NixOS/nixpkgs/pull/54425#discussion_r249489825
pythonPackages = python36Packages;
fetchNodeModules = callPackage ../applications/networking/instant-messengers/rambox/fetchNodeModules.nix { };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can also be put in a let statement inside the expression. However, I'm not sure if fetchNodeModules should be used, I'm not familiar with the Node ecosystem.

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please look into how to run pgpy's tests. It's currently giving Ran 0 tests in 0.000s.

pkgs/applications/version-management/sr-ht/core.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/git.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/git.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/git.nix Outdated Show resolved Hide resolved
pkgs/applications/version-management/sr-ht/git.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/asyncpg/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/atpublic/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
@jb55
Copy link
Contributor

jb55 commented Jan 22, 2019

this is awesome

];

# Exclude test in root directory from 5 years ago, test_load_asc_bench.py
# Has progressbar dependency that doesn't look like even supports Python 3.x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are multiple progressbar successors in our python-packages.nix, some of which support Python 2 and 3.

Copy link
Member Author

@eadwu eadwu Jan 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it is just an outdated test, polyfilled progressbar with progressbar33 and I get an error of ImportError: cannot import name 'Exportable' from 'pgpy.types' (/build/source/pgpy/types.py) and checking pgpy/types.py there doesn't seem to any object called Exportable there.

Though progressbar2 seems to be blocked by a single test with no blunt error I can make out, https://hydra.nixos.org/build/87231550/nixlog/1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That error should be reported upstream if it has not been fixed yet.

To fix the progressbar2 build, I have pushed 6672304 to master.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/weasyprint/default.nix Outdated Show resolved Hide resolved
@eadwu
Copy link
Member Author

eadwu commented Aug 10, 2019

The status update that shows a [somewhat] viable sr.ht finally. Really only tested meta, git, and paste while todo, man, and hg works fine [so far] from my tinkering. Local nixpkgs consist of #66401, #65109, and #54425 on top of 2da00ae with a nixops configuration here [1].

In terms of other stuff, the mailing/mail server will probably be the last thing that'll work since I hardly have any experience setting that up. I'll probably just setup the web interfaces for dispatch, lists, and builds while focusing on builds so it'll be completed before I head for the dorms in college.

[1] https://gist.github.com/eadwu/e633df2660b316f3d09974594cb519e5

@bsima
Copy link
Contributor

bsima commented Aug 13, 2019 via email

@matthiasbeyer
Copy link
Contributor

Then again, setting up a mailserver could be outside the scope of this,

I agree. I also suggest the nixos-mailserver, which I have good experience with (used it only for journalwatch so far).
I would also say that this is out of scope of this package and a user should setup their own webserver for using sourcehut, not the package/service should ship one!

@joachifm
Copy link
Contributor

Seems to me all review items have been addressed. Unless there is something fundamentally wrong with this, it'd be nice to finally integrate it.

@eadwu
Copy link
Member Author

eadwu commented Aug 13, 2019

Rebased onto latest master and built successfully.

@joachifm joachifm merged commit 8f15669 into NixOS:master Aug 14, 2019
@joachifm
Copy link
Contributor

Thank you

@tomberek
Copy link
Contributor

Bravo! @eadwu thanks for sticking with it.

@matthiasbeyer
Copy link
Contributor

Yippie!

@aaronjanse
Copy link
Member

Thanks @eadwu! This closes #65109, right?

@eadwu
Copy link
Member Author

eadwu commented Aug 21, 2019

#65109 is an incomplete implementation of an actual sourcehut network (internals, not webserver/mailserver). This PR just included the packages needed for it to function.

@pinpox
Copy link
Member

pinpox commented Oct 25, 2020

Is it possible to run sourcehut with these packages and systemd services until the module in #65109 is merged?
This is what I am trying, but the service doesn't find the dependencies:

  systemd.services.sourcehut-meta = {
      description = "meta.sr.ht website service";
      requires = ["postgresql.service"];
      after = [ "postgresql.service network.target" ];

      environment = let
        penv = pkgs.python.buildEnv.override {
          extraLibs = [ pkgs.sourcehut.metasrht ] ;
        };
      in
      {
              PYTHONPATH = "${penv}/${pkgs.python.sitePackages}";
      };

      serviceConfig = {
        Type = "simple";
        User = "sourcehut";
        Restart = "always";
        ExecStart = ''${pkgs.python38Packages.gunicorn}/bin/gunicorn app.py --chdir ${pkgs.sourcehut.metasrht}/lib/python3.8/site-packages/metasrht -b 127.0.0.1:5000'';
      };
   };

journalctl -u sourcehut-meta.service shows: ModuleNotFoundError: No module named 'metasrht'

@eadwu
Copy link
Member Author

eadwu commented Oct 25, 2020

meta.sr.ht was implemented to a fully functional state already when the PR was created.

@pinpox
Copy link
Member

pinpox commented Oct 25, 2020

meta.sr.ht was implemented to a fully functional state already when the PR was created.

Can you link me any kind of information or docs on how to actually run it?

@eadwu
Copy link
Member Author

eadwu commented Oct 25, 2020

I believe a nixops implementation was referred on IRC? And there is the nixops implementation I used locally within nixos-configuration, although it's now moved from sr.ht to github.

There's no difference between the nixops configuration and normal NixOS configuration (just combine it all), nixops is just a easier way to distribute sourcehut across multiple machines (VMs).

Wrong thread to post on though since you're pinging the 20 or so other people by commenting.

@eadwu eadwu deleted the init/srhtPackages branch November 17, 2020 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet