photoprism: init at 2020-09-21 #99016

Sohalt · 2020-09-28T16:15:15Z

Motivation for this change

Things done

Sohalt · 2020-09-28T16:21:19Z

pkgs/servers/web-apps/photoprism/default.nix

+    stripRoot = false;
+  };
+
+  nasnet = fetchModel {


This fetches the pretrained tensorflow models for classifying images.
The models don't have a stable URL, so I'm not sure how to best handle that. We could upload the models to archive.org.
In addition the models are quite large (~40MB and ~90MB), so I'm not sure if we should include them in the derivation at all.

The size of the models doesn't bother me particularly. You could also add an optional argument to the derivation to enable/disable them as well. But you're right, we'd need a stable URL

Sohalt · 2020-09-28T16:23:22Z

pkgs/servers/web-apps/photoprism/default.nix

+
+  inherit pname version;
+
+  buildInputs = [


These are only needed at runtime and can be specified with a commandline flag, environment variable, or config entry.
We probably don't need them here then?

Yep. I don't think they would have any effect here.

One alternative would be to patch the default path in the source to explicitly refer to the path to the tool from nixpkgs. E.g. ffmpeg here: https://github.com/photoprism/photoprism/blob/46b9239026bae49a3ae2dbb57b0a374750d6f9e2/internal/config/filenames.go#L241

Currently, adding video files fails if I don't have ffmpeg in my PATH or pass it in the commandline / env vars.

Any opinion on what would be cleaner?

patching the source

creating a wrapper with the proper PATH

creating a wrapper that passes the commandline args

not doing this in the derivation at all, but instead in the module

@danielfullmer @andir @SuperSandro2000 any opinions?

I think creating a wrapper is maybe nicer.

if there's a commandline flag for this in photoprism, might as well use it, feels cleaner indeed

Sohalt · 2020-09-28T16:24:46Z

pkgs/servers/web-apps/photoprism/generate-dependencies.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env nix-shell
+#! nix-shell -i bash -p nodePackages.node2nix


This currently needs a development build of https://github.com/svanderburg/node2nix because it depends on svanderburg/node2nix#199

The PR you mentioned was merged. Waiting for a release.

There's the general question, if we want to keep the node2nix approach or rather use https://github.com/tweag/npmlock2nix
I've briefly tried to make the change, but failed.

Sohalt · 2020-09-28T16:28:53Z

pkgs/servers/web-apps/photoprism/default.nix

+
+    mkdir -p $out/{bin,assets}
+    # install backend
+    cp ${backend}/bin/photoprism $out/bin/photoprism


When launching the frontend you currently need to specify the path to the assets. This should probably be done automatically with a wrapper.

danielfullmer · 2020-10-01T18:15:51Z

I have the following error when trying to build this:

unpacking source archive /nix/store/yfxwlbc1r7k62y4k1v5hc5g9vd6xdlbz-wrap-ansi-5.1.0.tgz
unpacking source archive /nix/store/c5mrzczb3k2j5n8387wiqn12lqvz8w1j-yargs-14.2.3.tgz
unpacking source archive /nix/store/llms5dwk7sv6fcfcm1hjwg1n56md09p9-yargs-parser-15.0.1.tgz
unpacking source archive /nix/store/ldjjxy75frr4acjywaggs00mpdpqzvc2-yauzl-2.10.0.tgz
unpacking source archive /nix/store/lbb37yhx0sdqzp6aswllfw37y7m3llp4-yeast-0.1.2.tgz
cp: cannot stat '/nix/store/dmy4bwy0as5xl4l0zh424viypi8viz0a-photoprism/package.json': No such file or directory
error: --- Error --- nix-daemon
builder for '/nix/store/mzwzf5ivgp5larzmbhrv3y6pvpgn5yn3-node-dependencies-photoprism-1.0.0.drv' failed with exit code 1; last 10 log lines:

Sohalt · 2020-10-02T14:57:00Z

@danielfullmer sorry, I forget to push the package.json. Can you try again?

andir · 2020-10-03T10:50:50Z

Nice to see more people that are using photoprism in the Nix community! I've for a few months now had a private module for that https://github.com/andir/infra/blob/master/config/modules/photoprism.nix as well as an expression to package it: https://github.com/andir/infra/blob/master/nix/packages/photoprism/default.nix . Maybe this can be of value for this work here.

I didn't consider adding it to nixpkgs as I really dislike generating Nix code and checking that into the repo. I instead built (at first ranz2nix (https://github.com/andir/ranz2nix/ which eventually to) npmlock2nix which only requires the lock files and no generated nix expressions.

I'll try to give this PR a review in the next days as that would remove the sole maintenance burden from me. Thank you for the work so far! 👍

Sohalt · 2020-10-03T11:03:04Z

@andir thanks! As said, I sadly didn't see your work before starting this. I think your derivation is a bunch cleaner than mine. I wanted to incorporate some of your approach into this PR (or maybe just get your stuff upstreamed instead?) I just didn't get around to it yet.
I agree that npmlock2nix seems cleaner than having all the node2nix generated stuff. Would be nice to see that adopted more throughout nixpkgs.

danielfullmer

I tested this package out briefly. Seems to be mostly working, but requires setting a number of env vars / command line args that could probably be handled automatically. I'd also be interested in a NixOS module that makes this trivial to set up. Thanks for the work thus far!

danielfullmer · 2020-10-03T22:51:08Z

pkgs/servers/web-apps/photoprism/default.nix

+
+  inherit pname version;
+
+  buildInputs = [


Yep. I don't think they would have any effect here.

One alternative would be to patch the default path in the source to explicitly refer to the path to the tool from nixpkgs. E.g. ffmpeg here: https://github.com/photoprism/photoprism/blob/46b9239026bae49a3ae2dbb57b0a374750d6f9e2/internal/config/filenames.go#L241

Currently, adding video files fails if I don't have ffmpeg in my PATH or pass it in the commandline / env vars.

danielfullmer · 2020-10-03T22:51:59Z

pkgs/servers/web-apps/photoprism/default.nix

+    stripRoot = false;
+  };
+
+  nasnet = fetchModel {


The size of the models doesn't bother me particularly. You could also add an optional argument to the derivation to enable/disable them as well. But you're right, we'd need a stable URL

Sohalt · 2020-11-12T19:37:14Z

@andir I've tried to adapt your derivation to use npmlock2nix, but failed (When I try to set packageLockJson to src + "/frontend/package-lock.json" I get /nix/store/ib6rys6q6pybfll90pdm4z3gzh2bhm95-source/frontend/package-lock.json must a path). Do you think you can find the time to upstream your version? Not sure, if I can be of any help, but if you think so, just ping me.

SuperSandro2000 · 2020-11-24T01:17:33Z

pkgs/servers/web-apps/photoprism/default.nix

@@ -0,0 +1,127 @@
+{ lib
+, pkgs


Don't pass in pkgs and you only need stdenv and not stdenv and lib.

SuperSandro2000 · 2020-11-24T01:18:34Z

pkgs/servers/web-apps/photoprism/default.nix

+      pushd frontend
+      ln -s ${nodeDependencies}/lib/node_modules ./node_modules
+      export PATH="${nodeDependencies}/bin:$PATH"
+      NODE_ENV=production npm run build


Should we set this globally?

No, this is only required for the build of the static files.

SuperSandro2000 · 2020-11-24T01:19:03Z

pkgs/servers/web-apps/photoprism/default.nix

+    description = "Personal Photo Management powered by Go and Google TensorFlow ";
+    platforms = platforms.linux;
+    license = licenses.agpl3;
+    maintainers = with maintainers; [ ];


Please add yourself as a maintainer.

Feel free to add me as well. I'm using this actively and wouldn't mind having a look at this every now and then.

SuperSandro2000 · 2020-11-24T01:19:49Z

pkgs/servers/web-apps/photoprism/generate-dependencies.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env nix-shell
+#! nix-shell -i bash -p nodePackages.node2nix


The PR you mentioned was merged. Waiting for a release.

stale · 2021-06-03T19:34:50Z

I marked this as stale due to inactivity. → More info

polynomial · 2021-06-09T01:32:44Z

Found this, thankfully before starting my own. Guess I will now see what it takes to get both this and/or andir's running. Thanks to everyone who provided this(@andir @Sohalt ). I will now attempt to stand on the toes of giants.

makefu · 2021-07-24T18:51:12Z

any chance to ever get a working photoprism package in nixpkgs?

SuperSandro2000 · 2021-07-24T23:26:43Z

pkgs/servers/web-apps/photoprism/default.nix

+}:
+
+let
+  version = "2020-09-21";


Suggested change

version = "2020-09-21";

version = "unstable-2020-09-21";

SuperSandro2000 · 2021-07-24T23:27:49Z

pkgs/servers/web-apps/photoprism/default.nix

+    runHook postInstall
+  '';
+
+  meta = with stdenv.lib; {


Suggested change

meta = with stdenv.lib; {

meta = with lib; {

mbrgm · 2021-07-30T08:02:36Z

@Sohalt @polynomial Are you going to to tackle this anytime soon? I'm desperately waiting for non-dockered photoprism in nixpkgs and would postpone my own efforts if this PR will be merged soon.

SuperSandro2000 · 2021-08-02T07:51:49Z

pkgs/servers/web-apps/photoprism/node-composition.nix

+import ./node-packages.nix {
+  inherit (pkgs) fetchurl fetchgit;
+  inherit nodeEnv;
+}


Missing final new line.

Sohalt · 2021-08-04T11:17:48Z

@Sohalt @polynomial Are you going to to tackle this anytime soon? I'm desperately waiting for non-dockered photoprism in nixpkgs and would postpone my own efforts if this PR will be merged soon.

I won't have time in the immediate future, but am hoping to get this done before November, if nobody else does it until then.

aanderse · 2021-12-09T15:09:46Z

ping @Sohalt - any chance you're going to tackle this again?

Sohalt · 2021-12-09T23:36:45Z

I've had another look at this but I don't really like the node2nix approach, because it generates so many intermediary files. I'd prefer using npmlock2nix, but that cannot be included in nixpkgs. In the meantime I found a flake at https://github.com/GTrunSec/photoprism2nix, which I might improve on. If someone else want's to continue this effort here, feel free.

pinpox · 2022-01-27T14:25:18Z

In the meantime I found a flake at https://github.com/GTrunSec/photoprism2nix, which I might improve on.

Did you end up using that flake? It seems to be unmaintained or at least not get many updates. I'm looking for a good solution on how to run photoprism on nixOS and not sure where to go from here.

Sohalt · 2022-01-27T14:58:56Z

I've started to implement some adjustments at https://github.com/Sohalt/photoprism2nix/, but left it in a rough state and didn't deploy it yet, because I'm waiting on the multi-user support in photoprism to land first. That should hopefully happen in Q1 of 2022.

pinpox · 2022-01-28T07:58:50Z

Thanks, I'll keep an eye on that repository. Had not noticed that multi user support is still missing, that is definitely a dealbreaker for me. Not sure what the alternatives are for now, everything seems either unmaintained or rudimentary.

Sohalt force-pushed the photoprism branch from bcc1a20 to a294a9c Compare September 28, 2020 16:17

Sohalt commented Sep 28, 2020

View reviewed changes

Sohalt mentioned this pull request Sep 28, 2020

Package Photoprism #96043

Closed

Sohalt commented Sep 28, 2020

View reviewed changes

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Sep 28, 2020

photoprism: init at 2020-09-21

72a1547

Sohalt force-pushed the photoprism branch from a294a9c to 72a1547 Compare October 2, 2020 14:56

danielfullmer reviewed Oct 3, 2020

View reviewed changes

SuperSandro2000 reviewed Nov 24, 2020

View reviewed changes

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 3, 2021

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 9, 2021

SuperSandro2000 reviewed Jul 24, 2021

View reviewed changes

SuperSandro2000 reviewed Aug 2, 2021

View reviewed changes

Sohalt closed this Dec 9, 2021

Misterio77 mentioned this pull request Apr 30, 2022

photoprism: init at 220302-0059f429 #170953

Closed

13 tasks

		@@ -0,0 +1,8 @@
		#!/usr/bin/env nix-shell
		#! nix-shell -i bash -p nodePackages.node2nix

photoprism: init at 2020-09-21 #99016

photoprism: init at 2020-09-21 #99016

Conversation

Sohalt commented Sep 28, 2020

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sohalt Oct 13, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielfullmer commented Oct 1, 2020

Sohalt commented Oct 2, 2020

andir commented Oct 3, 2020

Sohalt commented Oct 3, 2020

danielfullmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sohalt commented Nov 12, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stale bot commented Jun 3, 2021

polynomial commented Jun 9, 2021

makefu commented Jul 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbrgm commented Jul 30, 2021 • edited

Choose a reason for hiding this comment

Sohalt commented Aug 4, 2021

aanderse commented Dec 9, 2021

Sohalt commented Dec 9, 2021

pinpox commented Jan 27, 2022

Sohalt commented Jan 27, 2022

pinpox commented Jan 28, 2022

Sohalt Oct 13, 2020 •

edited

mbrgm commented Jul 30, 2021 •

edited