Patchwork #35334

manveru · 2018-02-22T13:04:17Z

Motivation for this change

Closes #31052 and #24784

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

manveru · 2018-02-22T13:05:29Z

Sorry, didn't mean to ping you zimba.

manveru · 2018-02-22T13:09:23Z

Smilies aren't working yet... will have to strace to see where it's looking for them or maybe it's a font thing.

matthiasbeyer · 2018-02-22T13:09:31Z

$ nix-build -A patchwork -Q
error: unsupported argument ‘sha256’ to ‘fetchTarball’, at 0x1aedb448
(use ‘--show-trace’ to show detailed location information)

manveru · 2018-02-22T13:10:50Z

Argh, I'm always on nix 1.12 so I didn't notice that. Will try to come up with a fix.

manveru · 2018-02-22T13:15:16Z

Should be fixed now.

matthiasbeyer · 2018-02-22T13:20:52Z

I love you.

This works for me.

dtzWill · 2018-02-22T13:22:38Z

electron update should probably be a separate commit
... That yarn.lock file is huge (4k+ lines). Can it be fetched instead or something (without causing IFD problems)?

manveru · 2018-02-22T13:24:05Z

Unfortunately the derivation can't be built using npm2nix, so I had to make a yarn.lock file. I could in theory host it somewhere else and download it, but not sure what a good place for that would be.

I can split up electron update into a separate commit. I don't think it'll cause much problems.

dtzWill · 2018-02-22T13:33:25Z

Same PR is fine, just good practice and makes it more manageable (to cherrypick, revert, ...)

If the file can be fetched without IFD, then definitely let's find a different home for it. Otherwise every single channel updates for forever will include a copy of it! Haha

zimbatm · 2018-02-22T14:12:36Z

pkgs/applications/networking/instant-messengers/patchwork/default.nix

+    repo = "yarn2nix";
+    rev = "d6e05a521bd92b2647bb7e853363d234f21b2cfd";
+    sha256 = "1nvpii9p41vrb6zvr8rqcvmycrl6lnzzaif85qj1aavizncgb4wy";
+  }) {


last time I checked import from derivation was not allowed on Hydra, we would have import yarn2nix into nixpkgs first

Just need to find a good location for it... I'll start working on a separate PR. Just hope @moretea is OK with it.

I'm quite sure he's OK with it. The only reason not to include it is that a new yarn2nix nix emerged that is even more powerful: https://github.com/Profpatsch/yarn2nix

zimbatm · 2018-02-22T14:14:24Z

pkgs/applications/networking/instant-messengers/patchwork/default.nix

+  nodeHeaders = fetchurl {
+    url = "https://nodejs.org/download/release/v${nodejs.version}/node-v${nodejs.version}-headers.tar.gz";
+    sha256 = "1993kcghzr56zmw5sdj8wr8c42mna25806bcjknfxnh62zl4hwpg";
+  };


I'm doing the exact same thing in another project and it breaks every time I update nixpkgs and the nodejs has been bumped. I think the nodejs package should export the headers in another output like nodejs.dev

matthiasbeyer · 2018-03-09T15:52:54Z

Can we please get this ready before the actual nixos 18.03 release?

manveru · 2018-03-09T15:57:03Z

This depends entirely on #35340 making it in.

zimbatm · 2018-03-09T21:28:49Z

@manveru can you rebase on master?

zimbatm · 2018-03-10T11:38:11Z

cannot read ‘/nix/store/j9w77q7mb7dnx31h1rnwa4kvzcnwnfq3-source/package.json’, since path ‘/nix/store/knid138m3yqmnm6b8kvlpr94z18kf7pn-source.drv’ is not valid, at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/ogden/lib/trivial.nix:122:24

This is the same error we were seeing in yarn2nix

manveru · 2018-03-10T11:54:14Z

I'm not sure how to solve that then. Really would like to avoid adding the package.json and yarn.lock as well to nixpkgs...

zimbatm · 2018-03-10T12:16:30Z

I don't think we have a choice

dtzWill · 2018-03-10T18:44:32Z

... That yarn.lock file is huge (4k+ lines). Can it be fetched instead or something (without causing IFD problems)?

Sounds like "without causing IFD problems" is indeed not possible, yes? Still 4K is heavy, not sure how important this is to everyone. It certainly would be a problem if we had many such files...

manveru · 2018-03-11T23:11:28Z

So I finally added the yarn.lock and generated yarn.nix to this PR.
I doubt it'll be merged due to the size of over 11k lines, but it should provide some nice food for thought about how we can handle cases like this.

I think this is a really good example to show that for some derivations, import-from-derivation (IFD) can be very valuable. I'm not sure what needs to be done to make this an option though.

If this is not merged you can still get it from my nixpkgs fork, I'll keep the branch around, but probably won't be using Patchwork much myself, so don't expect frequent updates.

matthiasbeyer · 2018-03-12T09:32:24Z

Thanks for providing this patch!

I want patchwork in nixpkgs, but I don't want this huge file in nixpkgs. This is definitively not the path to take for packages like this and we should look for a better solution how to package things like this (in the general case for nixpkgs, not only in this case).

I would say No! don't merge this.

manveru · 2018-03-12T10:45:47Z

I talked with @globin and @domenkozar yesterday, and they think we could have a combined yarn-packages.nix like we do for other languages. I think it's possible to do that, but due to the nature of NPM, it will have a lot more entries. The benefit is that we can probably share quite some dependencies between different electron apps.

I'll try my hand at packaging #32516 as well and see how much they have in common.

matthiasbeyer · 2018-03-21T19:25:53Z

Any progress?

matthiasbeyer · 2018-04-25T14:00:04Z

Any progress?

mguentner · 2018-04-25T15:42:17Z

I believe that electron apps are (too) high hanging fruits for nixpkgs with the current amount of humans working on it.

I propose to loosen the sandbox model for such apps, meaning that you can run "npm install" / "yarn install" prior to the buildPhase in some specialDepsPhase.
That would drastically reduce the work needed for packaging electron apps. It will be similar to writing PKGBUILDs for ArchLinux.

Package output:

$ yaourt -Ql ssb-patchwork | cut -d '/' -f 1-4 | uniq
ssb-patchwork /opt/
ssb-patchwork /opt/ssb-patchwork/
ssb-patchwork /opt/ssb-patchwork/LICENSE
ssb-patchwork /opt/ssb-patchwork/README.md
ssb-patchwork /opt/ssb-patchwork/assets
ssb-patchwork /opt/ssb-patchwork/code-of-conduct.md
ssb-patchwork /opt/ssb-patchwork/contributing.md
ssb-patchwork /opt/ssb-patchwork/index.js
ssb-patchwork /opt/ssb-patchwork/lib
ssb-patchwork /opt/ssb-patchwork/locales
ssb-patchwork /opt/ssb-patchwork/main-window.js
ssb-patchwork /opt/ssb-patchwork/modules
ssb-patchwork /opt/ssb-patchwork/node_modules
ssb-patchwork /opt/ssb-patchwork/overrides
ssb-patchwork /opt/ssb-patchwork/package-lock.json
ssb-patchwork /opt/ssb-patchwork/package.json
ssb-patchwork /opt/ssb-patchwork/plugs
ssb-patchwork /opt/ssb-patchwork/sbot
ssb-patchwork /opt/ssb-patchwork/screenshot.jpg
ssb-patchwork /opt/ssb-patchwork/scripts
ssb-patchwork /opt/ssb-patchwork/server-process.js
ssb-patchwork /opt/ssb-patchwork/styles
ssb-patchwork /usr/
ssb-patchwork /usr/bin/
ssb-patchwork /usr/bin/ssb-patchwork
ssb-patchwork /usr/share/
ssb-patchwork /usr/share/applications
ssb-patchwork /usr/share/icons

All deps reside in /opt/ssb-patchwork/node_modules.

These packages would be second class with no guarantees.
Is there already some kind of fetchScript = "npm install" in nixpkgs?

▶️ Apparently the alternative is to add 4000 line yarn lock files to nixpkgs.

bb010g · 2018-07-02T19:30:50Z

Why would node2nix not work here? Patchwork provides a package-lock.json upstream that could probably be used instead of a yarn.lock downstream.

Ekleog · 2018-10-01T00:48:43Z

(triage) If I read correctly, this is more or less blocked on Hydra turning on IFD? If so, maybe tag it “WIP” or something?

matthiasbeyer · 2018-10-01T07:36:28Z

For the curious: what is IFD?

tilpner · 2018-10-01T07:39:51Z

"Import-from-derivation": import (runCommand "foo" {} "echo 42 > $out")

matthiasbeyer · 2019-03-16T16:35:27Z

Any progress?

c0bw3b · 2019-05-27T17:18:21Z

#61530 provided an AppImage version of patchwork.
Building from source would still be better but it has stalled for now.

@manveru feel free to open a new PR or ask for reopening this one if you are still working on this

manveru · 2019-05-27T19:58:54Z

@c0bw3b it's fine, not using it myself anyway.

manveru requested a review from zimbatm as a code owner February 22, 2018 13:04

manveru force-pushed the patchwork branch from 9e1548b to 854b36b Compare February 22, 2018 13:05

manveru mentioned this pull request Feb 22, 2018

patchwork: init at 3.8.0 (WIP) #31052

Closed

8 tasks

zimbatm reviewed Feb 22, 2018

View reviewed changes

manveru mentioned this pull request Feb 22, 2018

electron: 1.7.11 -> 1.8.2 #35342

Merged

8 tasks

manveru added 2 commits March 9, 2018 22:53

Patchwork: init at 3.8.10

211e74f

Use fetchFromGitHub instead of fetchTarball

78cc9cb

manveru force-pushed the patchwork branch from 42ddba7 to 78cc9cb Compare March 9, 2018 21:54

Use mkYarnPackage

66beeaf

zimbatm mentioned this pull request Mar 11, 2018

yarn2nix #20637

Closed

2 tasks

Add yarn.nix to make it build

8633e52

manveru force-pushed the patchwork branch from 53d00e4 to 10a795e Compare March 11, 2018 22:42

Add package.json

36e5d73

manveru force-pushed the patchwork branch from 10a795e to 36e5d73 Compare March 11, 2018 22:47

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Mar 11, 2018

globin mentioned this pull request Mar 13, 2018

nodejs: add headers output #36910

Closed

8 tasks

matthewbauer added the 6.topic: nodejs label May 5, 2018

mrVanDalo mentioned this pull request Jun 18, 2018

patchwork-classic : add 2.12.0 #42151

Merged

8 tasks

timokau added the 2.status: work-in-progress label Dec 29, 2018

c0bw3b closed this May 27, 2019

manveru deleted the patchwork branch December 12, 2019 17:13

c0bw3b removed the 2.status: work-in-progress label Feb 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patchwork #35334

Patchwork #35334

manveru commented Feb 22, 2018 •

edited

manveru commented Feb 22, 2018

manveru commented Feb 22, 2018

matthiasbeyer commented Feb 22, 2018

manveru commented Feb 22, 2018

manveru commented Feb 22, 2018

matthiasbeyer commented Feb 22, 2018

dtzWill commented Feb 22, 2018

manveru commented Feb 22, 2018

dtzWill commented Feb 22, 2018

zimbatm Feb 22, 2018

manveru Feb 22, 2018

zimbatm Feb 22, 2018

zimbatm Feb 22, 2018

matthiasbeyer commented Mar 9, 2018

manveru commented Mar 9, 2018 •

edited

zimbatm commented Mar 9, 2018

zimbatm commented Mar 10, 2018 •

edited

manveru commented Mar 10, 2018

zimbatm commented Mar 10, 2018

dtzWill commented Mar 10, 2018

manveru commented Mar 11, 2018

matthiasbeyer commented Mar 12, 2018

manveru commented Mar 12, 2018 •

edited

matthiasbeyer commented Mar 21, 2018

matthiasbeyer commented Apr 25, 2018

mguentner commented Apr 25, 2018 •

edited

bb010g commented Jul 2, 2018

Ekleog commented Oct 1, 2018

matthiasbeyer commented Oct 1, 2018

tilpner commented Oct 1, 2018 •

edited

matthiasbeyer commented Mar 16, 2019

c0bw3b commented May 27, 2019 •

edited

manveru commented May 27, 2019

Patchwork #35334

Patchwork #35334

Conversation

manveru commented Feb 22, 2018 • edited

Motivation for this change

Things done

manveru commented Feb 22, 2018

manveru commented Feb 22, 2018

matthiasbeyer commented Feb 22, 2018

manveru commented Feb 22, 2018

manveru commented Feb 22, 2018

matthiasbeyer commented Feb 22, 2018

dtzWill commented Feb 22, 2018

manveru commented Feb 22, 2018

dtzWill commented Feb 22, 2018

zimbatm Feb 22, 2018

Choose a reason for hiding this comment

manveru Feb 22, 2018

Choose a reason for hiding this comment

zimbatm Feb 22, 2018

Choose a reason for hiding this comment

zimbatm Feb 22, 2018

Choose a reason for hiding this comment

matthiasbeyer commented Mar 9, 2018

manveru commented Mar 9, 2018 • edited

zimbatm commented Mar 9, 2018

zimbatm commented Mar 10, 2018 • edited

manveru commented Mar 10, 2018

zimbatm commented Mar 10, 2018

dtzWill commented Mar 10, 2018

manveru commented Mar 11, 2018

matthiasbeyer commented Mar 12, 2018

manveru commented Mar 12, 2018 • edited

matthiasbeyer commented Mar 21, 2018

matthiasbeyer commented Apr 25, 2018

mguentner commented Apr 25, 2018 • edited

bb010g commented Jul 2, 2018

Ekleog commented Oct 1, 2018

matthiasbeyer commented Oct 1, 2018

tilpner commented Oct 1, 2018 • edited

matthiasbeyer commented Mar 16, 2019

c0bw3b commented May 27, 2019 • edited

manveru commented May 27, 2019

manveru commented Feb 22, 2018 •

edited

manveru commented Mar 9, 2018 •

edited

zimbatm commented Mar 10, 2018 •

edited

manveru commented Mar 12, 2018 •

edited

mguentner commented Apr 25, 2018 •

edited

tilpner commented Oct 1, 2018 •

edited

c0bw3b commented May 27, 2019 •

edited