[wip] appimageTools: add a new buildAppImage wrapper #82158

bignaux · 2020-03-09T15:41:21Z

Motivation for this change

Provide a new bundle of appimage without pollute the nixpkgs with small appimageTools.wrapType derivation.
in a first time, discussion was about adding a recurseIntoAttrs method and a ./pkgs/top-level/appimage-packages.nix on the example of haxe-packages.nix . Now we add a new method on appimageTools.

will close #38037 , #76849 ...

Things done

cc @worldofpeace
cc @jtojnar
cc @tilpner
cc @Mic92
cc @dtzWill

edolstra · 2020-03-09T16:22:32Z

What's the use case exactly? Nixpkgs might not be the best vehicle for distributing appimages.

bignaux · 2020-03-09T16:28:42Z

@edolstra : the use case is people trying to manage their appimage with nixpkgs, of generally hard to package or closed source software, and we spend time reviewing and writing such appimages that could be acceptable in such way i guess.

@GrahamcOfBorg build appimagePackages.retroshare

bignaux · 2020-03-09T16:35:08Z

@GrahamcOfBorg build appimagePackages.colobot

bignaux · 2020-03-09T16:42:14Z

Btw there are 1800 PR in nixpkgs, we don't lack of reviewing, so if people can wait for a difficult app to be package and merge this way, i think it doesn't hurt. I think the opposite, there is a choice to start using an appimage that is identify as it (prefixed) then do a better integration at time.

Mic92 · 2020-03-09T17:36:27Z

pkgs/top-level/appimage-packages.nix

+  self = appimagePackages;
+  appimagePackages = with self; {
+
+  buildAppImage = {


I would prefer to just put packages in to top-level/all-packages.nix rather than one big expression file. Hiding packages in attribute set also might lead to people packaging an application twice. After all packages should be built without using app images if possible. Having an abstraction is ok, but currently it does not seem to provide enough to justify the abstraction.

"all packages should be built without using app images if possible"

we are very agree on that, but you know it's not always possible.

currently it does not seem to provide enough to justify the abstraction.

I made all my possible to put the interesting stuff in appimageTools instead, but i think i get something useful here too, we still can someapp = appimagePackages.someapp to top-level/all-packages.nix , people would identify clearly it's not a native one.

I think it makes most sense to merge it as is for now and introduce other features as needed, i've some on plan but not yet ready.

Ok. Make it obvious that by doing someapp = appimagePackages.someapp looks like a good idea on the second thought - we should do this for all packages with no exception for visibility reasons though. However I would use callPackage in appimage-packages.nix for all in this for two reasons: It allows to override dependencies using override and one file per package is easier to maintain. We had at some point a huge pkgs/top-level/python-packages.nix which was not fun at all.

yes the main interest was to simplify the maintainer and package work . Perhaps i better move this buildappimage to appimagetools, and let people creating an one file per package, but the drawback is it's harder to maintain for me as author of buildAppImage. since i've some other nixpkgs job on the fire, i put this on wip again, let's this maturate.

can't we have buildAppImage provided as a library function implementing all the somewhat repetitive bits of packaging a appimage as a nix derivation, and then still callPackage it from all-packages.nix?

I still like to be able to have packages in all-packages.nix, and be able to build them by building the top-level attribute.
Also, I like all the metadata we have, and that each package is in its own directory.

I still don't really see a point in a separate appimage-packages.nix, and having them inside a appimagePackages.* namespace - I'd say if there's some software only (reasonably) obtainable by extracting it from an appimage, then package it by extracting the appimage, but let's not make the way these are added into nixpkgs more alien than another regular package.

We already do that for some software, be it extracting a .deb, snap or whatever.

Since we have ~150 recurseIntoAttrs in all-packages.nix, think in a first time that was possible to use that possibility for simplest appimages (not all). But a drawback is to not be able to use other callPackages like qt5.callPackage ( deprecated ? https://nixos.org/nix-dev/2017-February/022900.html ), libsForQt5 ... but i think that's not a real issue since this should make it unnecessary by the buildFHSUserEnv. ( what about python3Packages ? pkgsi686Linux ? ) . So i wonder if i have interest to provide a appimages.callPackage . So for the moment i will just move buildAppImage in appimageTools.

Mic92 · 2020-03-09T17:37:18Z

pkgs/top-level/appimage-packages.nix

+      } // attrs.meta;
+  });
+
+  beakerbrowser = buildAppImage rec {


Are you using any of these packages? I would prefer if we have a maintainer and actual user who keeps the packages up to date/in a working state.

i need examples to make the thing, and i took some that will find maintainer soon, @kamilchm ... or people needs it or colobot i just think was fun to provide as it to children.

bignaux · 2020-03-09T17:56:31Z

Please, take in count that i'm working a lot on the appimage integration on nixpkgs, and i can give some hints why it's cleaner when we have such things in our repo :

pkgs/development/tools/unityhub/default.nix => no version because was not forced (so current sha is broken @tesq0 ).
pkgs/applications/editors/standardnotes/default.nix => pollute HOME since it use appimage-run (i fix it in standardnotes: 3.0.15 -> 3.3.3 #81812 .
soulseekqt: fix icons, change unpack method, update homepage, more generic appimage handling #75392 my own soulseekqt package, month to merge
ripcord: init at 0.4.23 #80402 that would work as it is, still in PR, lot of job for a closed chat software.
Packaging request: beaker browser #38037 people won't contribute, step to high...

i know there still improvement to be done, specially automatise desktop integration in installPhase or extraInstallCommands, but the best is ennemy of the good.

flokli · 2020-03-11T23:47:44Z

@bignaux please keep this PR focused on the actual thing it's proposing - otherwise it becomes harder to review.

As explained in #82158 (comment), I don't really see the point of adding a appimage-packages.nix, but some shared tooling that helps avoiding some of the repetitive boilerplate when building a package out an appimage might help -

I'd propose closing this PR, and refactoring some of the existing derivations that already extract appimages to use the tooling you propose. That'd make it easier to understand what it'd be improving.

bignaux · 2020-03-12T00:00:14Z

i'd propose you to not close it since it is done by main author of appimagetools and 3rd of https://github.com/AppImage/AppImageKit/graphs/contributors ...

flokli · 2020-03-12T01:03:15Z

Well, it was a suggestion. You're the author of the PR, so you can close it for yourself, if you agree. If you disagree, you can keep it open ;-)

I just wanted to clarify this PR (as it is currently) is about adding a appimage-packages.nix file, which is IMHO not the way we usually package packages in nixpkgs.

Creating nix packages from appimages is one of the many ways we obtain software, next to fetching from source and building, extracting binary tarballs, extracting debian packages, extracting snaps, … - while we of course prefer building from source where possible.

I don't think nixpkgs should be an appimage registry - hence my reservation against a appimage-packages.nix - but I think there might be some code duplication in how we extract from appimages, and moving some common code into a library function might be beneficial (and for the sake of clarity, that should probably go into another PR).

edolstra · 2020-03-12T08:39:23Z

This sounds like something that should be in a separate repository (or eventually, flake). The monolithic nixpkgs repo is already too big and too much of a bottleneck to contributors.

Mic92 · 2020-03-12T11:49:22Z

I would agree that we should not mirror every possible appimage applications in existence especially if we have them as a normal package already. However as I understand the author, it is more about improving our existing appimage infrastructure in order to support packages that we would not have otherwise at all in NixOS, since they would be too hard to package.

flokli · 2020-03-12T13:41:39Z

Well, in that case this should be a new library function, plus some changes to the existing packages to make use of that function - instead of the addition of appimage-packages.nix with some "template packages", as proposed in this PR.

Mic92 · 2020-03-12T13:43:36Z

Ok. Let's follow this approach.

bignaux · 2020-03-12T14:04:12Z

I'd prefer change the title and continue on that branch since this issue/PR have been linked on others threads ... I don't see any good reason to close it. Where i did the stuff to simplify the appimage packaging is not so important compare to the real discussion i'd have (that never happens anyway, each time it's only review trailing whitespace) is about what user need in it, and you seems not to be the users.

flokli · 2020-03-12T16:35:49Z

@bignaux can you then update the PR title and description with what this is about?

bignaux · 2020-03-12T16:46:18Z

Yes i'll do it quickly as i split and rewrite the PR.
Edit : a first drawback is that is less easy to let people choose between a broken retroshare and a working appimage retroshare cc @domenkozar (perhaps you can fix it now there is 0.6.5 ?).

worldofpeace · 2020-03-30T01:37:22Z

This contributor has since been banned from the org, closing.

bignaux force-pushed the appimage-packages.nix branch from 45ee8eb to 1e5ac3e Compare March 9, 2020 16:26

bignaux force-pushed the appimage-packages.nix branch from 1e5ac3e to e2fb3e1 Compare March 9, 2020 16:33

bignaux changed the title ~~[wip] top-level: add appimage-packages.nix~~ top-level: add appimage-packages.nix Mar 9, 2020

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Mar 9, 2020

Mic92 reviewed Mar 9, 2020

View reviewed changes

bignaux changed the title ~~top-level: add appimage-packages.nix~~ [wip] top-level: add appimage-packages.nix Mar 11, 2020

Mic92 closed this Mar 12, 2020

Mic92 reopened this Mar 12, 2020

flokli added the 9.needs: reporter feedback label Mar 12, 2020

bignaux changed the title ~~[wip] top-level: add appimage-packages.nix~~ [wip] appimageTools: add a new buildAppImage wrapper Mar 12, 2020

appimageTool: add build-appimage.nix

572948e

bignaux force-pushed the appimage-packages.nix branch from e2fb3e1 to 572948e Compare March 12, 2020 20:42

worldofpeace closed this Mar 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wip] appimageTools: add a new buildAppImage wrapper #82158

[wip] appimageTools: add a new buildAppImage wrapper #82158

bignaux commented Mar 9, 2020 •

edited

edolstra commented Mar 9, 2020

bignaux commented Mar 9, 2020 •

edited

bignaux commented Mar 9, 2020

bignaux commented Mar 9, 2020 •

edited

Mic92 Mar 9, 2020 •

edited

bignaux Mar 9, 2020 •

edited

bignaux Mar 10, 2020

Mic92 Mar 11, 2020

bignaux Mar 11, 2020

flokli Mar 11, 2020

bignaux Mar 12, 2020

Mic92 Mar 9, 2020

bignaux Mar 9, 2020

bignaux commented Mar 9, 2020 •

edited

flokli commented Mar 11, 2020

bignaux commented Mar 12, 2020

flokli commented Mar 12, 2020

edolstra commented Mar 12, 2020

Mic92 commented Mar 12, 2020 •

edited

flokli commented Mar 12, 2020

Mic92 commented Mar 12, 2020

bignaux commented Mar 12, 2020 •

edited

flokli commented Mar 12, 2020

bignaux commented Mar 12, 2020 •

edited

worldofpeace commented Mar 30, 2020

[wip] appimageTools: add a new buildAppImage wrapper #82158

[wip] appimageTools: add a new buildAppImage wrapper #82158

Conversation

bignaux commented Mar 9, 2020 • edited

Motivation for this change

Things done

edolstra commented Mar 9, 2020

bignaux commented Mar 9, 2020 • edited

bignaux commented Mar 9, 2020

bignaux commented Mar 9, 2020 • edited

Mic92 Mar 9, 2020 • edited

Choose a reason for hiding this comment

bignaux Mar 9, 2020 • edited

Choose a reason for hiding this comment

bignaux Mar 10, 2020

Choose a reason for hiding this comment

Mic92 Mar 11, 2020

Choose a reason for hiding this comment

bignaux Mar 11, 2020

Choose a reason for hiding this comment

flokli Mar 11, 2020

Choose a reason for hiding this comment

bignaux Mar 12, 2020

Choose a reason for hiding this comment

Mic92 Mar 9, 2020

Choose a reason for hiding this comment

bignaux Mar 9, 2020

Choose a reason for hiding this comment

bignaux commented Mar 9, 2020 • edited

flokli commented Mar 11, 2020

bignaux commented Mar 12, 2020

flokli commented Mar 12, 2020

edolstra commented Mar 12, 2020

Mic92 commented Mar 12, 2020 • edited

flokli commented Mar 12, 2020

Mic92 commented Mar 12, 2020

bignaux commented Mar 12, 2020 • edited

flokli commented Mar 12, 2020

bignaux commented Mar 12, 2020 • edited

worldofpeace commented Mar 30, 2020

bignaux commented Mar 9, 2020 •

edited

bignaux commented Mar 9, 2020 •

edited

bignaux commented Mar 9, 2020 •

edited

Mic92 Mar 9, 2020 •

edited

bignaux Mar 9, 2020 •

edited

bignaux commented Mar 9, 2020 •

edited

Mic92 commented Mar 12, 2020 •

edited

bignaux commented Mar 12, 2020 •

edited

bignaux commented Mar 12, 2020 •

edited