Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatpak #33371

Merged
merged 11 commits into from May 15, 2018
Merged

Flatpak #33371

merged 11 commits into from May 15, 2018

Conversation

jtojnar
Copy link
Contributor

@jtojnar jtojnar commented Jan 3, 2018

Motivation for this change

Allows running sandboxed applications.

Closes: #32807

How to test

Enable the service by adding services.flatpak.enable = true; to your configuration.nix

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak update
flatpak search bustle
flatpak install flathub org.freedesktop.Bustle
flatpak run org.freedesktop.Bustle
Things done

Installing and running applications works, we still need to check whether all the portals and interactions work. Portals work as well.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@jtojnar
Copy link
Contributor Author

jtojnar commented Jan 3, 2018

When running org.freedesktop.Bustle, I see:

flatpak run: Can't get document portal: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.portal.Documents was not provided by any .service files
/lib/libc.so.6: version `GLIBC_2.25' not found (required by /nix/store/5pbbfiwq7lvg6jq8iplfj9gw3jp35g9a-gnutls-3.6.1/lib/libgnutls.so.30)
Failed to load module: /nix/store/cahyfwfc2hznx61sqz2j7p0a6f07n3aw-glib-networking-2.54.1/lib/gio/modules/libgiognutls.so

@jtojnar
Copy link
Contributor Author

jtojnar commented Jan 3, 2018

Weird, if I run /nix/store/f8algjxyc5dal7jls16rnax7c5ibbhqx-flatpak-0.10.2.1/libexec/xdg-document-portal manually, it works. For some reason the service does not start automatically.

There is also another message from org.gnome.Photos:

Tracker-Message: Could not get mtime for 'file:///home/jtojnar/.var/app/org.gnome.Photos/cache/tracker/meta.db': Error when getting information for file “/home/jtojnar/.var/app/org.gnome.Photos/cache/tracker/meta.db”: No such file or directory

(gnome-photos:2): Tracker-WARNING **: Falling back to bus backend, the direct backend failed to initialize: Could not open sqlite3 database:'/home/jtojnar/.var/app/org.gnome.Photos/cache/tracker/meta.db': unable to open database file

Edit: Upstream bug: https://gitlab.gnome.org/GNOME/gnome-photos/issues/51

@Mounium
Copy link
Contributor

Mounium commented Feb 2, 2018

I used to be able to run flatpak flawlessly with your commits (not sure whether there was an update to them since, I usually just use current trunk with some cherry-picked commits, like yours) but for quite some time now I only get error: Failed to execute child process ?bwrap? (No such file or directory) when I try to do flatpak run with any package (install, remote-add works just fine). I've even tried deleting the whole flatpak directory in /var but it didn't help either. Can I help somehow, any useful logs I'm supposed to post?

@jtojnar
Copy link
Contributor Author

jtojnar commented Feb 2, 2018

@Mounium Sorry about that, I switched from the vendored bubblewrap to the one in Nix and did not check if it works. (I though bubblewrap is used as a library, which would be linked correctly, but it uses the bwrap binary). Should be fixed now.

@Mounium
Copy link
Contributor

Mounium commented Feb 4, 2018

I have been using it in the last couple of days, most flatpak packages work just fine, awesome work 👍

One strange package, pulseeffects doesn't seem to work, or at least not in a stable manner (wwmm/easyeffects#156), as I can get it to work from within the gnome software center, but not from command line, probably due to some environment variable set by the wrapper script on gnome-software.

@GrahamcOfBorg GrahamcOfBorg added 6.topic: GNOME GNOME desktop environment and its underlying platform 8.has: module (update) labels Feb 4, 2018
@jtojnar
Copy link
Contributor Author

jtojnar commented Feb 13, 2018

I tested the portals with https://github.com/matthiasclasen/portal-test (follow the changes in pull requests) and now they work. The last remaining thing is adding an automated test.

I created a prototype but the portal-test does not build because it does not have access to GNOME platform and SDK. We cannot fetch it due to sandboxing, we will need fetchostree that will produce a fixed-output derivation like fetchgit does.

Test prototype
--- a/nixos/release.nix
+++ b/nixos/release.nix
@@ -255,6 +255,7 @@
   tests.env = callTest tests/env.nix {};
   tests.ferm = callTest tests/ferm.nix {};
   tests.firefox = callTest tests/firefox.nix {};
+  tests.flatpak = callTest tests/flatpak.nix {};
   tests.firewall = callTest tests/firewall.nix {};
   tests.fleet = callTestOnTheseSystems ["x86_64-linux"] tests/fleet.nix {};
   #tests.gitlab = callTest tests/gitlab.nix {};
--- /dev/null
+++ b/nixos/tests/flatpak-portal-test.patch
@@ -0,0 +1,42 @@
+--- a/flatpak/build.sh
++++ b/flatpak/build.sh
+@@ -1,3 +1,3 @@
+ #!/bin/sh
+ 
+-flatpak-builder --force-clean --ccache --require-changes --repo=repo app org.gnome.PortalTest.json
++flatpak-builder --force-clean --require-changes --repo=repo app org.gnome.PortalTest.json
+--- a/flatpak/org.gnome.PortalTest.json
++++ b/flatpak/org.gnome.PortalTest.json
+@@ -10,23 +10,6 @@
+         "--socket=pulseaudio"
+     ],
+     "modules": [
+-        {
+-            "name": "meson",
+-            "no-autogen": true,
+-            "cleanup": [ "/bin", "/lib" ],
+-            "sources": [
+-                {
+-                    "type": "archive",
+-                    "url": "https://github.com/mesonbuild/meson/releases/download/0.36.0/meson-0.36.0.tar.gz",
+-                    "sha256": "dc087ec40dacb5e256e6ee6467f2d004faf4ef284d3c1ce5e89faa1e16540950"
+-                },
+-                {
+-                    "type": "file",
+-                    "path": "meson-makefile",
+-                     "dest-filename": "Makefile"
+-                }
+-            ]
+-        },
+         {
+             "name": "portal-test",
+             "buildsystem": "meson",
+@@ -34,7 +17,7 @@
+             "sources": [
+                 {
+                     "type": "git",
+-                    "url": "https://github.com/matthiasclasen/permission-viewer.git"
++                    "url": ".."
+                 }
+             ]
+         }
--- /dev/null
+++ b/nixos/tests/flatpak.nix
@@ -0,0 +1,48 @@
+import ./make-test.nix ({ pkgs, ... }:
+
+let
+  portal-test = with pkgs; stdenv.mkDerivation {
+    name = "portal-test";
+    src = fetchFromGitHub {
+      owner = "matthiasclasen";
+      repo = "portal-test";
+      rev = "a2468457cee72b86be51d8a0ee5911c37d6894c2";
+      sha256 = "00c7wcjawh80f9vfqm187hmv325ghl9c6r620iq4bf7q4sqj1ab6";
+    };
+    phases = [ "unpackPhase" "patchPhase" "buildPhase" "installPhase" ];
+    patches = [ ./flatpak-portal-test.patch ];
+    buildInputs = [ flatpak-builder gitMinimal libdwarf ];
+    buildPhase = ''
+      cd flatpak
+      ./build.sh
+    '';
+    installPhase = ''
+      cp -r repo $out/repo
+    '';
+  };
+in {
+  name = "flatpak";
+  meta = {
+    maintainers = pkgs.flatpak.meta.maintainers;
+  };
+
+  machine =
+    { config, pkgs, ... }:
+
+    {
+      imports = [ ./common/x11.nix ];
+      services.flatpak.enable = true;
+    };
+
+  testScript =
+    ''
+      $machine->waitForX;
+      $machine->execute("flatpak remote-add --user --no-gpg-verify portal-test ${portal-test}/repo");
+      $machine->execute("flatpak install --user portal-test org.gnome.PortalTest");
+      $machine->execute("flatpak run org.gnome.PortalTest");
+      $machine->waitForWindow(qr/Portal Test/);
+      $machine->succeed("xwininfo -root -tree | grep 'Portal Test'");
+      $machine->screenshot("screen");
+    '';
+
+})

Sources:

@jtojnar jtojnar added this to the 18.03 milestone Feb 15, 2018
@jtojnar jtojnar force-pushed the flatpak branch 2 times, most recently from 32f36fc to 8d31005 Compare February 16, 2018 11:23
@jtojnar
Copy link
Contributor Author

jtojnar commented Feb 22, 2018

Since the last time, I tried switching to installed tests which look more promising than the test prototype. Unfortunately, they try to create a runtime by copying files from /usr/lib etc. This will need to be replaced.

@peterhoeg
Copy link
Member

There is also xdg-desktop-portal-kde as part of KDE applications.

@jtojnar
Copy link
Contributor Author

jtojnar commented Feb 28, 2018

More path patching and adding some dependencies fixed most of the tests:

SUMMARY: total=16; passed=9; skipped=0; failed=7; user=279.1s; system=38.6s; maxrss=507736
FAIL: Flatpak/test-bundle.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-extensions.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-run-deltas.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-run-system.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-run-system-deltas.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-bundle-system.sh.test (Child process exited with code 1)
FAIL: Flatpak/test-run.sh.test (Child process exited with code 1)
error: command `gnome-desktop-testing-runner' did not succeed (exit code 2)

The seven remaining tests depend on generating a platform, which I did by copying a Nix store closure to the export $DIR and adding symlinks from FHS in the $DIR. This however fails, as flatpak build-export seems to discard files outside $DIR/usr. I have opened an upstream issue asking about that.

@GrahamcOfBorg
Copy link

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

shrinking RPATHs of ELF executables and libraries in /nix/store/i869w73vh1g1xakcvkl6dmjvsp2319mx-flatpak-builder-0.10.10-man
gzipping man pages under /nix/store/i869w73vh1g1xakcvkl6dmjvsp2319mx-flatpak-builder-0.10.10-man/share/man/
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/i869w73vh1g1xakcvkl6dmjvsp2319mx-flatpak-builder-0.10.10-man
checking for references to /build in /nix/store/i869w73vh1g1xakcvkl6dmjvsp2319mx-flatpak-builder-0.10.10-man...
/nix/store/47qnd74gyfgpphy47ad1w3ag25ap9rzf-flatpak-0.11.7
/nix/store/pj8gzxjwhgdpda63caas5v9a03xvwrjj-flatpak-builder-0.10.10
/nix/store/9jkvcaxj7gvs451h8822pwm03x006gb7-gnome-software-3.28.1
/nix/store/fq2lgygl84swnlx34zqnzy27qz95vkvk-xdg-desktop-portal-0.11
/nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11

When a package contains a directory in one of the systemd directories
(like flatpak does), it is symlinked into the *-units derivation.
Then later, the derivation will try to create the directory, which
will fail:

mkdir: cannot create directory '/nix/store/…-user-units/dbus.service.d': File exists
builder for '/nix/store/…-user-units.drv' failed with exit code 1

Closes: NixOS#33233
@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

shrinking /nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11/libexec/xdg-desktop-portal-gtk
strip is /nix/store/gp7fylxwn18b7pl2c18ks89hsiaxyfvf-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11/lib  /nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11/libexec
patching script interpreter paths in /nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11
checking for references to /build in /nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11...
/nix/store/4q8zlfq383gz6n91vlh9wzxr76v4n13m-flatpak-0.11.7
/nix/store/15zk9y06l2yz3bndwj7d1b8qfkin7x1q-flatpak-builder-0.10.10
/nix/store/xxbmr0qg39ag6srb8wvg7ydibva5742h-gnome-software-3.28.1
/nix/store/ymxjiqd4px3y27x6bj61w7gpivrzx6ji-xdg-desktop-portal-0.11
/nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11

@GrahamcOfBorg
Copy link

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

/nix/store/4q8zlfq383gz6n91vlh9wzxr76v4n13m-flatpak-0.11.7
/nix/store/15zk9y06l2yz3bndwj7d1b8qfkin7x1q-flatpak-builder-0.10.10
/nix/store/xxbmr0qg39ag6srb8wvg7ydibva5742h-gnome-software-3.28.1
/nix/store/ymxjiqd4px3y27x6bj61w7gpivrzx6ji-xdg-desktop-portal-0.11
/nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

shrinking /nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11/libexec/xdg-desktop-portal-gtk
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11/lib  /nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11/libexec
patching script interpreter paths in /nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11
checking for references to /build in /nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11...
/nix/store/47qnd74gyfgpphy47ad1w3ag25ap9rzf-flatpak-0.11.7
/nix/store/pj8gzxjwhgdpda63caas5v9a03xvwrjj-flatpak-builder-0.10.10
/nix/store/9jkvcaxj7gvs451h8822pwm03x006gb7-gnome-software-3.28.1
/nix/store/fq2lgygl84swnlx34zqnzy27qz95vkvk-xdg-desktop-portal-0.11
/nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11

services.xserver.desktopManager.gnome3.enable = true; # TODO: figure out minimal environment where the tests work
services.flatpak.enable = true;
environment.systemPackages = with pkgs; [ gnupg gnome-desktop-testing ostree python2 ];
virtualisation.memorySize = 2048;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want the test to run on i686, this should be 2047 due to a memory limitation in qemu-system-i386

@jtojnar jtojnar merged commit d5060ac into NixOS:master May 15, 2018
GNOME automation moved this from Ready to Done May 15, 2018
@jtojnar jtojnar deleted the flatpak branch May 15, 2018 12:19
@GrahamcOfBorg
Copy link

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

/nix/store/47qnd74gyfgpphy47ad1w3ag25ap9rzf-flatpak-0.11.7
/nix/store/pj8gzxjwhgdpda63caas5v9a03xvwrjj-flatpak-builder-0.10.10
/nix/store/9jkvcaxj7gvs451h8822pwm03x006gb7-gnome-software-3.28.1
/nix/store/fq2lgygl84swnlx34zqnzy27qz95vkvk-xdg-desktop-portal-0.11
/nix/store/cr2d72nj7baan815qxzpwr3sc9dh5g78-xdg-desktop-portal-gtk-0.11

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: flatpak, flatpak-builder, gnome3.gnome-software, xdg-desktop-portal, xdg-desktop-portal-gtk

Partial log (click to expand)

/nix/store/4q8zlfq383gz6n91vlh9wzxr76v4n13m-flatpak-0.11.7
/nix/store/15zk9y06l2yz3bndwj7d1b8qfkin7x1q-flatpak-builder-0.10.10
/nix/store/xxbmr0qg39ag6srb8wvg7ydibva5742h-gnome-software-3.28.1
/nix/store/ymxjiqd4px3y27x6bj61w7gpivrzx6ji-xdg-desktop-portal-0.11
/nix/store/9aafq37xfr9a6q4q6bag6g3vfkk76mfa-xdg-desktop-portal-gtk-0.11

@ghost
Copy link

ghost commented Jan 15, 2019

It'd be nice if users could use Flatpak too. As it is now, any operations having to do with Flatpak have to be done as root making Flatpak support a non-start for me.

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
error: Flatpak system operation ConfigureRemote not allowed for user

@hedning
Copy link
Contributor

hedning commented Jan 15, 2019

Have you tried to supply the --user flag? I'm at least able to install using my user account, though I can't quite remember if I had to do something special.

@alexeymuranov
Copy link
Contributor

On NixOS i am using flatpak without sudo and without --user.

@ghost
Copy link

ghost commented Jan 15, 2019

@hedning, the --user flag works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
GNOME
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

8 participants