at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: updates, build with meson, adding updareScript #41365

Anton-Latukha · 2018-06-01T16:34:15Z

Please, let's merge this progress.
I gathered information and would open new pull request on new work.

Motivation for this change

Number of projects support Meson builds.
This would make Hydra rebuilds faster for this packages.
Also doing version updates on these packages.
Light linting of this packages
Since many of these packages are under GNOME umbrella - migrating them to use fresh GNOME GitLab.

Packages

Notes on encountered bugs

gnome-logs (GNOME)

Can not find meson.build with fetchurl
Upon migration on GitLab AND Meson:

Running custom install script '/usr/bin/env python3 /build/source/meson_post_install.py'
Failed to run install script '/usr/bin/env python3 /build/source/meson_post_install.py'
FAILED: meson-install
/nix/store/yshj6nmgf5zl9yyhw9qxdjkfmz0k1ijh-python3-3.6.5/bin/python3.6m /nix/store  /ayjq6cvqm0a3yd615ksv3va2a455vq5b-meson-0.46.1/bin/meson --internal install /build/source/build/meson-private/install.dat
ninja: build stopped: subcommand failed.

gnome-nettool (GNOME)
- Meson build not seen stable release
grilo (GNOME)
- Can not find meson.build with fetchurl
- Made bugreport: https://gitlab.gnome.org/GNOME/grilo/issues/4
pango
- Can not find meson.build with fetchurl
gnome3.gtk
- Can not find meson.build with fetchurl

For later work

Things done

Tested using sandboxing
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

jtojnar · 2018-06-01T20:27:31Z

Using mirror://gnome should be preferred to fetchFromGitLab, as it will automatically choose geographically close mirror.

jtojnar · 2018-06-01T20:39:10Z

What does “Can not find configuration files with GitLab fetch” mean?
Opened an issue for gnome-logs: https://gitlab.gnome.org/GNOME/gnome-logs/issues/25
What is gnome3-gtk?
systemd and zathura packages already use meson.
GNOME 2 is dead, no need to port it.

jtojnar · 2018-06-01T20:43:06Z

For fribidi, does adding mesonFlags = [ "-Ddocs=false" ]; as suggested by the error message work?

Anton-Latukha · 2018-06-01T20:48:58Z

Reverted commits.

Anton-Latukha · 2018-06-01T20:49:42Z

mesonFlags = [ "-Ddocs=false" ]; does not work:

checking for references to /build in /nix/store/6dmhgimanfkwa6bsmklcrhg8bz4ksb8f-fribidi-1.0.3...
strip is /nix/store/92d2ifxcni4n3zx9s8wnkcjlvnx5ajlc-binutils-2.30/bin/strip
builder for '/nix/store/wk0m4b3i7n07kmg8gg6278m9ynd532zg-fribidi-1.0.3.drv' failed to produce output path '/nix/store/agpvl0lqg9kphgh4mdbccrb8zj8aa0j4-fribidi-1.0.3-devdoc'
error: build of '/nix/store/wk0m4b3i7n07kmg8gg6278m9ynd532zg-fribidi-1.0.3.drv' failed

Anton-Latukha · 2018-06-01T20:56:38Z

Yes, systemd and zathura packages already uses meson.
I written a small script for the task, and it produced some false-positives.

jtojnar · 2018-06-01T21:10:09Z

@Anton-Latukha then we either need to remove devdoc from fribidi outputs, or package https://github.com/fribidi/c2man

Anton-Latukha · 2018-06-01T21:31:52Z

The behavior of fribidi upstream about c2man is unknown. They make remarks about it, but leave it, like here fribidi/fribidi#72, fribidi/fribidi#44, but say to disable documentation.

Anton-Latukha · 2018-06-01T21:35:21Z

I would go rest.

Created bugreport for core GNOME library grilo.
https://gitlab.gnome.org/GNOME/grilo/issues/4

Others would do later.

jtojnar · 2018-06-02T03:37:37Z

pkgs/development/libraries/fribidi/default.nix

  };

+  # FIXME: Please build with Meson after https://github.com/fribidi/fribidi/issues/79 solved
  nativeBuildInputs = [ autoreconfHook pkgconfig ];

  # Configure script checks for glib, but it is only used for tests.


glib dependency was removed i 1.0.2.

jtojnar · 2018-06-02T03:39:01Z

pkgs/development/libraries/fribidi/default.nix

-    rev = version;
-    sha256 = "10q5jfch5qzrj2w4fbkr086ank66plx8hp7ra9a01irj80pbk96d";
+    rev = "v${version}";
+    sha256 = "02483nscxc695j9b92clcdf0xb7xkfjry09kqdkkhkzl3vdcj039";


Maybe try fetchurl with "https://github.com/fribidi/fribidi/releases/download/v${version}/${name}.tar.bz2", it should have the manpage bundled.

I don't know how man pages should look in this packaging.
To me it seems like this tarball is the same deal. Build requires c2man OR mesonFlags = [ "-Ddocs=false" ]; and I see the in doc c2man input files.

I would elaborate more about the topic in that upstream bugreport fribidi/fribidi#79.

You are right. I think we can probably do without man pages for now.

jtojnar · 2018-06-02T10:17:04Z

Could you replace the underscore in the at-spi2-core commit message, so that @GrahamcOfBorg could recognize the package?

jtojnar · 2018-06-02T10:18:19Z

I tested several packages depending on fribidi (abiword, mlterm, translate-shell, fbreader, renpy and enligtement.efl), and they build fine.

Anton-Latukha · 2018-06-02T10:28:22Z

fribidi/fribidi#79 resulted in a dev PR that is waiting to merge there.
Meson checking c2man when shipped with docs, this gets dodged in PR.

c2man seems mostly only their dependency.

I would make changes.

Anton-Latukha · 2018-06-02T12:25:27Z

During my work:

Is our Xorg packages more granular in some way to other distributions?
https://www.archlinux.org/packages/extra/x86_64/at-spi2-core/

I see that not developers, not packages in other distros do not use: xextproto libSM libICE for this package.

Do we need include API documentation generation for the package?
As a general rule, do we include developer-related documentation.
Since I would include D-Bus:
Is this really needed:

nixpkgs/pkgs/development/libraries/at-spi2-core/default.nix

Lines 22 to 27 in c30bd1c

    
           # ToDo: on non-NixOS we create a symlink from there? 
        
           configureFlags = "--with-dbus-daemondir=/run/current-system/sw/bin/"; 
        
           doCheck = false; # needs dbus daemon

jtojnar · 2018-06-02T12:44:57Z

Is our Xorg packages more granular in some way to other distributions?

Not sure, but other distributions often omit transitive dependencies from packages’ dependency list. Nix needs to list all dependencies that are not propagated from other dependencies.

Though some of the dependencies might no longer be needed, for example dbus-glib is not (it was replaced by glib and dbus).

Do we need include API documentation generation for the package? As a general rule, do we include developer-related documentation.

Having a developer documentation is preferred, we use a devdoc output for that. Unfortunately, building it is often not trivial so packagers do not bother with it.

Since I would include D-Bus:
Is this really needed:

meson does not use configureFlags. The corresponding thing would be dbus_daemon flag, but it is not needed since dbus-daemon is auto-detected.

jtojnar · 2018-06-02T12:46:35Z

It would be also nice to add updateScript to at-spi2-core: #36150

Anton-Latukha · 2018-06-02T13:34:17Z

Yes. I tuned down some info, dbus, glib change.

Deps I asked about not needed I am removing them.

Updated package.

Created upstream bug report, they not updated install info for 8 years.

They have good documentation otherwise.

Would send changes, when get internet access.

Anton-Latukha · 2018-06-02T13:36:04Z

Since they are twins, I would review at-spi2-atk also.

Anton-Latukha · 2018-06-02T13:39:25Z

Thanks for support, lots of info, and you double my effort on most cases. It is hardcore.

jtojnar · 2018-06-04T07:40:23Z

pkgs/development/libraries/at-spi2-atk/default.nix


  meta = with stdenv.lib; {
+    description = "D-Bus bridge for Assistive Technology Service Provider Interface (AT-SPI) and Accessibility Toolkit (ATK)";
+    homepage = "https://gitlab.gnome.org/GNOME/at-spi2-atk";


No need to wrap the URL in quotes.

jtojnar · 2018-06-04T07:42:20Z

pkgs/development/libraries/at-spi2-atk/default.nix

  name = "${moduleName}-${versionMajor}.${versionMinor}";
+  moduleName   = "at-spi2-atk";
+  versionMajor = "2.26";
+  versionMinor = "2";

  src = fetchurl {
    url = "mirror://gnome/sources/${moduleName}/${versionMajor}/${name}.tar.xz";


It is better to use stdenv.lib.versions.majorMinor version so that the various update tools could update the single version attribute, see #36150.

jtojnar · 2018-06-04T07:43:50Z

pkgs/development/libraries/at-spi2-atk/default.nix

    platforms = platforms.unix;
+


While at it, could you also add a license and me as a maintainer?

Anton-Latukha · 2018-06-06T21:42:01Z

Then it says it does not know what is gnome3.
So I put it up in the head.

You say I need to do with gnome3;. Ok. Maybe I not tried this.

jtojnar · 2018-06-06T21:43:11Z

You need to add gnome3 argument to the top-level function.

Anton-Latukha · 2018-06-06T21:45:22Z

You marked it in reviews. So that is why I am asking how to do it in other way.

Anton-Latukha · 2018-06-06T21:53:35Z

It is strange to see update helper script in pkgs/desktop/gnome3 near definitions of packages, while script is used, but itself it is not a package.

Maybe it belongs to stdenv.lib.gnome3.updateScript in lib/gnome3/updateScript.nix.
Then gnome3 would not be needed in top level function to some core packages.

jtojnar · 2018-06-06T22:00:59Z

Perhaps I did not express clearly. Qualified names should be used instead of with statement, except in localized situations (e.g. meta attribute). majorMinor has fully qualified name stdenv.lib.versions.majorMinor, GNOME’s updateScript has gnome3.updateScript.

GNOME’s update script is in gnome3 package set because the scripts are only relevant to gnome packages. Again this is the locality principle.

Anton-Latukha · 2018-06-06T22:02:31Z

So this mean all this currently seems OK?

Anton-Latukha · 2018-06-06T22:06:05Z

fribidi does not have maintainers.
I use LANG=en_US.UTF-8 so I am not a candidate there.

clutter-gtk I left mostly untouched, it is important package and I decided to left it to maintainers who know the stack. It can have devdoc, probably this:

nixpkgs/pkgs/development/libraries/clutter-gtk/default.nix

Line 17 in dc48863

postBuild = "rm -rf $out/share/gtk-doc";

was before split-packages, if I know what I am talking about.

jtojnar · 2018-06-06T22:05:13Z

pkgs/development/libraries/fribidi/default.nix

-    sha256 = "10q5jfch5qzrj2w4fbkr086ank66plx8hp7ra9a01irj80pbk96d";
+  # NOTE: 2018-06-06 v1.0.4: Only URL tarball has "Have pre-generated man pages: true", which works-around upstream usage of rare ancient `c2man` fossil.
+  src = fetchurl {
+    url = "https://github.com/${pname}/${pname}/releases/download/v{$version}/${name}.tar.bz2";


This looks wrong

The URL or the comment? ;]

Ok, all further changes I going to do on the next day.

The {$version} should be ${version}.

jtojnar · 2018-06-06T22:06:44Z

pkgs/development/libraries/at-spi2-core/default.nix

+  passthru = {
+    updateScript = gnome3.updateScript {
+      packageName = pname;
+      versionPolicy = "none";


This can be removed as well.

Note: at-spi2-atk is version 2.26.2, while *this (at-spi2-core) is 2.28.0 version.

Yeah, but it still uses the default version policy.

Anton-Latukha · 2018-06-07T08:03:22Z

@jtojnar

Jan, thank you.

jtojnar · 2018-06-07T13:59:30Z

@GrahamcOfBorg build

Anton-Latukha · 2018-06-07T19:52:17Z

What it can be? Looks like bot did not built them.

https://monitoring.nix.ci/dashboard/db/ofborg?orgId=1&from=1528343308390&to=1528400908390

I am interested what it can be? We need to wait more, or bot drops/does not return some requests?

jtojnar · 2018-06-08T00:57:53Z

It sometimes requires to list the package names, not sure why

@GrahamcOfBorg build at-spi2-core at-spi2-atk clutter-gtk fribidi gnome3.lightsoff

GrahamcOfBorg · 2018-06-08T00:59:37Z

Success on x86_64-darwin (full log)

Attempted: at-spi2-core, at-spi2-atk, fribidi

The following builds were skipped because they don't evaluate on x86_64-darwin: clutter-gtk, gnome3.lightsoff

Partial log (click to expand)

Installing /private/tmp/nix-build-at-spi2-atk-2.26.2.drv-0/at-spi2-atk-2.26.2/build/atk-bridge-2.0.pc to /nix/store/hrbrxqf00vranv3z3fz9x3zj97whvyjs-at-spi2-atk-2.26.2/lib/pkgconfig
Installing /private/tmp/nix-build-at-spi2-atk-2.26.2.drv-0/at-spi2-atk-2.26.2/at-spi2-atk.desktop to /nix/store/hrbrxqf00vranv3z3fz9x3zj97whvyjs-at-spi2-atk-2.26.2/lib/gnome-settings-daemon-3.0/gtk-modules
glibPreFixupPhase
post-installation fixup
strip is /nix/store/8q05v40n0f7amgw70pk6bkb23r7hrzgn-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/hrbrxqf00vranv3z3fz9x3zj97whvyjs-at-spi2-atk-2.26.2/lib
patching script interpreter paths in /nix/store/hrbrxqf00vranv3z3fz9x3zj97whvyjs-at-spi2-atk-2.26.2
/nix/store/wn2d5zz7dp6kd8mjg07dywgs59avziag-at-spi2-core-2.28.0
/nix/store/hrbrxqf00vranv3z3fz9x3zj97whvyjs-at-spi2-atk-2.26.2
/nix/store/19m8mzy9vqqmjhg892a6fmlxqf689wz6-fribidi-1.0.4

GrahamcOfBorg · 2018-06-08T01:23:28Z

Success on x86_64-linux (full log)

Attempted: at-spi2-core, at-spi2-atk, clutter-gtk, fribidi, gnome3.lightsoff

Partial log (click to expand)

glibPreFixupPhase
hicolorPreFixupPhase
post-installation fixup
Wrapping program /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0/bin/lightsoff
shrinking RPATHs of ELF executables and libraries in /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0
shrinking /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0/bin/.lightsoff-wrapped
strip is /nix/store/4mf2xm9p32lzrim927yk92xhx35yaz62-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0/bin
patching script interpreter paths in /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0
checking for references to /build in /nix/store/g480s3qadzgpqjr8ipa86viqp3r49xm7-lightsoff-3.28.0...

GrahamcOfBorg · 2018-06-08T02:12:30Z

Success on aarch64-linux (full log)

Attempted: at-spi2-core, at-spi2-atk, clutter-gtk, fribidi, gnome3.lightsoff

Partial log (click to expand)

shrinking /nix/store/qxzw98vvqlgd6nk5q5j5yxkvqhz9x5lp-lightsoff-3.28.0/bin/.lightsoff-wrapped
strip is /nix/store/3ki967c3ij4zi1q9ycc3g1vfwr608nrb-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/qxzw98vvqlgd6nk5q5j5yxkvqhz9x5lp-lightsoff-3.28.0/bin
patching script interpreter paths in /nix/store/qxzw98vvqlgd6nk5q5j5yxkvqhz9x5lp-lightsoff-3.28.0
checking for references to /build in /nix/store/qxzw98vvqlgd6nk5q5j5yxkvqhz9x5lp-lightsoff-3.28.0...
/nix/store/xq43jmz0qxacc9v1m25j5k96snxhsfp1-at-spi2-core-2.28.0
/nix/store/xm381r7cimvvvdr8yv69zd5dkzgyi2z0-at-spi2-atk-2.26.2
/nix/store/7y4cn4sh7c9g75gi71qlsfd9hdaiwwa0-clutter-gtk-1.8.4
/nix/store/mijw0lrsxavi4gjvjqabfdxdpn1p1251-fribidi-1.0.4
/nix/store/qxzw98vvqlgd6nk5q5j5yxkvqhz9x5lp-lightsoff-3.28.0

jtojnar · 2018-06-08T10:31:57Z

Thank you.

GrahamcOfBorg added 10.rebuild-darwin: 101-500 10.rebuild-linux: 501+ labels Jun 1, 2018

Anton-Latukha mentioned this pull request Jun 1, 2018

Convert remaining GNOME packages to meson #32803

Closed

22 tasks

Anton-Latukha changed the title ~~[WIP] Meson builds, fetchFromGitLab for GNOME ones~~ Meson builds, fetchFromGitLab for GNOME ones Jun 1, 2018

Anton-Latukha force-pushed the migrateToMeson branch from 3c4b145 to 49c3aef Compare June 1, 2018 20:38

Anton-Latukha changed the title ~~Meson builds, fetchFromGitLab for GNOME ones~~ at-spi2-atk, at-spi2-core, clutter-gtk: build with meson Jun 1, 2018

jtojnar reviewed Jun 2, 2018

View reviewed changes

Anton-Latukha force-pushed the migrateToMeson branch from 6b571f6 to d03d219 Compare June 2, 2018 08:07

Anton-Latukha changed the title ~~at-spi2-atk, at-spi2-core, clutter-gtk: build with meson~~ at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: build with meson Jun 2, 2018

Anton-Latukha force-pushed the migrateToMeson branch from d03d219 to dbac6c8 Compare June 3, 2018 01:04

Anton-Latukha changed the title ~~at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: build with meson~~ [WIP] at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: build with meson Jun 3, 2018

jtojnar reviewed Jun 4, 2018

View reviewed changes

Anton-Latukha force-pushed the migrateToMeson branch from 160226c to d21d2f2 Compare June 6, 2018 21:58

jtojnar reviewed Jun 6, 2018

View reviewed changes

Anton-Latukha added 8 commits June 7, 2018 01:28

at-spi2-core: add updateScript hook

a381349

at-spi2-atk: build with meson

d663d63

at-spi2-atk: 2.26.1 -> 2.26.2, clean-up

ed2497f

at-spi2-atk: maintainers, license, clean-up

d4fe62d

at-spi2-atk: add updateScript hook

d4e5243

clutter-gtk: build with meson

45e5a49

fribidi: 0.19.7 -> 1.0.3

930db86

fribidi: build with meson, clean-up

1d84ebc

Anton-Latukha force-pushed the migrateToMeson branch from d21d2f2 to 1d84ebc Compare June 6, 2018 22:32

Anton-Latukha changed the title ~~at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: build with meson~~ at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: updates, build with meson, adding updareScript Jun 7, 2018

jtojnar merged commit 2890ff7 into NixOS:staging Jun 8, 2018

Anton-Latukha deleted the migrateToMeson branch June 8, 2018 15:21

Anton-Latukha mentioned this pull request Jun 10, 2018

NixOS: default to Gnome #41797

Closed

at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: updates, build with meson, adding updareScript #41365

at-spi2-atk, at-spi2-core, clutter-gtk, fribidi: updates, build with meson, adding updareScript #41365

Conversation

Anton-Latukha commented Jun 1, 2018 • edited

Motivation for this change

Packages

Notes on encountered bugs

For later work

Things done

jtojnar commented Jun 1, 2018

jtojnar commented Jun 1, 2018 • edited

jtojnar commented Jun 1, 2018

Anton-Latukha commented Jun 1, 2018

Anton-Latukha commented Jun 1, 2018

Anton-Latukha commented Jun 1, 2018

jtojnar commented Jun 1, 2018

Anton-Latukha commented Jun 1, 2018 • edited

Anton-Latukha commented Jun 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtojnar Jun 2, 2018 • edited

Choose a reason for hiding this comment

jtojnar commented Jun 2, 2018

jtojnar commented Jun 2, 2018

Anton-Latukha commented Jun 2, 2018

Anton-Latukha commented Jun 2, 2018 • edited

jtojnar commented Jun 2, 2018 • edited

jtojnar commented Jun 2, 2018

Anton-Latukha commented Jun 2, 2018

Anton-Latukha commented Jun 2, 2018 • edited

Anton-Latukha commented Jun 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anton-Latukha commented Jun 6, 2018 • edited

jtojnar commented Jun 6, 2018

Anton-Latukha commented Jun 6, 2018 • edited

Anton-Latukha commented Jun 6, 2018 • edited

jtojnar commented Jun 6, 2018

Anton-Latukha commented Jun 6, 2018

Anton-Latukha commented Jun 6, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anton-Latukha commented Jun 7, 2018

jtojnar commented Jun 7, 2018

Anton-Latukha commented Jun 7, 2018 • edited

jtojnar commented Jun 8, 2018

GrahamcOfBorg commented Jun 8, 2018

GrahamcOfBorg commented Jun 8, 2018

GrahamcOfBorg commented Jun 8, 2018

jtojnar commented Jun 8, 2018

Anton-Latukha commented Jun 1, 2018 •

edited

jtojnar commented Jun 1, 2018 •

edited

Anton-Latukha commented Jun 1, 2018 •

edited

jtojnar Jun 2, 2018 •

edited

Anton-Latukha commented Jun 2, 2018 •

edited

jtojnar commented Jun 2, 2018 •

edited

Anton-Latukha commented Jun 2, 2018 •

edited

Anton-Latukha commented Jun 6, 2018 •

edited

Anton-Latukha commented Jun 6, 2018 •

edited

Anton-Latukha commented Jun 6, 2018 •

edited

Anton-Latukha commented Jun 6, 2018 •

edited

Anton-Latukha commented Jun 7, 2018 •

edited