[WIP] Fix taffybar #40186

abbradar · 2018-05-08T19:17:19Z

Motivation for this change

Fix taffybar.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

This makes all the dependencies for taffybar build but taffybar itself fails with:

gcc: error trying to exec '/nix/store/ng5kbr6kmh979qaachifbsbblydrcmqc-gcc-7.3.0/libexec/gcc/x86_64-unknown-linux-gnu/7.3.0/collect2': execv: Argument list too long

Not sure how to fix this -- there may be duplicated arguments in configureFlags for Cabal, should we filter them somehow? @peti, any ideas?

For testing one can skip 76f629e and 7c698dd -- those are mass rebuild changes.

This was already used in Stack builder.

Fixed in 809c70d0594bc6cc898841ec3b2684c61822c9ba

It's needed to propagate GI dependencies.

puffnfresh · 2018-05-08T21:10:28Z

pkgs/development/haskell-modules/generic-builder.nix

@@ -214,6 +214,10 @@ stdenv.mkDerivation ({

  LANG = "en_US.UTF-8";         # GHC needs the locale configured during the Haddock phase.

+  # XXX: workaround for https://ghc.haskell.org/trac/ghc/ticket/11042.
+  LD_LIBRARY_PATH = stdenv.lib.makeLibraryPath (stdenv.lib.filter (x: x != null) systemBuildInputs);
+                    # ^^^ Internally uses `getOutput "lib"` (equiv. to getLib)


Is this comment is pointing out the definition of the function or is it something special?

Pointing out the definition -- I copied that from stack-builder.nix.

I'm sketched out by adding this. is ghci really needed? The rest looks good.

abbradar · 2018-05-08T21:21:17Z

@Ericson2314 I've noticed that hooks are executed several times for propagated build inputs in some cases. For example:

$ nix-shell -A gtk3
(shell) $ echo $PKG_CONFIG_PATH | grep -o /nix/store/fg6asqlsnc0wnv9fj6zp29w9aadbm6ir-libffi-3.2.1-dev/lib/pkgconfig | wc -l
4

From a quick look at our source code this shouldn't happen because we keep track of traversed packages. pkgconfig setup hook looks fairly simple. Any ideas (sorry for asking, you was the first I had in mind whom to ask modern stdenv stuff :D)?

Ericson2314 · 2018-05-08T22:26:13Z

@abbradar yes its because 469fd89 and some custom setup script with which makes various things two different types of deps. We really want to get revert that workaround but that would break packages with miscategorized deps.

abbradar · 2018-05-08T22:44:52Z

Yeah, this library loading is used by doctests. We have the same code in stack-builder.nix, I only copied it. This was supposed to be in answer to @Ericson2314's comment on ghci.

colonelpanic8 · 2018-05-08T23:26:24Z

any progress on the gcc error?

abbradar · 2018-05-08T23:44:53Z

@IvanMalison I don't see a good solution now; this could either be fixed in stdenv (removing duplication), in cc-wrapper (adding support for argument files) or maybe somehow in Cabal. The first way is, if I understood @Ericson2314, problematic; the second is doable but still constitutes a mass rebuild. I'm not sure what could we try to do in Cabal.

Ericson2314 · 2018-05-09T04:06:47Z

Ah doctests. Good use. I'd maybe mention that (as an example) explicitly. Yeah I want to teach *-wrapper to use response files, so that's a good approach. Cabal may in fact use them, but the wrappers need to unpack to use their black magic.

Ericson2314 · 2018-05-09T04:09:58Z

For the duplication, add crossConfig = true; to generic builder. I realize that looks super weird, I'll clean up in mass rebuild very soon.

peti · 2018-05-09T18:03:47Z

pkgs/development/haskell-modules/generic-builder.nix

@@ -214,6 +214,10 @@ stdenv.mkDerivation ({

  LANG = "en_US.UTF-8";         # GHC needs the locale configured during the Haddock phase.

+  # XXX: workaround for https://ghc.haskell.org/trac/ghc/ticket/11042.
+  LD_LIBRARY_PATH = stdenv.lib.makeLibraryPath (stdenv.lib.filter (x: x != null) systemBuildInputs);


I think it's a bad idea to set LD_LIBRARY_PATHS in the generic builder. The haskell-gi build is supposed to fail, honestly, because upstream calls their doctest suite with incorrect flags. I don't want us to add kludges to our code so that their broken code can succeed. The proper solution is to fix the haskell-gi build. I've committed 3e05301c94d72979da3c5f304196c7dd01f76a15 to accomplish that until a new release comes out.

abbradar · 2018-05-09T18:05:40Z

Ah, so it can be done with proper flags - I see now! Why is this used in Stack builder though? -- Nikolay.

peti · 2018-05-09T18:27:47Z

Why is this used in Stack builder though?

I have no clue what that stack builder is doing. :-/

peti · 2018-05-09T18:35:04Z

pkgs/development/haskell-modules/configuration-nix.nix

@@ -144,6 +144,9 @@ self: super: builtins.intersectAttrs super {
  gtk = disableHardening (addPkgconfigDepend (addBuildTool super.gtk self.gtk2hs-buildtools) pkgs.gtk2) ["fortify"];
  gtksourceview2 = addPkgconfigDepend super.gtksourceview2 pkgs.gtk2;
  gtk-traymanager = addPkgconfigDepend super.gtk-traymanager pkgs.gtk3;
+  gi-gdkx11 = addPkgconfigDepend super.gi-gdkx11 pkgs.gtk3;


I believe that this override is no longer necessary due to NixOS/cabal2nix@2923434. gi-gdkx11 compiles fine in haskell-updates without this override.

peti · 2018-05-09T18:36:41Z

pkgs/development/haskell-modules/configuration-nix.nix

@@ -144,6 +144,9 @@ self: super: builtins.intersectAttrs super {
  gtk = disableHardening (addPkgconfigDepend (addBuildTool super.gtk self.gtk2hs-buildtools) pkgs.gtk2) ["fortify"];
  gtksourceview2 = addPkgconfigDepend super.gtksourceview2 pkgs.gtk2;
  gtk-traymanager = addPkgconfigDepend super.gtk-traymanager pkgs.gtk3;
+  gi-gdkx11 = addPkgconfigDepend super.gi-gdkx11 pkgs.gtk3;
+  gi-dbusmenu = super.gi-dbusmenu.override { dbusmenu-glib = pkgs.libdbusmenu-glib; };
+  gi-dbusmenugtk3 = super.gi-dbusmenugtk3.override { dbusmenu-gtk3 = pkgs.libdbusmenu-gtk3; };


These two overrides should be unnecessary after NixOS/cabal2nix@d8bc043.

I think NixOS/cabal2nix#353 is also needed, unless upstream is an even better place.

abbradar · 2018-05-09T19:49:10Z

@peti Nice! That leaves us only with a libdbusmenu GTK propagation patch and some way to fix taffybar. I'm testing response files patch for cc-wrapper now, if that doesn't help I'll try crossConfig = true too. @Ericson2314 what is this flag? I mean, what are the other possible consequences if we enable it for the whole Haskell builder?

Ericson2314 · 2018-05-09T21:45:34Z

@abbradar see 469fd89 and the new conditions based on crossConfig. It was just a convenient way to detect cross compilation from bash. Longer term I want to rename those to use some new var like strictDeps and define strictDeps = buildPlatform != hostPlatform; in nixpkgs. So tl;dr don't worry about crossConfig other than the usages in that commit.

abbradar · 2018-05-10T07:02:15Z

It seems response files don't fix this because gcc itself calls binaries with too long arguments:

using a response file
gcc: error trying to exec '/nix/store/1c65cfy34vqgkfavd0hlvcsyj1qdax04-gcc-7.3.0/libexec/gcc/x86_64-unknown-linux-gnu/7.3.0/collect2': execv: Argument list too long

So... what do we do from here?

EDIT: reading gcc's source code -- there's no visible way to force it to pass arguments to its child processes via a file too.

srhb · 2018-05-11T11:48:53Z

@abbradar fwiw, that error message can be a little misleading, if you didn't know: Everything in the environment is counted as well as the command line arguments. If the content of the variables was just stuck in response files but the variables themselves never cleared, they still count towards the limit.

abbradar · 2018-05-11T11:52:32Z

Yeah, I understand - the problem is that this limit is being hit inside GCC itself where we have no control over usage of response files (which get overall size under the limit). I thought of clearing environment variables to get over this limit but that feels too hacky and may lead to subtle errors if we clear NIX_*.

…

On May 11, 2018 2:48:56 PM GMT+03:00, Sarah Brofeldt ***@***.***> wrote: @abbradar fwiw, that error message can be a little misleading, if you didn't know: Everything in the environment is counted _as well_ as the command line arguments. If the content of variables were just stuck in response files but the variables themselves never cleared, they still count towards the limit. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #40186 (comment)

-- Nikolay.

srhb · 2018-05-11T11:53:26Z

Yes, it does seem hacky. It's a fundamental builder problem if we take up this much space in the environment, though...

Ericson2314 · 2018-05-15T03:47:34Z

#40539 means one can do strictDeps = true, which is a slightly nicer solution to the too-many-deps problem.

Ericson2314 · 2018-05-15T22:33:22Z

Post-#40529 do strictDeps = true; instead of crossConfig = true;.

colonelpanic8 · 2018-05-30T03:17:25Z

@abbradar Are we still stuck?

colonelpanic8 · 2018-05-30T06:58:31Z

@Ericson2314 I'm running in to this issue with stack builds of taffybar as well (I have the nix integration enabled). How would I set strictDeps = true; for this case?

puffnfresh

I think either NixOS/cabal2nix#353 or haskell-gi/haskell-gi#172 are better places for these changes.

puffnfresh · 2018-05-31T11:06:10Z

pkgs/development/haskell-modules/configuration-nix.nix

@@ -144,6 +144,9 @@ self: super: builtins.intersectAttrs super {
  gtk = disableHardening (addPkgconfigDepend (addBuildTool super.gtk self.gtk2hs-buildtools) pkgs.gtk2) ["fortify"];
  gtksourceview2 = addPkgconfigDepend super.gtksourceview2 pkgs.gtk2;
  gtk-traymanager = addPkgconfigDepend super.gtk-traymanager pkgs.gtk3;
+  gi-gdkx11 = addPkgconfigDepend super.gi-gdkx11 pkgs.gtk3;
+  gi-dbusmenu = super.gi-dbusmenu.override { dbusmenu-glib = pkgs.libdbusmenu-glib; };
+  gi-dbusmenugtk3 = super.gi-dbusmenugtk3.override { dbusmenu-gtk3 = pkgs.libdbusmenu-gtk3; };


I think NixOS/cabal2nix#353 is also needed, unless upstream is an even better place.

colonelpanic8 · 2018-06-13T20:00:41Z

@abbradar I believe that you should be able to fully compile taffybar-2.1.2 with this pull request because I have removed all hsc files. The gcc call with too many arguments was occuring in this call, (cabal actually seems to properly handle deduplicating these flags AFAICT).

Can you verify?

glglwty · 2018-07-11T18:44:58Z

Is this still an issue? I've been using taffybar from 18.03 for a while and just discovered that taffybar works in nixos-unstable channel without this patch.

puffnfresh · 2018-07-12T00:43:46Z

Yeah, I think it's fine now @glglwty

abbradar · 2019-05-30T12:39:38Z

Let's close it; it's stale now.

abbradar added 0.kind: bug 6.topic: haskell 1.severity: mass-rebuild 2.status: work-in-progress labels May 8, 2018

abbradar requested a review from peti May 8, 2018 19:17

abbradar force-pushed the haskell-ldlib branch from 506d5c0 to cc70773 Compare May 8, 2018 19:18

abbradar added 5 commits May 8, 2018 22:19

haskellPackages.callPackage: fix search for shared libraries in GHCi

76f629e

This was already used in Stack builder.

haskellPackages.gi-gdkx11: fix build

c793219

haskellPackages.haskell-gi: enable checks back

7c698dd

Fixed in 809c70d0594bc6cc898841ec3b2684c61822c9ba

libdbusmenu: propagate gtk

c25adb7

It's needed to propagate GI dependencies.

haskellPackages.gi-dbusmenu: fix build

8ea0e0c

abbradar force-pushed the haskell-ldlib branch from cc70773 to 8ea0e0c Compare May 8, 2018 19:19

GrahamcOfBorg added 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ labels May 8, 2018

abbradar mentioned this pull request May 8, 2018

Can't be built Haskell gi-debusmenu (needed by taffybar-2.0.0) #40164

Closed

colonelpanic8 mentioned this pull request May 8, 2018

Taffybar fails to build in nixpackages taffybar/taffybar#298

Closed

puffnfresh approved these changes May 8, 2018

View reviewed changes

peti requested changes May 9, 2018

View reviewed changes

Ericson2314 mentioned this pull request May 16, 2018

taffybar-1.0.2: gcc command line is too long because of repeated flags #40013

Closed

puffnfresh suggested changes May 31, 2018

View reviewed changes

abbradar closed this May 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Fix taffybar #40186

[WIP] Fix taffybar #40186

abbradar commented May 8, 2018 •

edited

puffnfresh May 8, 2018

abbradar May 8, 2018

Ericson2314 May 8, 2018

abbradar commented May 8, 2018

Ericson2314 commented May 8, 2018

abbradar commented May 8, 2018 via email •

edited

colonelpanic8 commented May 8, 2018

abbradar commented May 8, 2018

Ericson2314 commented May 9, 2018

Ericson2314 commented May 9, 2018

peti May 9, 2018

abbradar commented May 9, 2018 via email

peti commented May 9, 2018

peti May 9, 2018

peti May 9, 2018

puffnfresh May 31, 2018

abbradar commented May 9, 2018

Ericson2314 commented May 9, 2018

abbradar commented May 10, 2018 •

edited

srhb commented May 11, 2018 •

edited

abbradar commented May 11, 2018 via email

srhb commented May 11, 2018

Ericson2314 commented May 15, 2018

Ericson2314 commented May 15, 2018

colonelpanic8 commented May 30, 2018

colonelpanic8 commented May 30, 2018

puffnfresh left a comment

puffnfresh May 31, 2018

colonelpanic8 commented Jun 13, 2018

glglwty commented Jul 11, 2018

puffnfresh commented Jul 12, 2018

abbradar commented May 30, 2019

[WIP] Fix taffybar #40186

[WIP] Fix taffybar #40186

Conversation

abbradar commented May 8, 2018 • edited

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abbradar commented May 8, 2018

Ericson2314 commented May 8, 2018

abbradar commented May 8, 2018 via email • edited

colonelpanic8 commented May 8, 2018

abbradar commented May 8, 2018

Ericson2314 commented May 9, 2018

Ericson2314 commented May 9, 2018

Choose a reason for hiding this comment

abbradar commented May 9, 2018 via email

peti commented May 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abbradar commented May 9, 2018

Ericson2314 commented May 9, 2018

abbradar commented May 10, 2018 • edited

srhb commented May 11, 2018 • edited

abbradar commented May 11, 2018 via email

srhb commented May 11, 2018

Ericson2314 commented May 15, 2018

Ericson2314 commented May 15, 2018

colonelpanic8 commented May 30, 2018

colonelpanic8 commented May 30, 2018

puffnfresh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

colonelpanic8 commented Jun 13, 2018

glglwty commented Jul 11, 2018

puffnfresh commented Jul 12, 2018

abbradar commented May 30, 2019

abbradar commented May 8, 2018 •

edited

abbradar commented May 8, 2018 via email •

edited

abbradar commented May 10, 2018 •

edited

srhb commented May 11, 2018 •

edited