Update Gambit and Gerbil #44847

fare · 2018-08-09T22:14:38Z

Motivation for this change

Upstream update + make Gambit stable buildable with recent Nix.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

xeji · 2018-08-09T22:31:42Z

pkgs/development/compilers/gambit/default.nix

@@ -2,6 +2,7 @@

 callPackage ./build.nix {
  version = "4.8.9";
+  git-version = "v${version}";


Eval error because this isn't a recursive set. Try using callPackage ./build.nix rec { to make it recursive.

Oops. Thanks a lot! Fixed.

xeji · 2018-08-10T07:30:20Z

pkgs/development/compilers/gambit/build.nix

@@ -1,4 +1,4 @@
-{ stdenv, git, openssl, autoconf, pkgs, makeStaticLibraries, version, SRC }:
+{ stdenv, git, openssl, autoconf, pkgs, makeStaticLibraries, version, SRC, git-version }:


The git-version argument isn't used anywhere. Why did you add it?

xeji · 2018-08-10T07:31:14Z

pkgs/development/compilers/gambit/default.nix

  version = "4.8.9";
+  git-version = "v${version}";


Defining git-version only makes sense if it is actually used by build.nix.

Well,

1- I include it by symmetry with gerbil, that actually uses it. I actually intended to use it in gambit in the very same way, but that requires changes to the gambit build system similar to those done in Gerbil, that haven't made it on top of my todo list yet.

2- It's somewhat useful for me to track the progress of gambit unstable, and needed by symmetry in the stable build, too.

3- If you object to it, I can make it a comment instead, until I get to modifying the gambit build system (and get the changes upstreamed).

Thank you for explaining. While I understand your intentions, unused code tends to confuse people and make packages harder to maintain - you might not be the only one modifying these in the future. So please remove the unused code but feel free to add your comments.

No problem. I understand your concerns. Here is an updated version where only the unstable build has the git-version, and only in a comment.

git-version will be commented out for gambit, until actually used. NixOS/nixpkgs#44847

xeji · 2018-08-10T17:35:32Z

@GrahamcOfBorg build gambit-unstable gerbil-unstable

GrahamcOfBorg · 2018-08-10T18:06:19Z

Timed out, unknown build status on x86_64-darwin (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
clang -O2 -Wno-deprecated-declarations -I"/usr/local/opt/openssl/include"   -Wno-unused -Wno-write-strings -Wdisabled-optimization -fwrapv -fno-strict-aliasing -fno-math-errno -fmodulo-sched -fomit-frame-pointer -fPIC -fno-common   -I"../include" -c -o "_t-univ-2.o" -I. -DHAVE_CONFIG_H _t-univ-2.c -D___LIBRARY
clang-5.0: warning: optimization flag '-fmodulo-sched' is not supported [-Wignored-optimization-argument]
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
clang -O2 -Wno-deprecated-declarations -I"/usr/local/opt/openssl/include"   -Wno-unused -Wno-write-strings -Wdisabled-optimization -fwrapv -fno-strict-aliasing -fno-math-errno -fmodulo-sched -fomit-frame-pointer -fPIC -fno-common   -I"../include" -c -o "_t-c-2.o" -I. -DHAVE_CONFIG_H _t-c-2.c -D___LIBRARY
clang-5.0: warning: optimization flag '-fmodulo-sched' is not supported [-Wignored-optimization-argument]
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
building of '/nix/store/1igkjj9g5l4xxwrivhqf51prs0bdkag3-gambit-unstable-2018-08-06.drv' timed out after 1800 seconds
cannot build derivation '/nix/store/79xvdl57h4dfgr6xag2ysf6r6v9r83as-gerbil-unstable-2018-08-09.drv': 1 dependencies couldn't be built
error: build of '/nix/store/1igkjj9g5l4xxwrivhqf51prs0bdkag3-gambit-unstable-2018-08-06.drv', '/nix/store/79xvdl57h4dfgr6xag2ysf6r6v9r83as-gerbil-unstable-2018-08-09.drv' failed

fare · 2018-08-10T18:11:40Z

It does take a lot of time to build it — tens of minutes on my machine from last year.

Also, I see you use clang. I believe gambit currently works better with GCC. How do I force the use of GCC for Gambit and Gerbil?

fare · 2018-08-10T18:13:58Z

Excerpt from an email last May by Marc Feeley, author of Gambit, on the gambit-list:

the build time of a “make -j8” on my 4 core laptop is over 10x slower when using CLANG. As for speed of the generated code, CLANG produces code that is about 3x slower than GCC.

GrahamcOfBorg · 2018-08-10T18:16:08Z

Success on x86_64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/lazy__1.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/sort__0.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/coroutine__0.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/pregexp__1.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/event__rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/generic__rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/std/test__1.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gx-gambc.o1
patching script interpreter paths in /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09
checking for references to /tmp/nix-build-gerbil-unstable-2018-08-09.drv-0 in /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09...

xeji · 2018-08-10T18:18:24Z

clang is the default compiler on darwin (MacOS), while x86_64-linux and aar64_linux use gcc7.
Not sure we support gcc on darwin.

fare · 2018-08-10T18:34:51Z

It makes total sense as a default on Darwin. But can I somehow override the default?

GrahamcOfBorg · 2018-08-10T19:30:23Z

Timed out, unknown build status on aarch64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

[*] Cleaning up bootstrap
[*] Building gerbil stdlib
... generate stdlib depgraph
... compile build-config
... compile format
... compile pregexp
... compile sort
... compile sugar
building of '/nix/store/8rfl3x1qxiv4vjz74rpkdnml76iq0hgb-gerbil-unstable-2018-08-09.drv' timed out after 3600 seconds
error: build of '/nix/store/8rfl3x1qxiv4vjz74rpkdnml76iq0hgb-gerbil-unstable-2018-08-09.drv' failed

xeji · 2018-08-10T19:31:11Z

Timeouts on ofborg are not uncommon since ofborg uses community-provided build machines that don't guarantee good performance. These packages usually build on hydra (higher time limit and better performance), so it should be fine without any changes.

To force gcc on darwin, you can add stdenv = gccStdenv; for the package in all-packages.nix, see other examples there. No guarantees, it may time out on ofborg as well. But feel free to give it a try.

fare · 2018-08-10T19:43:02Z

Thanks, xeji. I added a commit that forces Gambit and Gerbil to use GCC rather than Clang.

fare · 2018-08-10T19:44:20Z

NB: I checked that these changes are NOPs on my Linux amd64 NixOS box.

xeji · 2018-08-10T20:03:22Z

@GrahamcOfBorg build gambit-unstable gerbil-unstable

GrahamcOfBorg · 2018-08-10T20:33:10Z

Failure on x86_64-darwin (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

 ^~~~~~~~~
kqueue__0.c:8750:12: error: 'NOTE_MACHTIME' undeclared (first use in this function); did you mean 'NOTE_CHILD'?
 ___return (NOTE_MACHTIME);
            ^
kqueue__0.c:8750:1: note: in expansion of macro '___return'
 ___return (NOTE_MACHTIME);
 ^~~~~~~~~
*** ERROR; build failed
builder for '/nix/store/4vmycbc1m85g9y64pi9m7y7n3lwkqwa0-gerbil-unstable-2018-08-09.drv' failed with exit code 1
error: build of '/nix/store/4vmycbc1m85g9y64pi9m7y7n3lwkqwa0-gerbil-unstable-2018-08-09.drv' failed

GrahamcOfBorg · 2018-08-10T20:38:32Z

Success on x86_64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/compiler/optimize-top__rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/compiler/optimize-ann__0.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/compiler/optimize-ann__rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/compiler/driver__rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/core$_expander-runtime___rt.o1
shrinking /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09/lib/gerbil/core$_sugar_$_sugar_1___rt.o1
patching script interpreter paths in /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09
checking for references to /build in /nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09...
/nix/store/hl80a5v4s4krkcrc8zsjl4yi0hf4404p-gambit-unstable-2018-08-06
/nix/store/w6nrvxn2dmf4pgrqgcr11nph2jfri0a9-gerbil-unstable-2018-08-09

fare · 2018-08-10T20:50:28Z

@vyzo can you give a look at the darwin failure? If not, who is building Gerbil on Darwin?

fare · 2018-08-10T20:57:24Z

This looks like a recent upstream bug in the new kqueue support.

@grahamc What version of macOS is the Nix builder running? How do I tell?

GrahamcOfBorg · 2018-08-10T21:03:41Z

Timed out, unknown build status on aarch64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

[*] Cleaning up bootstrap
[*] Building gerbil stdlib
... generate stdlib depgraph
... compile build-config
... compile format
... compile pregexp
... compile sort
... compile sugar
building of '/nix/store/8rfl3x1qxiv4vjz74rpkdnml76iq0hgb-gerbil-unstable-2018-08-09.drv' timed out after 3600 seconds
error: build of '/nix/store/8rfl3x1qxiv4vjz74rpkdnml76iq0hgb-gerbil-unstable-2018-08-09.drv' failed

vyzo · 2018-08-11T07:15:54Z

The darwin failure should be fixed by mighty-gerbils/gerbil#118

clang builds gambit 10x slower to produce code that is 3x slower than when using GCC. So use GCC to build Gambit and Gerbil.

fare · 2018-08-11T18:43:54Z

@GrahamcOfBorg build gambit-unstable gerbil-unstable

I updated this PR to include @vyzo's latest upstream fixes.

7c6f434c · 2018-08-11T18:53:45Z

@GrahamcOfBorg build gambit-unstable gerbil-unstable

GrahamcOfBorg · 2018-08-11T19:14:59Z

Success on x86_64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/optimize-top__1.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/ssxi__1.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/optimize-top__rt.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/optimize-ann__0.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/optimize-ann__rt.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/compiler/driver__rt.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/core$_expander-runtime___rt.o1
shrinking /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11/lib/gerbil/core$_sugar_$_sugar_1___rt.o1
patching script interpreter paths in /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11
checking for references to /build in /nix/store/jqsaq6k804c2xfq0b2kjgn7dry0yr9nr-gerbil-unstable-2018-08-11...

GrahamcOfBorg · 2018-08-11T19:15:00Z

Success on x86_64-darwin (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

TAG lang/scheme/time.ss
TAG lang/scheme/write-impl.ss
TAG lang/scheme/write.ss
[*] Done
installing
post-installation fixup
moving /nix/store/lf2rkp8d7vlqkrh2amrc479iraki6lrf-gerbil-unstable-2018-08-11/doc to /nix/store/lf2rkp8d7vlqkrh2amrc479iraki6lrf-gerbil-unstable-2018-08-11/share/doc
patching script interpreter paths in /nix/store/lf2rkp8d7vlqkrh2amrc479iraki6lrf-gerbil-unstable-2018-08-11
/nix/store/1v612hm9ihd7iid12vpy6ivr58016yg2-gambit-unstable-2018-08-06
/nix/store/lf2rkp8d7vlqkrh2amrc479iraki6lrf-gerbil-unstable-2018-08-11

GrahamcOfBorg · 2018-08-11T20:48:39Z

Timed out, unknown build status on aarch64-linux (full log)

Attempted: gambit-unstable, gerbil-unstable

Partial log (click to expand)

[*] Cleaning up bootstrap
[*] Building gerbil stdlib
... generate stdlib depgraph
... compile build-config
... compile format
... compile pregexp
... compile sort
... compile sugar
building of '/nix/store/zh9klqqlxd76b9zsaaw3g3rknmhqi40s-gerbil-unstable-2018-08-11.drv' timed out after 3600 seconds
error: build of '/nix/store/zh9klqqlxd76b9zsaaw3g3rknmhqi40s-gerbil-unstable-2018-08-11.drv' failed

fare · 2018-08-11T21:24:31Z

Can we either add a larger timeout for AA64, or disable it wholly? As such, it's only a waste of CPU. Can this be overridden on a per-package basis?

7c6f434c · 2018-08-12T08:35:23Z

Let me just say we don't block Chromium build attempts on ofBorg (obviously, they time out always and everywhere). No tooling is perfect

xeji reviewed Aug 9, 2018

View reviewed changes

fare force-pushed the master branch from 5b98a42 to 93eb95f Compare August 9, 2018 22:50

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Aug 9, 2018

xeji reviewed Aug 10, 2018

View reviewed changes

gambit-unstable: 2018-05-30 -> 2018-08-06

0dd9403

fare force-pushed the master branch from 93eb95f to bb09835 Compare August 10, 2018 17:12

fare added a commit to mighty-gerbils/gerbil-utils that referenced this pull request Aug 10, 2018

Update nix update script

be21de1

git-version will be commented out for gambit, until actually used. NixOS/nixpkgs#44847

fare added 2 commits August 11, 2018 14:07

gerbil-unstable: 2018-05-12 -> 2018-08-11

c75576f

gambit, gerbil: use stdenv = gccStdenv

d9f419b

clang builds gambit 10x slower to produce code that is 3x slower than when using GCC. So use GCC to build Gambit and Gerbil.

fare force-pushed the master branch from 0d91f42 to d9f419b Compare August 11, 2018 18:42

7c6f434c merged commit f4f93d6 into NixOS:master Aug 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Gambit and Gerbil #44847

Update Gambit and Gerbil #44847

fare commented Aug 9, 2018

xeji Aug 9, 2018

fare Aug 9, 2018

xeji Aug 10, 2018

xeji Aug 10, 2018

fare Aug 10, 2018

xeji Aug 10, 2018

fare Aug 10, 2018

xeji commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018 •

edited

GrahamcOfBorg commented Aug 10, 2018

xeji commented Aug 10, 2018

fare commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

xeji commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018

xeji commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

vyzo commented Aug 11, 2018

fare commented Aug 11, 2018

7c6f434c commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

fare commented Aug 11, 2018

7c6f434c commented Aug 12, 2018

		@@ -1,4 +1,4 @@
		{ stdenv, git, openssl, autoconf, pkgs, makeStaticLibraries, version, SRC }:
		{ stdenv, git, openssl, autoconf, pkgs, makeStaticLibraries, version, SRC, git-version }:

Update Gambit and Gerbil #44847

Update Gambit and Gerbil #44847

Conversation

fare commented Aug 9, 2018

Motivation for this change

Things done

xeji Aug 9, 2018

Choose a reason for hiding this comment

fare Aug 9, 2018

Choose a reason for hiding this comment

xeji Aug 10, 2018

Choose a reason for hiding this comment

xeji Aug 10, 2018

Choose a reason for hiding this comment

fare Aug 10, 2018

Choose a reason for hiding this comment

xeji Aug 10, 2018

Choose a reason for hiding this comment

fare Aug 10, 2018

Choose a reason for hiding this comment

xeji commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018 • edited

GrahamcOfBorg commented Aug 10, 2018

xeji commented Aug 10, 2018

fare commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

xeji commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018

xeji commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

fare commented Aug 10, 2018

fare commented Aug 10, 2018

GrahamcOfBorg commented Aug 10, 2018

vyzo commented Aug 11, 2018

fare commented Aug 11, 2018

7c6f434c commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

GrahamcOfBorg commented Aug 11, 2018

fare commented Aug 11, 2018

7c6f434c commented Aug 12, 2018

fare commented Aug 10, 2018 •

edited