emscripten: 1.37.16 -> 1.37.34 #35636

ryantm · 2018-02-26T05:00:24Z

Semi-automatic update. These checks were performed:

built on NixOS
found 1.37.34 with grep in /nix/store/mdr47v3wdjmzic5c2nvdx5krdwl6bcxf-emscripten-1.37.34

Semi-automatic update. These checks were performed: - built on NixOS - found 1.37.34 with grep in /nix/store/mdr47v3wdjmzic5c2nvdx5krdwl6bcxf-emscripten-1.37.34

ryantm · 2018-02-26T05:01:04Z

@GrahamcOfBorg build emscripten

GrahamcOfBorg · 2018-02-26T05:02:54Z

Success on x86_64-darwin (full log)

Partial log (click to expand)

trying https://github.com/kripken/emscripten/archive/1.37.34.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   127    0   127    0     0    211      0 --:--:-- --:--:-- --:--:--   212
100 32.1M    0 32.1M    0     0  1776k      0 --:--:--  0:00:18 --:--:-- 2095k
unpacking source archive /private/tmp/nix-build-source.drv-0/1.37.34.tar.gz
copying path '/nix/store/fvw6b4kzw10hyfj4njwfxpxbldln171p-emscripten-fastcomp-wrapper-1.37.16' from 'https://cache.nixos.org'...
copying path '/nix/store/qjx6hyzg2fmn2psnjnpjxmw4zyy71izz-emscriptenfastcomp' from 'https://cache.nixos.org'...
building '/nix/store/29g20qrk06wr2zxwycv6z66lc8k3x8i2-emscripten-1.37.34.drv'...
/nix/store/1030pf5aly11mvq117jicnvxajgzi550-emscripten-1.37.34

GrahamcOfBorg · 2018-02-26T06:47:37Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

Package ‘oraclejre-8u161’ in /var/lib/gc-of-borg/nix-test-rs-2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-2/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix:71 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-26T09:32:18Z

Success on x86_64-linux (full log)

Partial log (click to expand)


*** Downloading ‘https://cache.nixos.org/nar/0qgd98xmqf322i945vzxf3xg8086x6451hf4vmc5ri8mrnlrs6s7.nar.xz’ (signed by ‘cache.nixos.org-1’) to ‘/nix/store/fky9air5q4ji9whdyribbpwj6drgiz4q-emscriptenfastcomp’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 27996  100 27996    0     0  84325      0 --:--:-- --:--:-- --:--:-- 84325


100 6535k  100 6535k    0     0  7005k      0 --:--:-- --:--:-- --:--:-- 6997k

building path(s) ‘/nix/store/mdr47v3wdjmzic5c2nvdx5krdwl6bcxf-emscripten-1.37.34’
/nix/store/mdr47v3wdjmzic5c2nvdx5krdwl6bcxf-emscripten-1.37.34

dtzWill · 2018-02-27T02:21:03Z

Is the compiler tested as part of building itself, or should that be done as well? Never poked at the emscripten bits...

joachifm · 2018-03-14T19:26:45Z

cc @qknight @matthiasbeyer

matthiasbeyer · 2018-03-14T20:05:00Z

You meant @matthewbauer

joachifm · 2018-03-14T20:14:14Z

@matthiasbeyer it worked out in the end :)

dtzWill · 2018-03-14T20:55:54Z

While debugging my follow-up PR, I found that this breaks emscriptenPackages?
Not familiar enough to say why, but it worked previously:

@GrahamcOfBorg build emscriptenPackages.zlib

Like I said before, does the resulting compiler still work? Apparently not... (at least not the way we used it) :(.

GrahamcOfBorg · 2018-03-14T20:56:19Z

No attempt on aarch64-linux

The following builds were skipped because they don't evaluate on aarch64-linux: emscriptenPackages.zlib

No log is available.

GrahamcOfBorg · 2018-03-14T20:56:45Z

Failure on x86_64-darwin (full log)

Attempted: emscriptenPackages.zlib

Partial log (click to expand)

setting SOURCE_DATE_EPOCH to timestamp 1484501380 of file zlib-1.2.11/zutil.c
patching sources
configuring
INFO:root:generating system asset: is_vanilla.txt... (this will be cached in "/private/tmp/nix-build-emscripten-zlib-1.2.11.drv-0/.emscripten_cache/is_vanilla.txt" for subsequent builds)
INFO:root: - ok
Compiler error reporting is too harsh for ./configure (perhaps remove -Werror).
** ./configure aborting.
ERROR:root:Configure step failed with non-zero return code 1! Command line: ['./configure', '--prefix=/nix/store/xvv659ix482wjz6jm4ff91dg2mm76lmj-emscripten-zlib-1.2.11'] at /private/tmp/nix-build-emscripten-zlib-1.2.11.drv-0/zlib-1.2.11
builder for '/nix/store/g7dmgqidgs837nz65d0yb76p0an71f0m-emscripten-zlib-1.2.11.drv' failed with exit code 1
error: build of '/nix/store/g7dmgqidgs837nz65d0yb76p0an71f0m-emscripten-zlib-1.2.11.drv' failed

ryantm · 2018-03-14T21:06:08Z

@joachifm based on @dtzWill's comment seems like we should revert.

…1.37.16-to-1.37.34" This reverts commit 5c02b9a, reversing changes made to 2f2ad74. #35636 (comment)

GrahamcOfBorg · 2018-03-14T21:27:09Z

Success on x86_64-linux (full log)

Attempted: emscriptenPackages.zlib

Partial log (click to expand)

strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/9rlgsjbf2hp35f29wi2z17vvz12qmmfk-emscripten-zlib-1.2.11-dev/lib 
patching script interpreter paths in /nix/store/9rlgsjbf2hp35f29wi2z17vvz12qmmfk-emscripten-zlib-1.2.11-dev
checking for references to /tmp/nix-build-emscripten-zlib-1.2.11.drv-0 in /nix/store/9rlgsjbf2hp35f29wi2z17vvz12qmmfk-emscripten-zlib-1.2.11-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/n7skclzybwm6c35yqb9b4h4myj06h7vx-emscripten-zlib-1.2.11-static
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/n7skclzybwm6c35yqb9b4h4myj06h7vx-emscripten-zlib-1.2.11-static/lib 
patching script interpreter paths in /nix/store/n7skclzybwm6c35yqb9b4h4myj06h7vx-emscripten-zlib-1.2.11-static
checking for references to /tmp/nix-build-emscripten-zlib-1.2.11.drv-0 in /nix/store/n7skclzybwm6c35yqb9b4h4myj06h7vx-emscripten-zlib-1.2.11-static...
/nix/store/4vjwq8rnqxz0dx34j8rcwcg3zdnxj4by-emscripten-zlib-1.2.11

qknight · 2018-03-14T21:46:00Z

@ryantm @dtzWill i've been playing with this for a while now but it seems that those packages didn't build before either. no clue why yet.

i'll look into this. don't revert for the moment.

ryantm · 2018-03-14T22:07:46Z

@qknight It was already reverted.

qknight · 2018-03-16T02:48:15Z

@ryantm thanks for your effort!

see this: emscripten-core/emscripten-fastcomp#224

... you should use the same emscripten version in all 3 repos (emcc should warn if you do not).

well, it didn't warn IIRC

You must also use the same version of LLVM and clang (unrelated to emscripten - if those repos don't match, LLVM won't be able to use clang properly). The same emscripten version in those repos will take care of that for you. Likewise the same emscripten version will ensure you are on the right LLVM version.

i found that out the hard way.

in general we should change nixpkgs expressions so that one defines the version number as 1.37.36 and all the 3 expressions download the same version.

our roadmap should be:

get emscriptenPackages to comiple with 1.37.36
- emcc main.c -> produces something working
- emconfigure seems to fail
get a meaningful set of tests in the emscripten package working
refactor the 3 expressions to use the same target version (only set once)

@everyone

i've been playing with 1.37.36 and implemented the unit testing:

altough there are lots of errors, this is not looking so bad. i'd love to select some meaningful tests and get these into nixpkgs. we can't do all, this requires at least 4220.752s to complete, which is too long!

building emscriptenPackage.zlib

still this won't compile with the new toolchain. no clue why, anyone?

joachim@lenovo-t530 ~/D/p/n/nixpkgs> nix-env -f default.nix -I nixpkgs=. -iA emscriptenPackages.zlib
installing ‘emscripten-zlib-1.2.11’
these derivations will be built:
  /nix/store/8xlwzkd9lsziq188fyznsdmyqgmg7w5f-emscripten-zlib-1.2.11.drv
building path(s) ‘/nix/store/1az5xq03k6qjms3mdvaiix25fk51whv4-emscripten-zlib-1.2.11-static’, ‘/nix/store/ak7rqj9bdhdf0ll4idr6sb06jzrcb6s3-emscripten-zlib-1.2.11-dev’, ‘/nix/store/qi1k9vbp9jzavx44n96zv2ihz96c16af-emscripten-zlib-1.2.11’
unpacking sources
unpacking source archive /nix/store/rrzja19jbqxbfryp2zchb2ma5h041rxh-zlib-1.2.11.tar.gz
source root is zlib-1.2.11
setting SOURCE_DATE_EPOCH to timestamp 1484501380 of file zlib-1.2.11/zutil.c
patching sources
configuring
INFO:root:generating system asset: is_vanilla.txt... (this will be cached in "/tmp/nix-build-emscripten-zlib-1.2.11.drv-0/.emscripten_cache/is_vanilla.txt" for subsequent builds)
INFO:root: - ok
Compiler error reporting is too harsh for ./configure (perhaps remove -Werror).
** ./configure aborting.
ERROR:root:Configure step failed with non-zero return code 1! Command line: ['./configure', '--prefix=/nix/store/qi1k9vbp9jzavx44n96zv2ihz96c16af-emscripten-zlib-1.2.11'] at /tmp/nix-build-emscripten-zlib-1.2.11.drv-0/zlib-1.2.11
builder for ‘/nix/store/8xlwzkd9lsziq188fyznsdmyqgmg7w5f-emscripten-zlib-1.2.11.drv’ failed with exit code 1
error: build of ‘/nix/store/8xlwzkd9lsziq188fyznsdmyqgmg7w5f-emscripten-zlib-1.2.11.drv’ failed

matthewbauer · 2018-03-16T02:50:02Z

Just a guess but maybe try setting 'hardeningDisable = [ "format" ]'

qknight · 2018-03-16T12:03:51Z

@matthewbauer where should i try hardeningDisable?

seems the python executable was missing and therefore emconfigure's internals fail. i might have json_c and zlib working.

i'm no facing:

make[3]: Entering directory '/tmp/nix-build-emscripten-libxml2-2.9.7.drv-0/libxml2-2.9.7/python'
Making all in .
make[4]: Entering directory '/tmp/nix-build-emscripten-libxml2-2.9.7.drv-0/libxml2-2.9.7/python'
  CC       libxml.lo
In file included from libxml.c:14:
In file included from /nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/include/python2.7/Python.h:58:
/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/include/python2.7/pyport.h:895:2: error: "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
#error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
 ^
1 error generated.
ERROR:root:compiler frontend failed to generate LLVM bitcode, halting
make[4]: *** [Makefile:623: libxml.lo] Error 1

qknight · 2018-03-16T19:08:28Z

LONG_BIT is no longer a problem, i've added this:

emconfigure ./configure --prefix=$out --without-python

to libxml2

qknight · 2018-03-16T19:09:38Z

@matthewbauer maybe you could check emscripten-core/emscripten#6341 for an obvious error i've made.

matthewbauer · 2018-03-16T19:09:45Z

@qknight "hardeningDisable" was just a guess based on the "Compiler error reporting is too harsh" line. But it sounds like python was the real culprit.

matthewbauer · 2018-03-16T19:51:04Z

@qknight Looks like this might be the culprit:

nixpkgs/pkgs/development/libraries/zlib/default.nix

Line 53 in 5f020d5

    
           NIX_CFLAGS_COMPILE = stdenv.lib.optionalString (!hostPlatform.isDarwin) "-static-libgcc";

This was introduced in @Ericson2314's cross compiling patches.

PR #26007 used these to avoid causing a mass rebuild. Now that we know things work, we do that to clean up.

qknight · 2018-03-16T21:35:05Z

@matthewbauer you are amazing!

setting: NIX_CFLAGS_COMPILE=""; leads to being able to compile a libz shared library!

qknight · 2018-03-17T01:32:22Z

qknight · 2018-03-17T01:47:26Z

@ryantm @matthewbauer

if you want to test my changes:

source: https://github.com/qknight/nixpkgs/tree/emscripten-1.37.16-to-1.37.36
command: nix-env -f default.nix -I nixpkgs=. -iA emscriptenPackages

qknight · 2018-03-17T13:04:28Z

so everything from my perspective is now working. someone interested in helping me to write the documentation?

qknight · 2018-03-18T01:19:42Z

\o/ done!

qknight · 2018-03-18T01:31:43Z

@ryantm @matthewbauer please have a look at #37291

emscripten: 1.37.16 -> 1.37.34

71286aa

Semi-automatic update. These checks were performed: - built on NixOS - found 1.37.34 with grep in /nix/store/mdr47v3wdjmzic5c2nvdx5krdwl6bcxf-emscripten-1.37.34

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100 labels Feb 26, 2018

matthewbauer approved these changes Mar 14, 2018

View reviewed changes

joachifm merged commit 5c02b9a into NixOS:master Mar 14, 2018

ryantm deleted the auto-update/emscripten-1.37.16-to-1.37.34 branch March 14, 2018 20:14

dtzWill mentioned this pull request Mar 14, 2018

emscripten{,fastcomp}: -> 1.37.36 #37022

Closed

8 tasks

joachifm added a commit that referenced this pull request Mar 14, 2018

Revert "Merge pull request #35636 from ryantm/auto-update/emscripten-…

4e10832

…1.37.16-to-1.37.34" This reverts commit 5c02b9a, reversing changes made to 2f2ad74. #35636 (comment)

ryantm restored the auto-update/emscripten-1.37.16-to-1.37.34 branch March 14, 2018 22:08

qknight mentioned this pull request Mar 15, 2018

documentation emscripten-core/emscripten-fastcomp#224

Closed

ryantm deleted the auto-update/emscripten-1.37.16-to-1.37.34 branch March 16, 2018 01:21

qknight mentioned this pull request Mar 16, 2018

zlib only builds static library emscripten-core/emscripten#6341

Closed

matthewbauer referenced this pull request Mar 16, 2018

misc pkgs: Remove unneeded *Platform == *Platform comparisons

95c8277

PR #26007 used these to avoid causing a mass rebuild. Now that we know things work, we do that to clean up.

qknight mentioned this pull request Mar 17, 2018

xmlmirror build issues emscripten-core/emscripten#6344

Closed

qknight mentioned this pull request Mar 18, 2018

Emscripten 1.37.16 to 1.37.36 #37291

Merged

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

emscripten: 1.37.16 -> 1.37.34 #35636

emscripten: 1.37.16 -> 1.37.34 #35636

ryantm commented Feb 26, 2018

ryantm commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

dtzWill commented Feb 27, 2018

joachifm commented Mar 14, 2018

matthiasbeyer commented Mar 14, 2018

joachifm commented Mar 14, 2018

dtzWill commented Mar 14, 2018

GrahamcOfBorg commented Mar 14, 2018

GrahamcOfBorg commented Mar 14, 2018

ryantm commented Mar 14, 2018 •

edited

GrahamcOfBorg commented Mar 14, 2018

qknight commented Mar 14, 2018

ryantm commented Mar 14, 2018

qknight commented Mar 16, 2018

matthewbauer commented Mar 16, 2018

qknight commented Mar 16, 2018 •

edited

qknight commented Mar 16, 2018

qknight commented Mar 16, 2018

matthewbauer commented Mar 16, 2018 •

edited

matthewbauer commented Mar 16, 2018

qknight commented Mar 16, 2018 •

edited

qknight commented Mar 17, 2018 •

edited

qknight commented Mar 17, 2018

qknight commented Mar 17, 2018

qknight commented Mar 18, 2018

qknight commented Mar 18, 2018 •

edited

emscripten: 1.37.16 -> 1.37.34 #35636

emscripten: 1.37.16 -> 1.37.34 #35636

Conversation

ryantm commented Feb 26, 2018

ryantm commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

GrahamcOfBorg commented Feb 26, 2018

dtzWill commented Feb 27, 2018

joachifm commented Mar 14, 2018

matthiasbeyer commented Mar 14, 2018

joachifm commented Mar 14, 2018

dtzWill commented Mar 14, 2018

GrahamcOfBorg commented Mar 14, 2018

GrahamcOfBorg commented Mar 14, 2018

ryantm commented Mar 14, 2018 • edited

GrahamcOfBorg commented Mar 14, 2018

qknight commented Mar 14, 2018

ryantm commented Mar 14, 2018

qknight commented Mar 16, 2018

building emscriptenPackage.zlib

matthewbauer commented Mar 16, 2018

qknight commented Mar 16, 2018 • edited

qknight commented Mar 16, 2018

qknight commented Mar 16, 2018

matthewbauer commented Mar 16, 2018 • edited

matthewbauer commented Mar 16, 2018

qknight commented Mar 16, 2018 • edited

qknight commented Mar 17, 2018 • edited

qknight commented Mar 17, 2018

qknight commented Mar 17, 2018

qknight commented Mar 18, 2018

qknight commented Mar 18, 2018 • edited

ryantm commented Mar 14, 2018 •

edited

qknight commented Mar 16, 2018 •

edited

matthewbauer commented Mar 16, 2018 •

edited

qknight commented Mar 16, 2018 •

edited

qknight commented Mar 17, 2018 •

edited

qknight commented Mar 18, 2018 •

edited