Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

erlang: disable parallel building #51644

Merged
merged 1 commit into from Dec 8, 2018
Merged

erlang: disable parallel building #51644

merged 1 commit into from Dec 8, 2018

Conversation

ivan
Copy link
Member

@ivan ivan commented Dec 7, 2018 •
edited

Motivation for this change

All of the Erlang runtimes fail to build on my Hydra with parallel building enabled (implemented in #45484). The same problem was observed on an official Hydra machine (and again), but ignored as a temporary issue.

This PR disables parallel building to avoid this build failure.

<3> ERLC   ../ebin/asn1ct_gen_ber_bin_v2.beam
<3> ERLC   ../ebin/asn1ct_rtt.beam
<3> ERLC   ../ebin/asn1ct_constructed_ber_bin_v2.beam
<3> GEN    asn1ct_eval_ext.erl
<3>{"init terminating in do_boot",{undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{file,"prepare_templates.erl"},{line,58}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
<3>init terminating in do_boot ()
<3>
<3>Crash dump is being written to: erl_crash.dump...done
<3>make[2]: *** [Makefile:139: asn1ct_eval_ext.erl] Error 1
<3>make[2]: Leaving directory '/build/source/lib/asn1/src'
<3>make[1]: *** [/build/source/make/otp_subdir.mk:29: opt] Error 2
<3>make[1]: Leaving directory '/build/source/lib'
<3>make: *** [Makefile:550: secondary_bootstrap_build] Error 2
<3>builder for '/nix/store/5565imavv7imy514fycp2r1qi61qx2m6-erlang-18.3.4.8.drv' failed with exit code 2

<3> ERLC   ../ebin/asn1ct_gen_ber_bin_v2.beam
<3> ERLC   ../ebin/asn1ct_rtt.beam
<3> ERLC   ../ebin/asn1ct_constructed_ber_bin_v2.beam
<3> GEN    asn1ct_eval_ext.erl
<3>{"init terminating in do_boot",{undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{file,"prepare_templates.erl"},{line,58}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
<3>init terminating in do_boot ()
<3>
<3>Crash dump is being written to: erl_crash.dump...done
<3>make[2]: *** [Makefile:139: asn1ct_eval_ext.erl] Error 1
<3>make[2]: Leaving directory '/build/source/lib/asn1/src'
<3>make[1]: *** [/build/source/make/otp_subdir.mk:29: opt] Error 2
<3>make[1]: Leaving directory '/build/source/lib'
<3>make: *** [Makefile:550: secondary_bootstrap_build] Error 2
<3>builder for '/nix/store/sd9kp905syyfqb7gczrp3qpzxvhvl1pj-erlang-19.3.6.11.drv' failed with exit code 2

<3> ERLC   ../ebin/asn1ct_gen_ber_bin_v2.beam
<3> ERLC   ../ebin/asn1ct_rtt.beam
<3> ERLC   ../ebin/asn1ct_constructed_ber_bin_v2.beam
<3> GEN    asn1ct_eval_ext.erl
<3>{"init terminating in do_boot",{undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{file,"prepare_templates.erl"},{line,58}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
<3>init terminating in do_boot ({undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{_},{_}]},{init,start_em,1,[]},{init,do_boot,3,[]}]})
<3>
<3>Crash dump is being written to: erl_crash.dump...done
<3>make[2]: *** [Makefile:138: asn1ct_eval_ext.erl] Error 1
<3>make[2]: Leaving directory '/build/source/lib/asn1/src'
<3>make[1]: *** [/build/source/make/otp_subdir.mk:29: opt] Error 2
<3>make[1]: Leaving directory '/build/source/lib'
<3>make: *** [Makefile:557: secondary_bootstrap_build] Error 2
<3>builder for '/nix/store/2dq6bp93q788bnz4gbbhhiyzrpgh38nz-erlang-20.3.8.9.drv' failed with exit code 2

<3> ERLC   ../ebin/asn1ct_gen_ber_bin_v2.beam
<3> ERLC   ../ebin/asn1ct_rtt.beam
<3> ERLC   ../ebin/asn1ct_constructed_ber_bin_v2.beam
<3> GEN    asn1ct_eval_ext.erl
<3>{"init terminating in do_boot",{undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{file,"prepare_templates.erl"},{line,58}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
<3>init terminating in do_boot ({undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{_},{_}]},{init,start_em,1,[]},{init,do_boot,3,[]}]})
<3>
<3>Crash dump is being written to: erl_crash.dump...done
<3>make[2]: *** [Makefile:138: asn1ct_eval_ext.erl] Error 1
<3>make[2]: Leaving directory '/build/source/lib/asn1/src'
<3>make[1]: *** [/build/source/make/otp_subdir.mk:29: opt] Error 2
<3>make[1]: Leaving directory '/build/source/lib'
<3>make: *** [Makefile:575: secondary_bootstrap_build] Error 2
<3>builder for '/nix/store/f9h1xf6k0sjri4h0wkgx28b5z1xwxqkw-erlang-21.1.3.drv' failed with exit code 2

I don't know why these builds succeed elsewhere, but they fail reliably on my 8 core/16 thread Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz with a NixOS (4.19.7 kernel, btrfs) in VMware on a Windows 10 1803 host.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

hedning
hedning approved these changes Dec 7, 2018
View reviewed changes
Copy link
Contributor

@hedning hedning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we hit this close to a 100% on aarch_64 which are mostly build with at least -j8, but much less on x86_64, which is mostly built with -j4, but it does happen: here's a x86_64 failure from staging-next with -j8. That suggests we could limit jobs to 4, but I can't find any simple way to do that. So I'd say it's best to just disable.

@ivan
erlang: disable parallel building
82e5590 
This fixes build errors like

 GEN    asn1ct_eval_ext.erl
{"init terminating in do_boot",{undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{file,"prepare_templates.erl"},{line,58}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
init terminating in do_boot ({undef,[{asn1ct_func,start_link,[],[]},{prepare_templates,gen_asn1ct_eval,1,[{_},{_}]},{init,start_em,1,[]},{init,do_boot,3,[]}]})

that occur on some machines but not others.
@ivan ivan force-pushed the erlang-disable-parallel-building branch from 649ffbc to 82e5590 Compare December 8, 2018 01:00
@Mic92 Mic92 merged commit b4a2ed8 into NixOS:staging Dec 8, 2018
@trofi trofi mentioned this pull request Jan 17, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mic92 Mic92 Mic92 left review comments

@hedning hedning hedning approved these changes

Assignees
No one assigned
Labels
6.topic: erlang 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ivan @hedning @Mic92 @GrahamcOfBorg