Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chromium: do jumbo builds #43658

Closed
wants to merge 1 commit into from
Closed

chromium: do jumbo builds #43658

wants to merge 1 commit into from

Conversation

Ekleog
Copy link
Member

@Ekleog Ekleog commented Jul 17, 2018

Motivation for this change

This should decrease compile times, see [1].

Unfortunately, couldn't test anything, as compiler is killed (presumably due to OOM, given my complete lack of buff/cache after the kill and the fact my computer froze for a few seconds) after ~1hr on my machine. If someone has a big enough machine to build chromium and run tests/chromium.nix, that'd likely be better than merging and letting hydra figure it out, but…

[1] https://chromium.googlesource.com/chromium/src/+/lkcr/docs/jumbo.md

cc @chaoflow @bendlas

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@Ekleog
Copy link
Member Author

Ekleog commented Jul 17, 2018

(for the record, I have gentoo-using friends claiming a reduction from 9hrs to 3hrs of build time with jumbo builds)

@grahamc
Copy link
Member

grahamc commented Jul 17, 2018

Doing some testing!

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: chromium

Partial log (click to expand)

[4172/19320] CXX obj/skia/skia_core_and_effects/SkMatrixImageFilter.o
[4173/19320] CXX obj/skia/skia_core_and_effects/SkMipMap.o
[4174/19320] CXX obj/skia/skia_core_and_effects/SkMaskFilter.o
[4175/19320] CXX obj/skia/skia_core_and_effects/SkDeferredDisplayListRecorder.o
[4176/19320] CXX obj/services/video_capture/public/mojom/constants/constants.mojom.o
[4177/19320] CXX obj/skia/skia_core_and_effects/SkBlurMF.o
ninja: build stopped: subcommand failed.
builder for '/nix/store/m3wkwxdrzhyvf0azff9h45yfki1hv97q-chromium-67.0.3396.99.drv' failed with exit code 1
cannot build derivation '/nix/store/3afxqcy0n90jaiwjkap240hda67j430k-chromium-67.0.3396.99.drv': 1 dependencies couldn't be built
error: build of '/nix/store/3afxqcy0n90jaiwjkap240hda67j430k-chromium-67.0.3396.99.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: chromium

Partial log (click to expand)

[4421/19699] CXX obj/third_party/angle/libANGLE/Device.o
[4422/19699] CXX obj/third_party/angle/libANGLE/Error.o
[4423/19699] CXX obj/third_party/angle/libANGLE/Fence.o
[4424/19699] CXX obj/third_party/angle/libANGLE/Display.o
[4425/19699] CXX obj/third_party/angle/libANGLE/Context_gles_1_0.o
[4426/19699] CXX obj/third_party/angle/libANGLE/Framebuffer.o
[4427/19699] CXX obj/third_party/angle/libANGLE/HandleAllocator.o
building of '/nix/store/9c35n200x682fpbc4l51sp80gkrdkxl1-chromium-67.0.3396.99.drv' timed out after 1800 seconds
cannot build derivation '/nix/store/cf2na67i201aj0rr4a0d311d1c0a9b9r-chromium-67.0.3396.99.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cf2na67i201aj0rr4a0d311d1c0a9b9r-chromium-67.0.3396.99.drv' failed

@grahamc
Copy link
Member

grahamc commented Jul 17, 2018 •
edited

Well, it took a while to spin up the server, and there was a slow-down at the 16,000/20,000 mark, but it looks good from here:

24329.88user 2296.98system 38:56.69elapsed 1139%CPU (0avgtext+0avgdata 4137400maxresident)k
344inputs+15076664outputs (1291major+710003272minor)pagefaults 0swaps

I'll try the old chromium config to give a direct comparison.

@grahamc
Copy link
Member

grahamc commented Jul 17, 2018

This change has reduced the build on this particular machine from about 2hrs to 40min -- pretty good I'd say!

matthewbauer
matthewbauer approved these changes Jul 24, 2018
View reviewed changes
Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading through https://chromium.googlesource.com/chromium/src/+/lkcr/docs/jumbo.md this looks fairly safe. I would definitely recommend trying it in nixos-unstable. It has been blocking nixos-18.03 (https://hydra.nixos.org/job/nixos/release-18.03/nixos.tests.chromium.x86_64-linux) so might be worth backporting as well.

@YorikSar
Copy link
Contributor

@Ekleog awesome find! I wonder why didn't we use it earlier, I remember looking at unity (they used to be called that) builds when I just started looking into Chromium in NixOS, but I thought it wasn't used for some reason not known to me.

I've started build with new Chromium version on my build VM, will report results back tomorrow (unless it finishes too fast).

Btw, upstream recently (couple of weeks ago) extracted gn from Chromium repo, so we will be able to build it separately and shave a bit more time off Chromium builds.

@YorikSar
Copy link
Contributor

From the start it brings down number of targets from 30k to 20k. Around 16k it moves really slow because it starts building core browser parts in several huge jumbo pieces.

After all, it finished within an hour or so, but with an error:

[20063/20063] LINK ./chrome
FAILED: chrome
python "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome" -- g++ -pie -Wl,--version-script=../../build/linux/chrome.map -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed -fuse-ld=gold -B/nix/store/frqm8gncayv83s3hrv7hpldp865gbspc-gcc-wrapper-7.3.0/bin -m64 -Wl,-O2 -Wl,--gc-sections -nodefaultlibs -Wl,-rpath-link=. -Wl,--disable-new-dtags -Wl,--export-dynamic -L/nix/store/lgl3w7k7rm2qi99r94aarcgippppwsc5-glib-2.56.0/lib -L/nix/store/rb6pjl41sfrvmc1xv2920f1fiiyjnpn8-nspr-4.19/lib -L/nix/store/1qvvy6jiypcl3k17cca98pm9h0vzzdy9-nss-3.38/lib64 -L/nix/store/7i5vzsl3smhc2cm4liv7h60j9cljbg68-dbus-1.12.8-lib/lib -L/nix/store/zn2wkqw41wfhzc1ic9acxna1wsm1qnai-libpng-apng-1.6.34/lib -L/nix/store/1zmgrk3n6k8g6dc9m0igd9w11c49x0qx-zlib-1.2.11/lib -L/nix/store/hppcvqph8s7iwxi3ic42fhddlh3jc6iy-libpulseaudio-11.1/lib -L/nix/store/kplm8akg9zqlp8c6s5p39286pnyyrf5v-libopus-1.2.1/lib -L/nix/store/fyzial432pxv898ra2i943qdh9dswj25-cairo-1.15.12/lib -L/nix/store/iib8rczppxxkq9pqvwi20hq1vm270h2l-pango-1.42.1/lib -L/nix/store/g6ajksi37j2fi71irw0d7lyikfa42mg5-fribidi-1.0.4/lib -L/nix/store/435xiwbkjva9jbvc4n5i3c0rkmca3kfr-atk-2.28.1/lib -L/nix/store/0hkf91b5bdgbf3rv954k20pmhix74jc2-at-spi2-atk-2.26.2/lib -L/nix/store/gii255q9k5nxkszp0rz3cy3ndrn1y9lb-flac-1.3.2/lib -L/nix/store/v7q76c3cccxqqhwhl0930gldn47pav2i-gdk-pixbuf-2.36.12/lib -L/nix/store/lbphi7iab5yqzk50p1dilcb89n913vjd-gtk+3-3.22.30/lib -L/nix/store/ijkl2zjm1igk199mjcimyfsan0j890wk-libxslt-1.1.32/lib -L/nix/store/566sfdyw4npfchy3raj5i4ipl59h36rb-libxml2-2.9.8/lib -o "./chrome" -Wl,--start-group @"./chrome.rsp"  -Wl,--end-group   -lc -lgcc_s -lm -lrt -ldl -lpthread -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender
-lXtst -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lcups -lz -lcrypt -lexpat -ldbus-1 -latomic -lXss -lpng16 -lwebp -lwebpdemux -lwebpmux -lXrandr -lresolv -lsnappy -lpci -lasound -lpulse -lopus -lpangocairo-1.0 -lpango-1.0 -lfribidi -lcairo -latk-1.0 -latk-bridge-2.0 -lminizip -lFLAC -lgtk-3 -lgdk-3 -lcairo-gobject -lgdk_pixbuf-2.0 -lgio-2.0 -lxslt -lxml2
obj/components/language/content/browser/language_code_locator/language_code_locator.o:language_code_locator.cc:function language::LanguageCodeLocator::GetLanguageCode(double, double) const: error: undefined reference to 'base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::BasicStringPiece(char const*)'
obj/components/language/content/browser/language_code_locator/language_code_locator.o:language_code_locator.cc:function language::LanguageCodeLocator::GetLanguageCode(double, double) const: error: undefined reference to 'base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::BasicStringPiece(char const*)'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

I cherry-picked this PR on top of #44132, it seems jumbo build doesn't always work. I'll recheck that PR separately, but I'm sure it used to work.

@YorikSar
Copy link
Contributor

Yeah, I definitely built that PR as it is merged since I still have results in nix store. I guess jumbo build needs some fixing for latest stable.

@Ekleog
Copy link
Member Author

Ekleog commented Jul 27, 2018

Hmm, that sounds pretty bad, if upstream isn't maintaining jumbo builds… or maybe this could be due to the fact gn has been split out of Chromium, maybe?

@YorikSar
Copy link
Contributor

I really don't know why did it fail. gn separation shouldn't be the reason since I'm testing it on stable where gn is still in Chromium bundle. I'll try to dig into this link issue in coming week - I'm very excited to get shorter build times for Chromium :) (not that it would make debugging any easier)

@matthewbauer
Copy link
Member

Closing for now. Open to trying this once more of the bugs have been fixed in Chromium.

@domenkozar
Copy link
Member

Applied to master as it works nowadays: 89ede97

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewbauer matthewbauer matthewbauer approved these changes

Assignees
No one assigned
Labels
10.rebuild-darwin: 0 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@Ekleog @grahamc @GrahamcOfBorg @YorikSar @matthewbauer @domenkozar