New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chromium: do jumbo builds #43658
chromium: do jumbo builds #43658
Conversation
This should decrease compile times, see [1]. [1] https://chromium.googlesource.com/chromium/src/+/lkcr/docs/jumbo.md
(for the record, I have gentoo-using friends claiming a reduction from 9hrs to 3hrs of build time with jumbo builds) |
Doing some testing! |
Failure on aarch64-linux (full log) Attempted: chromium Partial log (click to expand)
|
Failure on x86_64-linux (full log) Attempted: chromium Partial log (click to expand)
|
Well, it took a while to spin up the server, and there was a slow-down at the 16,000/20,000 mark, but it looks good from here:
I'll try the old chromium config to give a direct comparison. |
This change has reduced the build on this particular machine from about 2hrs to 40min -- pretty good I'd say! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading through https://chromium.googlesource.com/chromium/src/+/lkcr/docs/jumbo.md this looks fairly safe. I would definitely recommend trying it in nixos-unstable. It has been blocking nixos-18.03 (https://hydra.nixos.org/job/nixos/release-18.03/nixos.tests.chromium.x86_64-linux) so might be worth backporting as well.
@Ekleog awesome find! I wonder why didn't we use it earlier, I remember looking at unity (they used to be called that) builds when I just started looking into Chromium in NixOS, but I thought it wasn't used for some reason not known to me. I've started build with new Chromium version on my build VM, will report results back tomorrow (unless it finishes too fast). Btw, upstream recently (couple of weeks ago) extracted gn from Chromium repo, so we will be able to build it separately and shave a bit more time off Chromium builds. |
From the start it brings down number of targets from 30k to 20k. Around 16k it moves really slow because it starts building core browser parts in several huge jumbo pieces. After all, it finished within an hour or so, but with an error:
I cherry-picked this PR on top of #44132, it seems jumbo build doesn't always work. I'll recheck that PR separately, but I'm sure it used to work. |
Yeah, I definitely built that PR as it is merged since I still have results in nix store. I guess jumbo build needs some fixing for latest stable. |
Hmm, that sounds pretty bad, if upstream isn't maintaining jumbo builds… or maybe this could be due to the fact |
I really don't know why did it fail. |
Closing for now. Open to trying this once more of the bugs have been fixed in Chromium. |
Applied to master as it works nowadays: 89ede97 |
Motivation for this change
This should decrease compile times, see [1].
Unfortunately, couldn't test anything, as compiler is killed (presumably due to OOM, given my complete lack of buff/cache after the kill and the fact my computer froze for a few seconds) after ~1hr on my machine. If someone has a big enough machine to build chromium and run
tests/chromium.nix
, that'd likely be better than merging and letting hydra figure it out, but…[1] https://chromium.googlesource.com/chromium/src/+/lkcr/docs/jumbo.md
cc @chaoflow @bendlas
Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nox --run "nox-review wip"
./result/bin/
)nix path-info -S
before and after)