New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chromium: build with upstream llvm & PGO #101786
Conversation
So far I have only tested that the build starts without errors, I'll report back once it's finished. |
I am under the impression that #101467 already does this. |
It's not that simple unfortunately. https://source.chromium.org/chromium/chromium/src/+/master:build/config/compiler/pgo/BUILD.gn;l=58 doesn't actually fetch any profile files, it runs the To enable fetching profiles https://source.chromium.org/chromium/chromium/src/+/master:DEPS;l=121 can be set to true, which would run the update script at https://source.chromium.org/chromium/chromium/src/+/master:DEPS;l=4961 . |
50a8fb5
to
8432b6b
Compare
Good catch. |
Result of 1 package built:
|
For some reason |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I have two nitpicks though (at least the 1st one needs to be resolved so that running the script doesn't fail - depending on the installed Python libraries). Other than that it LGTM.
For some reason nixpkgs-review only built the stable variant, the binary runs without issues.
Could be because we only built chromium
on Hydra (meta.hydraPlatforms
), not sure though if nixpkgs-review
considers that.
pkgs/applications/networking/browsers/chromium/upstream-info.json
Outdated
Show resolved
Hide resolved
Weirdly enough every benchmark I've tried has worse results on pgo than on master, I don't think this should be merged until that's fixed. Speedometer master: 141.4 Kraken master: 696.9ms Octane master: 54105 |
I am pretty sure the performance decrease is caused by mismatching clang versions. PGO builds are disabled in Ubuntu bionic-dev, the commit message says this was done because Building with mismatched profile data would normally cause compiler errors, but this is disabled in the Chromium build files. Source |
The prebuilt LLVM binaries could be used, but we would need to get |
AFAIK network access is disabled during Nix builds. I just don't know how we'd get the binaries to run, can |
Again, it would be better (and maybe easier) to generate
|
8432b6b
to
3dc3903
Compare
Ok, I've applied the changes mentioned in the review and also implemented upstream llvm builds, nixpkgs-review builds everything without issues (excluding dev, as it's marked as broken). Benchmark results look better now, Speedometer improved by ~5%, MS Chalkboard by ~6%, Octane 2 and Kraken by ~0.6%. I am just not sure how to best handle the upstream llvm builds, I have a few questions (ping @primeos):
|
3dc3903
to
92e8246
Compare
targetLlvmLibraries = llvmPackages.libraries; | ||
buildOverride = upstream-info.deps.clang; | ||
}; | ||
|
||
stdenv = llvmPackages.stdenv; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I probably should have set stdenv = llvmPackages.lldClang.stdenv
in #101467. Maybe it would be good to include?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Never mind, llvmPackages.lldClang.stdenv
is totally broken.
Sorry for my delayed reply (I'm a bit behind and only briefly glanced over the diff so far).
Yes, we should avoid unverified binaries and build as much from source as possible (personally I'm completely against pre-built binaries).
Good question :o IIRC Chromium uses LLVM master, I'm not sure how often they update it though.
First, I'd hope they don't update LLVM during patch releases for the stable channel (though IIRC it doesn't take that long to compile, especially vs. Chromium of course). But at least for Chromium dev the LLVM is probably bumped every two weeks and might be enough ahead of the one for the stable channel that it causes problems (i.e. doesn't build). So we could introduce In any case we should also involve the LLVM maintainers regarding this (IIRC they also considered switching to the monorepo build). Another problem is that the LLVM will probably break after some new updates for the beta/dev channel and we should have a dedicated maintainer to take care of this (I'm already busy enough with Chromium :o). |
92e8246
to
4d969d3
Compare
Closing because I don't have the time to maintain the custom LLVM builds. |
Motivation for this change
Google reports up to 10% faster page loads.
Source: https://blog.chromium.org/2020/08/chrome-just-got-faster-with-profile.html
Can decrease overall compile time by 20%.
Source: https://llvm.org/docs/HowToBuildWithPGO.html
Things done
sandbox
innix.conf
on non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
./result/bin/
)nix path-info -S
before and after)