Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

firefox: use gold linker to speedup build #98331

Closed
wants to merge 1 commit into from

Conversation

avnik
Copy link
Contributor

@avnik avnik commented Sep 20, 2020

Motivation for this change

Passing --enable-linker=gold speed-up final linking a bit. (not significant, in compare with gkrust crate building, but should save half minute or so).

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@andir
Copy link
Member

andir commented Sep 20, 2020

Is there any change in binary output size?

@avnik
Copy link
Contributor Author

avnik commented Sep 22, 2020

With BFD ld (both listings manually cleaned from non-elf)

[avn@bulldozer:pts/7]~/nixos/nixpkgs% ls -la /nix/store/dvm88pk05kglwz36hmf1jzm12v6m511v-firefox-unwrapped-80.0.1/lib/firefox/
total 78M
417K -r-xr-xr-x     2 root root 668K Jan  1  1970 firefox
417K -r-xr-xr-x     2 root root 635K Jan  1  1970 firefox-bin
 25K -r-xr-xr-x     2 root root  42K Jan  1  1970 liblgpllibs.so
985K -r-xr-xr-x     2 root root 1,8M Jan  1  1970 libmozavcodec.so
125K -r-xr-xr-x     2 root root 203K Jan  1  1970 libmozavutil.so
4,5K -r-xr-xr-x     2 root root  14K Jan  1  1970 libmozgtk.so
 93K -r-xr-xr-x     2 root root 147K Jan  1  1970 libmozsandbox.so
681K -r-xr-xr-x     2 root root 896K Jan  1  1970 libmozsqlite3.so
8,5K -r-xr-xr-x     2 root root  18K Jan  1  1970 libmozwayland.so
 62M -r-xr-xr-x     2 root root 106M Jan  1  1970 libxul.so
421K -r-xr-xr-x     2 root root 639K Jan  1  1970 pingsender
413K -r-xr-xr-x     2 root root 627K Jan  1  1970 plugin-container

With ld.gold:

[avn@bulldozer:pts/7]~/nixos/nixpkgs% ls -la /nix/store/h48wy8rk885lcjj4fp6z878711n6jxg6-firefox-unwrapped-80.0.1/lib/firefox/                                                
total 78M
417K -r-xr-xr-x     2 root root 656K Jan  1  1970 firefox
413K -r-xr-xr-x     2 root root 627K Jan  1  1970 firefox-bin
 25K -r-xr-xr-x     2 root root  43K Jan  1  1970 liblgpllibs.so
989K -r-xr-xr-x     2 root root 1,8M Jan  1  1970 libmozavcodec.so
125K -r-xr-xr-x     2 root root 203K Jan  1  1970 libmozavutil.so
4,5K -r-xr-xr-x     2 root root 6,0K Jan  1  1970 libmozgtk.so
 93K -r-xr-xr-x     2 root root 151K Jan  1  1970 libmozsandbox.so
685K -r-xr-xr-x     2 root root 900K Jan  1  1970 libmozsqlite3.so
4,5K -r-xr-xr-x     2 root root  11K Jan  1  1970 libmozwayland.so
 62M -r-xr-xr-x     2 root root 107M Jan  1  1970 libxul.so
417K -r-xr-xr-x     2 root root 631K Jan  1  1970 pingsender
409K -r-xr-xr-x     2 root root 619K Jan  1  1970 plugin-container

I am not sure what should be preffered here -- build time/builder memory consumption or final executables size
(I will check one more time with exact with/without builds, but it take some time)

@ajs124
Copy link
Member

ajs124 commented Sep 22, 2020

IMHO what matters most is runtime performance of firefox. Which is why I've been trying to get around to implementing what #76484 (comment) describes.

That would switch the linker to lld, because it builds with clang.

@avnik
Copy link
Contributor Author

avnik commented Sep 22, 2020

@ajs124 I am ok with --enable-linker=lld as it should have build/memory time similar to gold. But idk if LTO/PGO make local builds impossible or not.

@andir
Copy link
Member

andir commented Oct 2, 2020

As a first step this should be fine. When we finally manage PGO/LTO we can still switch to lld.

@avnik
Copy link
Contributor Author

avnik commented Oct 2, 2020

There is --enable-linker=lld also possible.
How PGO/LTO would affect on build machine requirements btw?

@andir
Copy link
Member

andir commented Oct 2, 2020

There is --enable-linker=lld also possible.
How PGO/LTO would affect on build machine requirements btw?

Last time it at least doubled the required storage requirements. The build took easily 2x the time it already takes. Part of that is probably that you've to create profile first and then feed that back into a second build.

@S-NA
Copy link
Contributor

S-NA commented Oct 5, 2020

But idk if LTO/PGO make local builds impossible or not.

LTO does not make local builds impossible. Well, not that I have found. It builds within reasonable time comparable to non-LTO Firefox. As for PGO, it is inherently nondeterministic to my understanding because the profile created depends on the system it is built on. It might be possible for the PGO profiles to generated and then distributed so the determinism problem is in a way solved, but I have not looked into whether Firefox's build system supports that. That being said I have not given PGO a serious attempt.

As for LTO support please see #76484 (comment). It has a common.nix which supports it which depends on the linker being lld and compiler clang.

@andir
Copy link
Member

andir commented Oct 24, 2020

Since we've added LTO support (which unfortunately broke wayland support and is thus disabled by default) do we still want this as well? I am a bit more cautious with changing the linker etc..

@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Nov 27, 2020

@andir @avnik ping

@avnik
Copy link
Contributor Author

avnik commented Nov 30, 2020

We can try use lld as "isolated change of linker" and let look if something breaks. (but I need remember how to add lld to buildInputs first)

@andir
Copy link
Member

andir commented Nov 30, 2020

I would rather go to the full LLVM road after I've spent a sunday debugging aarch64 builds. Mozilla doesn't test builds with GCC anymore so it will always be a battle to support GCC. There are a enough distributions that will still use GCC so there will be support (and patches) for that but I am not convinced we want to fight that battle for a marginal benefit.

@stale
Copy link

stale bot commented Jun 2, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 2, 2021
@oxalica
Copy link
Contributor

oxalica commented Dec 1, 2021

firefox uses LLVM lld by default if possible now. This issue is outdated.

@oxalica oxalica closed this Dec 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants