Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ghc: always use ld.gold #65510

Merged
merged 1 commit into from Aug 3, 2019
Merged

Conversation

alpmestan
Copy link
Contributor

Motivation for this change

ld.gold has been known to speed up GHC compilations for some time, and is often recommended whenever linking time becomes unreasonably big. @bgamari and I thought it might be a good idea to use it by default for all GHCs that are built in nixpkgs. Both have bugs but ld.gold tends to be reliable and consistently faster.

I also have concrete numbers to back this up, at the end of this comment.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

First, a tiny script that measures the time that it takes just to link aeson's Setup.hs, producing a Setup executable, with ld.gold and ld.bfd,10 times over, recording the timings (all in seconds).

Here's a table with just the system times reported. The two files with system and user times are available at the end.

Linker Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10
bfd 15.51 15.32 15.82 17.90 17.77 16.49 16.57 15.54 15.68 15.43
gold 8.26 6.76 6.77 7.28 7.83 7.38 7.10 7.26 6.91 6.85

Second, a script that measures the time it takes to compile ghc-events-analyze (alone, without its deps) with ld.bfd and ld.gold. Like before, script & raw data available at the end.

Linker Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10
bfd 80.27 80.81 80.88 80.67 80.79 80.64 80.77 80.70 89.40 87.91
gold 74.41 74.52 73.07 75.05 74.45 74.22 73.61 74.14 75.00 85.80

Those numbers follow a clear trend that we have been seeing during our work on/around GHC, but I could definitely collect more data points if this is not convincing enough.


aeson Setup.hs data:

#!/run/current-system/sw/bin/env nix-shell
#!nix-shell -i bash -p cabal-install -p haskell.compiler.ghc844 -p binutils

# Using Cabal/Cabal's Setup.hs as an example
cabal get aeson; cd aeson-*
rm -rf Setup Setup.hi Setup.o
ghc -O Setup.lhs

for i in `seq 10`; do
  rm Setup
  command time -o ../gold -a -f '%S %U' ghc -O -optl-fuse-ld=gold Setup.lhs
  rm Setup
  command time -o ../bfd -a -f '%S %U' ghc -O -optl-fuse-ld=bfd Setup.lhs
done

cd ..

bfd.txt
gold.txt


ghc-events-analyze compilation data:

{ nixpkgs ? import <nixpkgs> {} }:
with nixpkgs;

let deps =
  [ cabal-install haskell.compiler.ghc844
    binutils zlib.dev zlib.out
  ];

in

mkShell {
  name = "ghc-gold-bench";
  buildInputs = deps;

  shellHook = ''
    export LD_LIBRARY_PATH=${zlib.out}/lib:$LD_LIBRARY_PATH
  '';
}
#!/usr/bin/env sh

# Building a whole (executable) package
cabal new-update
cabal get ghc-events-analyze
cd ghc-events-analyze*
cabal new-build --ghc-options="-optl-fuse-ld=gold"
cabal new-build --ghc-options="-optl-fuse-ld=bfd"

for i in `seq 10`; do
    rm -r dist-newstyle
    command time -o ../gold-gea -a -f '%S %U' cabal new-build --ghc-options="-optl-fuse-ld=gold"
    rm -r dist-newstyle
    command time -o ../bfd-gea -a -f '%S %U' cabal new-build --ghc-options="-optl-fuse-ld=bfd"
done
cd ..

bfd-gea.txt
gold-gea.txt

gold is known to speed up GHC builds considerably, saving up to many seconds
when linking large Haskell applications, so this patch configures all
non-binary GHC derivations to use gold.
Copy link
Member

@peti peti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@ofborg ofborg bot requested a review from peti July 29, 2019 17:03
@dtzWill
Copy link
Member

dtzWill commented Aug 1, 2019

👍 for moving to ld.gold (for GHC)! They seem to treat it as an important build option (kinda surprised it's not the default, shrug) so I don't think we'd be going out on much of a limb here.

On the subject of linkers other than ld.bfd, i'm curious what folks involved in GHC more closely (OP and such) and who might passively see the sorts of bugs/chatter/activity for early ideas-- any idea if LLD works, has benefits here, or any such tidbits?

@peti peti merged this pull request into NixOS:haskell-updates Aug 3, 2019
@alpmestan
Copy link
Contributor Author

@dtzWill I have not played around with LLD so far, maybe @bgamari has?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants