Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ghc, haskell infra: #40642 direct to master #40929

Merged
merged 12 commits into from May 22, 2018

Conversation

Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented May 22, 2018

Motivation for this change

#40642 but not to staging so @peti can merge it to haskell-updates and then master. [Be sure to merge not rebase or the commits won't be shared with staging.]

@peti I removed the staging merge, and then this merges with master cleanly.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

Ericson2314 and others added 12 commits May 18, 2018 17:49
Just like with the other `--extra-*` flags, cc/ld-wrapper already handles
this, but we need to make Cabal aware so that the haskell builds have
the correct metadata.
…Inputs

This is because they are just for Setup.hs, so they are just used at build time
and completely isolated from the normal components' dependencies.

This was previous implemented in 8a8f040, but
reverted in e69c7f5 because it broken
setup-depends non-cross in haskell shell environments (custom Setup.hs in cross
shell environments has never worked). This version adds a special native
exception to avoid that breakage.
…taticlib

The reason why this does not work is not that we can't built static
objects, we can, but we can't use `-staticlib` on GHC on windows.
`-staticlib` rolls all dependencies into a combined archive. While this
would work on windows if we used gnu ar and MRI script, GHC can't rely
on GNU ar, and as such has a quick archive concatenation module for GNU
and BSD archives only.
Shell glob works even as the exact set of executable (filenames) varries
beween configuations.
nixpkgs#37012 and  nixpkgs#37707 introduces the setup-hooks for libiconv, which inject `-liconv` into the `NIX_LDFLAGS`. This breaks horribly on windows where the linker end up having no idea how to linke `-liconv`. The configure.ac file specifically ignores libiconv on windows.
@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: ghc

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: ghc

Partial log (click to expand)

clang-5.0: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
"inplace/bin/ghc-stage1" -hisuf hi -osuf  o -hcsuf hc -static  -H32m -O -Wall      -this-unit-id base-4.10.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -I/nix/store/6xismd1mcjqnsj873wmx1fr0p62v13f2-libiconv-osx-10.11.6/include  -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id rts -package-id ghc-prim-0.5.1.1 -package-id integer-gmp-1.0.1.0 -this-unit-id base -XHaskell2010 -O2  -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags     -Wnoncanonical-monad-instances  -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build   -dynamic-too -c libraries/base/./GHC/Event/Manager.hs -o libraries/base/dist-install/build/GHC/Event/Manager.o -dyno libraries/base/dist-install/build/GHC/Event/Manager.dyn_o
"inplace/bin/ghc-stage1" -hisuf hi -osuf  o -hcsuf hc -static  -H32m -O -Wall      -this-unit-id base-4.10.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -I/nix/store/6xismd1mcjqnsj873wmx1fr0p62v13f2-libiconv-osx-10.11.6/include  -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id rts -package-id ghc-prim-0.5.1.1 -package-id integer-gmp-1.0.1.0 -this-unit-id base -XHaskell2010 -O2  -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags     -Wnoncanonical-monad-instances  -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build   -dynamic-too -c libraries/base/./GHC/Stack.hs -o libraries/base/dist-install/build/GHC/Stack.o -dyno libraries/base/dist-install/build/GHC/Stack.dyn_o
clang-5.0: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall      -this-unit-id base-4.10.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -I/nix/store/6xismd1mcjqnsj873wmx1fr0p62v13f2-libiconv-osx-10.11.6/include  -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id rts -package-id ghc-prim-0.5.1.1 -package-id integer-gmp-1.0.1.0 -this-unit-id base -XHaskell2010 -O2  -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags     -Wnoncanonical-monad-instances  -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build    -c libraries/base/./GHC/IO/Handle/FD.hs-boot -o libraries/base/dist-install/build/GHC/IO/Handle/FD.p_o-boot -dyno libraries/base/dist-install/build/GHC/IO/Handle/FD.dyn_o-boot
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall      -this-unit-id base-4.10.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -I/nix/store/6xismd1mcjqnsj873wmx1fr0p62v13f2-libiconv-osx-10.11.6/include  -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id rts -package-id ghc-prim-0.5.1.1 -package-id integer-gmp-1.0.1.0 -this-unit-id base -XHaskell2010 -O2  -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags     -Wnoncanonical-monad-instances  -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build    -c libraries/base/./GHC/IO/Handle.hs-boot -o libraries/base/dist-install/build/GHC/IO/Handle.p_o-boot -dyno libraries/base/dist-install/build/GHC/IO/Handle.dyn_o-boot
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall      -this-unit-id base-4.10.1.0 -hide-all-packages -i -ilibraries/base/. -ilibraries/base/dist-install/build -Ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/./autogen -Ilibraries/base/dist-install/build/./autogen -Ilibraries/base/include -I/nix/store/6xismd1mcjqnsj873wmx1fr0p62v13f2-libiconv-osx-10.11.6/include  -optP-DOPTIMISE_INTEGER_GCD_LCM -optP-include -optPlibraries/base/dist-install/build/./autogen/cabal_macros.h -package-id rts -package-id ghc-prim-0.5.1.1 -package-id integer-gmp-1.0.1.0 -this-unit-id base -XHaskell2010 -O2  -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags     -Wnoncanonical-monad-instances  -odir libraries/base/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build    -c libraries/base/./GHC/IO/Exception.hs -o libraries/base/dist-install/build/GHC/IO/Exception.p_o -dyno libraries/base/dist-install/build/GHC/IO/Exception.dyn_o
clang-5.0: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
building of '/nix/store/4w5bk39r25f5791f09p7gv31j42hl15g-ghc-8.2.2.drv' timed out after 1800 seconds
�[31;1merror:�[0m build of '/nix/store/4w5bk39r25f5791f09p7gv31j42hl15g-ghc-8.2.2.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: ghc

Partial log (click to expand)

<<ghc: 312580080 bytes, 64 GCs, 12455216/33973560 avg/max bytes residency (6 samples), 72M in use, 0.000 INIT (0.000 elapsed), 0.235 MUT (0.467 elapsed), 0.235 GC (0.245 elapsed) :ghc>>
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header   -this-unit-id ghc-8.2.2 -hide-all-packages -i -icompiler/backpack -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2   -optP-DGHCI -optP-include -optPcompiler/stage2/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id deepseq-1.4.3.0 -package-id directory-1.3.0.2 -package-id process-1.6.1.0 -package-id bytestring-0.10.8.2 -package-id binary-0.8.5.1 -package-id time-1.8.0.2 -package-id containers-0.5.10.2 -package-id array-0.5.2.0 -package-id filepath-1.4.1.2 -package-id template-haskell-2.12.0.0 -package-id hpc-0.6.0.3 -package-id transformers-0.5.2.0 -package-id ghc-boot-8.2.2 -package-id ghc-boot-th-8.2.2 -package-id ghci-8.2.2 -package-id hoopl-3.10.2.2 -package-id unix-2.7.2.2 -package-id terminfo-0.4.1.0 -Wall -fno-warn-name-shadowing -this-unit-id ghc -XHaskell2010 -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing -O2  -no-user-package-db -rtsopts      -Wnoncanonical-monad-instances  -odir compiler/stage2/build -hidir compiler/stage2/build -stubdir compiler/stage2/build    -c compiler/codeGen/StgCmmMonad.hs -o compiler/stage2/build/StgCmmMonad.p_o -dyno compiler/stage2/build/StgCmmMonad.dyn_o
<<ghc: 1270401792 bytes, 130 GCs, 17377147/54344376 avg/max bytes residency (8 samples), 117M in use, 0.000 INIT (0.000 elapsed), 1.135 MUT (1.431 elapsed), 0.592 GC (0.619 elapsed) :ghc>>
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header   -this-unit-id ghc-8.2.2 -hide-all-packages -i -icompiler/backpack -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2   -optP-DGHCI -optP-include -optPcompiler/stage2/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id deepseq-1.4.3.0 -package-id directory-1.3.0.2 -package-id process-1.6.1.0 -package-id bytestring-0.10.8.2 -package-id binary-0.8.5.1 -package-id time-1.8.0.2 -package-id containers-0.5.10.2 -package-id array-0.5.2.0 -package-id filepath-1.4.1.2 -package-id template-haskell-2.12.0.0 -package-id hpc-0.6.0.3 -package-id transformers-0.5.2.0 -package-id ghc-boot-8.2.2 -package-id ghc-boot-th-8.2.2 -package-id ghci-8.2.2 -package-id hoopl-3.10.2.2 -package-id unix-2.7.2.2 -package-id terminfo-0.4.1.0 -Wall -fno-warn-name-shadowing -this-unit-id ghc -XHaskell2010 -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing -O2  -no-user-package-db -rtsopts      -Wnoncanonical-monad-instances  -odir compiler/stage2/build -hidir compiler/stage2/build -stubdir compiler/stage2/build    -c compiler/main/GhcMonad.hs -o compiler/stage2/build/GhcMonad.p_o -dyno compiler/stage2/build/GhcMonad.dyn_o
<<ghc: 560395336 bytes, 83 GCs, 13511620/42420960 avg/max bytes residency (7 samples), 90M in use, 0.000 INIT (0.000 elapsed), 0.517 MUT (0.793 elapsed), 0.277 GC (0.298 elapsed) :ghc>>
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header   -this-unit-id ghc-8.2.2 -hide-all-packages -i -icompiler/backpack -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2   -optP-DGHCI -optP-include -optPcompiler/stage2/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id deepseq-1.4.3.0 -package-id directory-1.3.0.2 -package-id process-1.6.1.0 -package-id bytestring-0.10.8.2 -package-id binary-0.8.5.1 -package-id time-1.8.0.2 -package-id containers-0.5.10.2 -package-id array-0.5.2.0 -package-id filepath-1.4.1.2 -package-id template-haskell-2.12.0.0 -package-id hpc-0.6.0.3 -package-id transformers-0.5.2.0 -package-id ghc-boot-8.2.2 -package-id ghc-boot-th-8.2.2 -package-id ghci-8.2.2 -package-id hoopl-3.10.2.2 -package-id unix-2.7.2.2 -package-id terminfo-0.4.1.0 -Wall -fno-warn-name-shadowing -this-unit-id ghc -XHaskell2010 -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing -O2  -no-user-package-db -rtsopts      -Wnoncanonical-monad-instances  -odir compiler/stage2/build -hidir compiler/stage2/build -stubdir compiler/stage2/build    -c compiler/vectorise/Vectorise/Env.hs -o compiler/stage2/build/Vectorise/Env.p_o -dyno compiler/stage2/build/Vectorise/Env.dyn_o
<<ghc: 505135648 bytes, 87 GCs, 13997146/40359320 avg/max bytes residency (6 samples), 84M in use, 0.000 INIT (0.000 elapsed), 0.445 MUT (0.678 elapsed), 0.271 GC (0.286 elapsed) :ghc>>
"inplace/bin/ghc-stage1" -hisuf p_hi -osuf  p_o -hcsuf p_hc -static -prof -eventlog  -H32m -O -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header   -this-unit-id ghc-8.2.2 -hide-all-packages -i -icompiler/backpack -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2   -optP-DGHCI -optP-include -optPcompiler/stage2/build/./autogen/cabal_macros.h -package-id base-4.10.1.0 -package-id deepseq-1.4.3.0 -package-id directory-1.3.0.2 -package-id process-1.6.1.0 -package-id bytestring-0.10.8.2 -package-id binary-0.8.5.1 -package-id time-1.8.0.2 -package-id containers-0.5.10.2 -package-id array-0.5.2.0 -package-id filepath-1.4.1.2 -package-id template-haskell-2.12.0.0 -package-id hpc-0.6.0.3 -package-id transformers-0.5.2.0 -package-id ghc-boot-8.2.2 -package-id ghc-boot-th-8.2.2 -package-id ghci-8.2.2 -package-id hoopl-3.10.2.2 -package-id unix-2.7.2.2 -package-id terminfo-0.4.1.0 -Wall -fno-warn-name-shadowing -this-unit-id ghc -XHaskell2010 -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing -O2  -no-user-package-db -rtsopts      -Wnoncanonical-monad-instances  -odir compiler/stage2/build -hidir compiler/stage2/build -stubdir compiler/stage2/build    -c compiler/parser/Parser.hs -o compiler/stage2/build/Parser.p_o -dyno compiler/stage2/build/Parser.dyn_o
building of '/nix/store/531dg5as75brf09gj6zhd92zg2yyqq89-ghc-8.2.2.drv' timed out after 1800 seconds
error: build of '/nix/store/531dg5as75brf09gj6zhd92zg2yyqq89-ghc-8.2.2.drv' failed

@peti peti merged commit 0e35858 into NixOS:master May 22, 2018
@Ericson2314 Ericson2314 deleted the cross-ghc-for-master branch May 22, 2018 19:30
@dtzWill
Copy link
Member

dtzWill commented May 22, 2018

So I'm seeing build failures w/ghc on master-- has this been confirmed to work (x86_64, non-cross, Linux)?

If so I'll go debug why I'm seeing failures, but wanted to check things are expected to work before starting that adventure xD.

dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request May 22, 2018
…-for-master"

Avoid rebuilding (and at least for me, build failures) and postpone
these GHC changes for now.

This reverts commit 0e35858, reversing
changes made to 65c3273.
@dtzWill
Copy link
Member

dtzWill commented May 22, 2018

I posted this on IRC, but for anyone following here:

I've confirmed the failure on another machine :(. LMK if anyone's not seeing it...?

Full log (warning, 16M): https://gist.github.com/dtzWill/e17a89191bb9d5be25ddd5339b99e822
Last 1000 lines (tail -n1000): https://gist.github.com/dtzWill/b17d74f7ce81197be86540a529407ba9

@Ericson2314
Copy link
Member Author

Ericson2314 commented May 23, 2018

Worrisome. And sorry. I built some GHC on mac for sure. Building more now. Maybe I thought that used the generic builder from hscolour but it didn't.

@dtzWill
Copy link
Member

dtzWill commented May 23, 2018

No worries, I have plenty of rebuilds before this become my blocking point xD. GL sorting it out, thanks for taking a look!

@peti
Copy link
Member

peti commented May 23, 2018

ghc is broken on master: https://hydra.nixos.org/build/74555446

I'll revert this change.

jensbin pushed a commit to jensbin/nixpkgs that referenced this pull request May 25, 2018
dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request Jun 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants