Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Staging next #89388

Merged
merged 130 commits into from Jun 10, 2020
Merged

Staging next #89388

merged 130 commits into from Jun 10, 2020

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Jun 3, 2020

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

matthewbauer and others added 30 commits March 23, 2020 00:32
We can’t set this for cross-compiling since we use the GNU linker.
Instead, set these flags only when targetPlatform is macOS.

Fixes #80754

Fixes #83141
Addresses #71803:
Kernel options are not merged as described, especially the "optional"
aspects. The error silences legitimate warnings.
With the fix in kernel configuration merging, some kernel configuration items
marked as mandatory now correctly trigger an error when unused (while they
previously were unused).
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
This is achieved by patching qtbase `qmake/generators/makefile.cpp` to
unconditionally add the missing `-I${includedir}`. The include path is
otherwise conditioned on whether it is already available or not. Since there is
no unified set of system include paths in nix this cause problems as reported
in #52457.
*-wrapper; Switch from `infixSalt` to `suffixSalt`
This reverts commit 237ef30.

Looks like we're trying to bump binutils again?
bintools: only add macos flags when targeting macOS
The compiler does not need it anymore, has not needed it for many years
iirc. This just goes in and pollutes the environment overriding the
users GOPATH and causing grief.

Go even warns about it itself, without vs with this commit:

```sh
~> go env GOPATH
/home/manny/go
~> nix-shell -p go
~> go env GOPATH
warning: GOPATH set to GOROOT (/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go) has no effect
/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go
~> exit
~> nix-shell -I nixpkgs=cloned/NixOS/nixpkgs -p go
~> go env GOPATH
/home/manny/go
~> exit
```
There are two ways to build a package for aarch32 on an aarch64 machine:
either by cross compiling as normal, or by adding armv6l/armv7l to
extraPlatforms and doing a non-cross compile.

Previously, NSS failed to build with both methods: when using
extraPlatforms, things failed because NSS includes an armv8-specific
file (presumably based on the result of uname); when cross compiling,
NSS's build system expects to receive an architecture name of arm (not
armv6l or whatever), so was failing to include some arch-specific code
and failed with a linker error.

This commit fixes those things by a) always passing the arch, even when
not cross-compiling, and b) special-casing aarch32 to always pass in an
arch of arm.
nss: fix building for aarch32 on aarch64
* format with nixpkgs-fmt
* reorder the attributes
* use pkg-config instead of the pkgconfig alias
* optional → optionals
* remove top-level `with lib;`
* remove glibcLocales now that glibc contains C.UTF-8
* remove libintl, that should be in by default or something
* update homepage
* add gnome team to maintainers
* remove the temporary libregress closer its creation
stdenv-darwin: now with 50% less LLVM!
@FRidh
Copy link
Member Author

FRidh commented Jun 7, 2020

Could we not deliver staging-next as is, and go for the next cycle with the fix? It means cross is broken temporarily but it would save another rebuild.

edit:
note a large part of current staging is built already for python-unstable job.

@vcunat
Copy link
Member

vcunat commented Jun 7, 2020

For me personally, a few channel updates with broken cross certainly won't cause issues, but I don't know how it's generally in the community.

@FRidh
Copy link
Member Author

FRidh commented Jun 7, 2020

mplayer now fails with

/nix/store/klg7ribk7f834yb0hn1fxs6653wzlncb-binutils-2.31.1/bin/ld: sub/font_load_ft.o: undefined reference to symbol 'FT_Get_Glyph'
/nix/store/klg7ribk7f834yb0hn1fxs6653wzlncb-binutils-2.31.1/bin/ld: /nix/store/xvr907jajxllac8lygj6w0bkvwi84nfp-freetype-2.10.2/lib/libfreetype.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [Makefile:747: mplayer] Error 1
builder for '/nix/store/8wq1r408xl9wynyn4gc2b1kgvg52i3i7-mplayer-1.4.drv' failed with exit code 2

https://hydra.nixos.org/build/121519361

@vcunat
Copy link
Member

vcunat commented Jun 7, 2020

That sounds like some pkg-config as well

sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression
Package freetype2 was not found in the pkg-config search path.
Perhaps you should add the directory containing `freetype2.pc'
to the PKG_CONFIG_PATH environment variable
No package 'freetype2' found

@Ericson2314
Copy link
Member

I was still looking at autogen---the version we had already contained the fix, yet configure didn't seem to reflect it, so I was investingating what autoreconfHook would do.

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 7, 2020

Ah, so https://savannah.gnu.org/support/?109050 and autoreconfHook weren't enough, because until https://git.savannah.gnu.org/cgit/autogen.git/commit/?h=5cbe233387d7f7b36752736338d1cd4f71287daa there were scripts producing autoconf 😱 in our version, and one of those PKG_CONFIG from the patch was evaluated too early. I'll make a manual patch until we bump the versions.

See comments for details. Patch can be removed whend version is bumped.
@vcunat
Copy link
Member

vcunat commented Jun 7, 2020

Great, cross-compiles for me now. I hurried Hydra to build the commit.

Unless you object I'll just aim directly for staging with that autogen update.

@samueldr
Copy link
Member

samueldr commented Jun 7, 2020

I'm not sure what's the frequency of staging-next being merged into master, but it does mean that the mobile nixos jobset will fail for that whole duration.

Not ideal, though I know being tier 3 it's okay if it fails.

My main concern with accepting regressions in master for cross-compilation is that they sometimes end up hard to bisect. Having multiple staging-next rounds to bisect through is way harder than only one.

(Though for this particular round, the point is likely moot, I'll test with the added fix soon.)

@Ericson2314
Copy link
Member

@samueldr what other packages are broken?

I dunno what portion of the breakage is caused by me, but I do feel bad when people bisect an error which I can tell at a glance (like the pkg-config) so do feel free to ping me with issues before bisecting. I never got good at monitoring Hydra, but I do see GitHub pings pretty quick.

@vcunat
Copy link
Member

vcunat commented Jun 7, 2020

To be clear, autogen cross seems fixed in current staging-next.

@samueldr
Copy link
Member

samueldr commented Jun 7, 2020

@Ericson2314 that was more of a general sentiment. This round there was a difficulties bisecting because of breakage you already fixed "internal" to that round. Though that's not really much.

I'm more worried about a situation where, let's say no one catches issues for a month or two, it can become harder to untangle.

(Still haven't validated whether there are issues elsewhere, will be doing it soon, once what's burning my CPU time is over.)

Ericson2314 and others added 3 commits June 8, 2020 00:29
This was very similar to the Mesa issues fixed in
62e6d73: the user-written code is
looking up an unprefixed binutils program.

[I think we should have a way in Meson of specifying a program prefix in
the cross / native files, as a fallback for any program that isn't
explicitly specified. This could both be availible for user written
rules, and help with the default rules.]

Fixes NixOS/mobile-nixos#161
Otherwise we'd be delaying this staging-next cycle noticeably.
@samueldr samueldr dismissed their stale review June 8, 2020 18:29

Issue raised is fixed

@samueldr
Copy link
Member

samueldr commented Jun 8, 2020

Looking into it, there are issues still, but they are leafy, and less problematic, I guess they can be handled on then next cycle.

perlNetDBus and bcache-tools:

  • pkgsCross.aarch64-multiplatform.perlPackages.NetDBus
cannot run pkg-config to check dbus version at Makefile.PL line 8.
  • pkgsCross.aarch64-multiplatform.bcache-tools
/nix/store/n313xks5ym0s0a5v8a5285rmnmvy6ms9-bash-4.4-p23/bin/bash: pkg-config: command not found

Both issues seem to be related to the pkg-config change. I think the change may require some good documentation work as I fear many upstream packages are doing maybe the wrong thing. I say documentation, I mean it could be upstream pkg-config documentation explaining it, it even would be better, but the goal is to have some solid piece of evidence to point to when providing upstream patches.

@jtojnar
Copy link
Contributor

jtojnar commented Jun 8, 2020

We could make page like https://github.com/jtojnar/cmake-snips#please-fix-your-cmake-builds for cross-compilation.

In that case it specified $doc output but didn't even create it.
I expect it's better to do it this way instead of creating it
as an empty directory.
(Only the failed builds get rebuilt by this commit.)
@vcunat
Copy link
Member

vcunat commented Jun 9, 2020

With this gitMinimal fix we're running into a bit of hydra issue. When you fix a fixed-output derivation (fetchgit here), the output remains the same... therefore Hydra reuses the old build instead of creating a new one, and restarting that build naturally uses the first derivation that created it.

@vcunat
Copy link
Member

vcunat commented Jun 9, 2020

Builds of all kernels except default 5.4 are broken by this staging-next iteration: #88946 (review)

vcunat added a commit that referenced this pull request Jun 10, 2020
I re-checked that pkgsCross.aarch64-multiplatform.autogen builds.
#89388 (comment)
vcunat and others added 3 commits June 10, 2020 09:41
In particular, this fixes ISO evaluation.
Some of the options didn't have correct kernel version constraints,
others had been removed or made optional unnecessarily in #84032.
@vcunat vcunat merged commit a5f5d02 into master Jun 10, 2020
@vcunat
Copy link
Member

vcunat commented Jun 10, 2020

Next staging iteration: #90058

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet