Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

treewide: Remove stdenv.isCross #44367

Merged
merged 1 commit into from Aug 2, 2018

Conversation

Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Aug 2, 2018

Motivation for this change

I want cross-specific overrides to be verbose, so I rather not have this shorthand. This makes the syntactic overhead more proportional to the maintenance cost. Hopefully this pushes people towards fewer conditionals and more abstractions.

I'd like this to get in 18.09. stdenv.isCross was made just in time for 18.03, so if we are to remove it I wouldn't like it to be around for another stable cycle.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@GrahamcOfBorg GrahamcOfBorg added the 6.topic: stdenv Standard environment label Aug 2, 2018
, swig
, ncurses
, pam
, buildPackages
}:

assert withPerl -> perl.meta.availible or false;
assert withPython -> python.meta.availible or false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*available

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this redundant? It will throw an error anyway if not.

@@ -137,7 +142,9 @@ let

inherit doCheck;

meta = apparmor-meta "user-land utilities";
meta = apparmor-meta "user-land utilities" // {
broken = !(withPython && withPerl);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this broken?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It requires those features to work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry thought this was part of the same derivation as above.

targetPrefix = lib.optionalString stdenv.isCross
(targetPlatform.config + "-");
in python3Packages.buildPythonApplication rec {
{ lib, python3Packages, stdenv, targetPlatform, writeTextDir, substituteAll }:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does meson care about targetPlatform? Sounds like a bug (an existing one though).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed. Meson is a bit confused with cross :(. I was hoping to fix it, starting with mesonbuild/meson#3689 which I thought would be uncontroversial. But it was, and then I think I burnt some credibility in the process.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean a bug in the Nix expression. This can't be right?

 60     [host_machine]
 61     system = '${targetPlatform.parsed.kernel.name}'
 62     cpu_family = '${targetPlatform.parsed.cpu.family}'
 63     cpu = '${targetPlatform.parsed.cpu.name}'
 64     endian = ${if targetPlatform.isLittleEndian then "'little'" else "'big'"}

The only place where you care about targetPlatform is when building a cross compiler. I don't see why meson would ever apply here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I don't understand what "Meson is a bit confused with cross" means. Based on my reading, it has mostly the same semantics and names for everything as autoconf?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not that bad. I fixed some terminology in https://github.com/mesonbuild/meson/blob/master/docs/markdown/Cross-compilation.md without issue (i.e. the thing itself did the mostly right thing) for example.

The concrete issue is that pops up here is Meson handles compiler env vars incorrectly.

With autoconf, CC_FOR_BUILD (or long ago + still in Linux HOST_CC) targets the build platform, CC targets the host platform, and CC_FOR_TARGET targets the target platform. The _FOR_* ones are defaulted in the native case so the build system writer doesn't need any special casing. With Meson, CC always targets the native platform, so sometimes it's for build and host, and sometimes it's for build.

What's going in Nix is a) we need to wip out our autoconf-style CC (and other variables, which isn't done yet) and b) we are making a cross file baked into meson, which is also sketchy cause we don't know what compilers we'll have in actuality. Since meson is making the file, and already a nativeBuildInput it's only a little stretch to pretend it's a cross compile and use targetPlatform according too for the build hook bug yuck.

This reflects a larger issue that meson's implementation constantly thinks "native vs foreign" rather than "build vs host vs target" which leads to tons of needless and hazardous conditional code.

I opened mesonbuild/meson#3282 before I knew it used CC incorrectly in the cross case. I suppose I should open another making it use the env vars like autoconf.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Meson, CC always targets the native platform, so sometimes it's for build and host, and sometimes it's for build.

Right, if setting CC overrides the native compiler even when a cross file is present, that will be painful. Having them support CC_FOR_BUILD would be useful indeed.

we are making a cross file baked into meson, which is also sketchy cause we don't know what compilers we'll have in actuality.

You can create the cross file in the setup hook at build-time. I presume it would be needed anyway if you ever need to tell meson about say, a cross Fortran compiler and not have meson depend on gfortran all the time.

This reflects a larger issue that meson's implementation constantly thinks "native vs foreign" rather than "build vs host vs target" which leads to tons of needless and hazardous conditional code.

Any concrete example?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having them support CC_FOR_BUILD would be useful indeed.

Glad you agree. Made mesonbuild/meson#3969 for CC/CC_FOR_BUILD.

You can create the cross file in the setup hook at build-time.

Yeah agree making on the fly (probably via setup-hook) is a more reliable way to get this done. Of if we deem Meson important enough, we can make it part of -wrapper. (If I do the big refactors I want, probably at the end up the month, those should be a lot simpler freeing up some complexity budget.)

Any concrete example?

grep for is_cross I recall. I had some local changes I could dredge up too. mesonbuild/meson#3689 would have been the first step in ensuring that all of build, host, and target, are treated symmetrically. The next step would be to make cross files and environment detection present their conclusions through the same interface so downstream stuff can read the info in a cross-agnostic manner. Right now, tons of those is_cross conditions are deciding whether to query the (detected) native platform or cross file (for host and target, when cross compiling). The alternative would mean just querying build, host, and target information not caring about is_cross or provenance (explicitly specified vs inferred).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(In mesonbuild/meson#3689 (comment), I raise trying different plans of attack that people might be more on board with.)

@@ -92,11 +94,14 @@ let

postPatch = "cd ./libraries/libapparmor";
# https://gitlab.com/apparmor/apparmor/issues/1
configureFlags = stdenv.lib.optionalString (!stdenv.isCross) "--with-python --with-perl";
configureFlags = [
(stdenv.lib.withFeature withPerl "perl")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since when is this withFeature a thing? First time I see it used.

Copy link
Member

@matthewbauer matthewbauer Aug 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enableFeature has existed for a while but I think withFeature is pretty recent addition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed I added it in 9e9cdd7.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it probably should be asked from the community whether it's desirable to do this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured since enableFeature exists, it's OK, but happy to bring in more people. @aneeshusa was pushing for things like this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main interest was using structured data for configureFlags, cmakeFlags and the like; in the cmakeFlags case this makes a bunch of these helpers unnecessary. No strong opinions about withFeature and friends in the configureFlags case but seems reasonable enough.

@dezgeg
Copy link
Contributor

dezgeg commented Aug 2, 2018

IIRC, there are also cases in Nixpkgs where we import hostPlatforms, buildPlatforms from pkgs instead of using stdenv.hostPlatforms etc.. Can we fix up those cases to be consistent?

I *want* cross-specific overrides to be verbose, so I rather not have
this shorthand. This makes the syntactic overhead more proportional to
the maintainence cost. Hopefully this pushes people towards fewer
conditionals and more abstractions.
inherit (stdenv) cc isCross;
inherit (stdenv) cc;

isCross = stdenv.buildPlatform != stdenv.hostPlatform;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe get rid of this as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh it can't be that much better until upstream changes per my above response.

@dtzWill
Copy link
Member

dtzWill commented Aug 6, 2018

Looks like stage-1.nix has reference to isCross that needs fixing?

nixos/modules/system/boot/stage-1.nix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants