Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3: Add C++ compiler support for distutils (on non gcc systems) #27649

Closed
wants to merge 3 commits into from

Conversation

knedlsepp
Copy link
Member

@knedlsepp knedlsepp commented Jul 25, 2017

Motivation for this change

Python 3.x on darwin did not correctly distinguish between C++ and C builds, so we got errors relating to missing C++ headers. The fix is analogous to the one for python2.7 done in #19585.
This fixes #18194.

Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@knedlsepp knedlsepp changed the title python3: Add C++ compiler support for distutils python3: Add C++ compiler support for distutils (for non gcc systems) Jul 25, 2017
@knedlsepp knedlsepp changed the title python3: Add C++ compiler support for distutils (for non gcc systems) python3: Add C++ compiler support for distutils (on non gcc systems) Jul 25, 2017
@langston-barrett
Copy link
Contributor

langston-barrett commented Jul 25, 2017

I just tried this out and it fixes the build of gdal and so django on OSX with Python 3.6 😄 🎆 .

@FRidh
Copy link
Member

FRidh commented Jul 26, 2017

What bugs.python.org issue is this patch linked from?

@knedlsepp
Copy link
Member Author

It's from http://bugs.python.org/issue1222585 should I add that in the commit messages?

@FRidh
Copy link
Member

FRidh commented Jul 26, 2017

@knedlsepp in a comment in the code please. Also, what about http://bugs.python.org/issue1222585#msg242706?

@knedlsepp
Copy link
Member Author

@FRidh: Thank you for the thorough look at this. This is something that I overlooked and I need to incorporate before we merge!

@@ -64,6 +64,14 @@ in stdenv.mkDerivation {
substituteInPlace configure --replace '-Wl,-stack_size,1000000' ' '
'';

patches = optionals (!(stdenv.cc.isGNU or false)) [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason not to apply this for gcc?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well. The problem does not occur when using gcc, so I thought it would be best to make the intention clear. I'm not sure if applying this unconditionally might break something else, so I am basically being cautious.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I understand it is that it currently only works because gcc doesn't complain when compiling c++ code using $CC, clang does which is arguably the correct thing to do 😄 .
In cases like this I prefer not to use conditionals, making everything more consistent and the darwin build less brittle. (if the patch fails to apply the linux build won't break)

Copy link
Member Author

@knedlsepp knedlsepp Jul 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. In that case I should probably boot up a little NixOS VM and try if that also works fine. BTW: The python27 currently also does this only on !isGNU, should we maybe change that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FRidh Just saw your post on the 2.7 version of this issue: #19585 (comment) what's your take on !isGNU?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LnL7 has a valid point but I would prefer to keep the Python interpreter as vanilla as possible and therefore only apply patch when really necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally prefer to minimize amount of "branchy" logic between platforms, if we can pull it off. I'd make this unconditional, adding a comment saying what we say here, i.e., that gcc is okay with it and clang isn't, and we're doing this so we don't bake in assumptions about weird behavior in specific compilers.

@knedlsepp knedlsepp force-pushed the fix-python3x-distutils branch 3 times, most recently from 2796f21 to 5228146 Compare July 28, 2017 15:47
@knedlsepp
Copy link
Member Author

Please do not merge ATM, just broke it.

@FRidh FRidh self-assigned this Jul 28, 2017
@knedlsepp
Copy link
Member Author

@FRidh: I patched the patch based on the comment you mentioned and did upload it to their issue tracker. My reworked PR now links to the patch I uploaded. I also put a comment in the code mentioning the issue. Did a little test with compiling a C++ BOOST_PYTHON_MODULE, which looks fine for 3.4, 3.5 and 3.6.

@FRidh
Copy link
Member

FRidh commented Jul 29, 2017

@knedlsepp thanks! Is there anything that should be done with the 2.7 version of the patch?

@knedlsepp
Copy link
Member Author

@FRidh Hm, we might: #19585 (comment)

@knedlsepp
Copy link
Member Author

@FRidh: Could we merge this? While there might still be some open issues considering dropping the CFLAGS on 2.7, I think we should nevertheless get 3.x up and running.

@FRidh
Copy link
Member

FRidh commented Aug 12, 2017

This will apply the patch unconditionally unlike your other patch.

++ optionals (!(stdenv.cc.isGNU or false)) ?

@knedlsepp
Copy link
Member Author

knedlsepp commented Aug 12, 2017

I understand your point of wanting to keep the interpreter as vanilla as possible. As the motion in #27649 (review) however was mostly: Applying the patch unconditionally will be less painful for darwin maintenance, I went with that approach instead. I think the patch failing to apply for a maintainer only testing on linux could be seen as a useful information. The 2.7-patch was actually incorporated by veprbl, I only copied the fix. I wouldn't mind re-adding the conditional again if you think there's more value the vanilla python interpreter on linux.

@veprbl
Copy link
Member

veprbl commented Aug 31, 2017

Have you tested numpy+cython with this?
There is a bit of patching there that you don't seem to engage:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/numpy/default.nix#L16-L20

Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge conflict

@knedlsepp
Copy link
Member Author

Closing in favor of: #39576

@knedlsepp knedlsepp closed this Apr 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pythonPackages.gdal broken on x86_64-darwin
7 participants