Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: package set updates primarily for Python 3.8 support #71222

Merged
merged 284 commits into from Oct 18, 2019

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Oct 16, 2019

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @

@FRidh
Copy link
Member Author

FRidh commented Oct 16, 2019

@jonringer when pushing do so in batches, otherwise the evaluator runs often

@jonringer
Copy link
Contributor

@GrahamcOfBorg eval

@jonringer
Copy link
Contributor

im working on textacy, aiohttp, numpydoc, pybind11 right now. Just a heads up

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

ok, thanks

@jonringer
Copy link
Contributor

also, it seems like zope_exceptions is a common failing package for a lot of other packages..... it seems to build fine, looks like the site-packges installs fine... but none of the other packages seem to be able to import it

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

probably a namespace package using the wrong namespace implementation

@jonringer
Copy link
Contributor

jonringer commented Oct 17, 2019

that's what i thought as well, but if i load up the package in a pure nix-shell it seems to not be able to find it, even though it's on the PYTHONPATH

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

but let's ignore it, it was broken already if I am correct. I recall having looked at it before

@jonringer
Copy link
Contributor

jonringer commented Oct 17, 2019

I've also noticed that native bindings seem to be especially broken right now, similar to:

cytoolz/dicttoolz.c:11394:5: error: macro "__Pyx_PyCode_New" requires 16 arguments, but only 15 given

I'm assuming this is on the packages to update this?

I also noticed that the other python versions have cython patches, but py38 does not...

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

Correct, packages need to fix that for 3.8.

@jonringer
Copy link
Contributor

do you care for maintaining python2 support?

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

I don't. I try to keep the core packages functioning but aside from that, no. It's really up to individual maintainers. Given that python2 is EOL by the time we release 20.03 it is even less relevant.

@jonringer
Copy link
Contributor

That's what I thought, was just curious about freezing packages for python2, but I'll stop doing that as it is kind of time consuming

@jonringer
Copy link
Contributor

also, it looks like wheel==41.4 is causing some issues:

      File "/nix/store/fam3daybpxbx7jnqxvi7hmr9zqqdf80v-python3.8-setuptools-41.4.0/lib/python3.8/site-packages/setuptools/_vendor/packaging/specifiers.py", line 594, in __init__
        specifiers = [s.strip() for s in specifiers.split(",") if s.strip()]
    AttributeError: 'SpecifierSet' object has no attribute 'split'

related issue ansible/molecule#2350

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

About zope. I checked, and it appears they're migrating to the correct approach using __import__('pkg_resources').declare_namespace(__name__) but not all packages are using it yet it seems.

Don't know why they went straight for from pkgutil import extend_path __path__ = extend_path(__path__, __name__) instead.

@jonringer
Copy link
Contributor

That's the correct approach for python2... but now i dont remember where I saw the error messages.

@jonringer
Copy link
Contributor

i want to say the pkg_util is supposed to be python2 and python3 compatible:
https://packaging.python.org/guides/packaging-namespace-packages/

@FRidh
Copy link
Member Author

FRidh commented Oct 17, 2019

zope_testrunner is not failing. I am now collecting for all our zope packages the methods they use. Also consider writing a zope hook that will force the pkg_util approach instead.

@dasj19
Copy link
Contributor

dasj19 commented Dec 9, 2019

this also broke the package syncserver as it is dependent on webtest which is no longer built using python2. There is a python2 webtest package in stable but none in unstable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants