Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc python: advise against withPackages #24537

Closed
wants to merge 1 commit into from

Conversation

orivej
Copy link
Contributor

@orivej orivej commented Apr 1, 2017

This clarifies why use of withPackages is limited to other packages at best, and that it definitely should not be used in configuration.nix. Discovered in #24517.

@mention-bot
Copy link

@orivej, thanks for your PR! By analyzing the history of the files in this pull request, we identified @FRidh, @alexeymuranov and @bennofs to be potential reviewers.

@FRidh
Copy link
Member

FRidh commented Apr 1, 2017

You can use it in configuration.nix just fine, you just get collisions when using namespace packages in a Python 2.x environment.

@bennofs
Copy link
Contributor

bennofs commented Apr 1, 2017

Perhaps a better solution would be to fix withPackages to work correctly with namespace packages? Shouldn't the two __init__.py files have the same contents, so perhaps we can ignore collisions if the contents of the files don't differ?

@orivej
Copy link
Contributor Author

orivej commented Apr 1, 2017

You can use it in configuration.nix just fine, you just get collisions when using namespace packages in a Python 2.x environment.

Do you mean that it is fine to use with Python 3 and it should not be used only with Python 2?

Shouldn't the two __init__.py files have the same contents, so perhaps we can ignore collisions if the contents of the files don't differ?

They may not be identical. For example, /nix/store/12kp93bhx28ii1713p3xz9g0naz1hd3k-python2.7-configparser-3.5.0/lib/python2.7/site-packages/backports/__init__.py is

# A Python "namespace package" http://www.python.org/dev/peps/pep-0382/
# This always goes inside of a namespace package's __init__.py

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

and /nix/store/nn8chrxk6dk2k6css8x7v171ry8kw6js-python2.7-backports.shutil_get_terminal_size-1.0.0/lib/python2.7/site-packages/backports/__init__.py is just

pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

@FRidh
Copy link
Member

FRidh commented Apr 1, 2017

Note that the collisions do not occur with Python 3.x because of PEP 420.

@bennofs
Just ignoring the collisions is an easy solution but at the same time also increasing the risk of other serious collisions. Therefore I am not in favor of changing the default.

@FRidh
Copy link
Member

FRidh commented Apr 1, 2017

@orivej python.withPackages + namespace packages + Python 2.x => collisions.

All other situations work fine.

@bennofs
Copy link
Contributor

bennofs commented Apr 1, 2017

@FRidh yes that is why I was suggesting to ignore them only in the case of identical contents. But turns out that this is not enough for namespace packages, so I guess this can't really be fixed.

This is a deficit in the python 2 packaging approach, it is good that nix gives an error here and does not blindly allow a package to override the files of another package. It is similar to a merge conflict: there is sometimes no automatic solution to it, manual intervention is required.

@orivej
Copy link
Contributor Author

orivej commented Apr 1, 2017

Thanks, I updated this pull request.

Usually file collisions indicate packaging problems, but with Python 2 buildEnv environments collisions between __init__.py in namespace packages are expected and there is nothing to do but to enable ignoreCollisions = true. This should be reflected in the manual.

@bennofs
Copy link
Contributor

bennofs commented Apr 1, 2017

@FRidh can we perhaps detect namespace packages and strip comments from __init__.py if it matches the "official" __init__.py for namespace packages? I think buildEnv allows collisions between files with identical contents, so that should prevent the conflict in 99% of the cases I would assume.

For the remaining namespace packages not detected by this, we could fix up the __init__.py to match the officially recommended one in the build phase.

@Mic92
Copy link
Member

Mic92 commented Apr 17, 2017

@FRidh is the documentation good to go?

@FRidh
Copy link
Member

FRidh commented Apr 18, 2017

I've just pushed 8d491ec

@FRidh FRidh closed this Apr 18, 2017
@orivej orivej deleted the python-withpackages branch June 25, 2017 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants