New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc/python: improve description of community tools #75074
Conversation
Added pip2nix, nixpkgs-pytools and poetry2nix. Removed [python2nix](https://github.com/proger/python2nix), because it's been broken [since 2017](NixOS#30887) and is [still broken](proger/python2nix#6).
- [nixpkgs-pytools](https://github.com/nix-community/nixpkgs-pytools): tools for removing the tedious nature of creating nixpkgs derivations. | ||
- [poetry2nix](https://github.com/nix-community/poetry2nix): turn Poetry projects into Nix derivations without the need to actually write Nix expressions. | ||
|
||
Note that pypi2nix maintains a separate package tree from nixpkgs, so Python package derivations from pypi2nix and nixpkgs [should not be used together](https://github.com/nix-community/pypi2nix/issues/222#issuecomment-443497376). But Python package derivations from pip2nix or nixpkgs-pytools can be used together with those from nixpkgs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The part that's confusing here is it leaves out poetry2nix, can it be used together with nixpkgs?
If it can, can we phrase this like,
should not be used together. The other tools mentioned don't have this limitation.
Good idea. I think poetry2nix can be used this way, but I’m not 100% sure. I’m more familiar with the other tools.
… On Dec 5, 2019, at 6:48 PM, worldofpeace ***@***.***> wrote:
@worldofpeace requested changes on this pull request.
In doc/languages-frameworks/python.section.md:
>
-- [pypi2nix](https://github.com/nix-community/pypi2nix): Generate Nix expressions for your Python project. Note that [sharing derivations from pypi2nix with nixpkgs is possible but not encouraged](nix-community/pypi2nix#222 (comment)).
-- [python2nix](https://github.com/proger/python2nix) by Vladimir Kirillov.
+- [pypi2nix](https://github.com/nix-community/pypi2nix): generate Nix expressions for Python packages.
+- [pip2nix](https://github.com/nix-community/pip2nix): generate Nix expressions for Python packages.
+- [nixpkgs-pytools](https://github.com/nix-community/nixpkgs-pytools): tools for removing the tedious nature of creating nixpkgs derivations.
+- [poetry2nix](https://github.com/nix-community/poetry2nix): turn Poetry projects into Nix derivations without the need to actually write Nix expressions.
+
+Note that pypi2nix maintains a separate package tree from nixpkgs, so Python package derivations from pypi2nix and nixpkgs [should not be used together](nix-community/pypi2nix#222 (comment)). But Python package derivations from pip2nix or nixpkgs-pytools can be used together with those from nixpkgs.
The part that's confusing here is it leaves out poetry2nix, can it be used together with nixpkgs?
If it can, can we phrase this like,
should not be used together. The other tools mentioned don't have this limitation.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
I guess we can ask @adisbladis 👋 |
You can use Other than that poetry's lockfile contains the whole environment, so you shouldn't really need to. The short answer: You can but it's discouraged and should be discouraged for all other tools too because it's unsafe. |
Note: I took a quick look at how |
Thanks for the feedback, @adisbladis. Could you explain a little more why it’s unsafe to mix pip2nix python packages with nixpkgs python packages? Note the pip2nix README says:
“Major difference between pip2nix and pypi2nix is that pip2nix can be used to extend nixpkgs Python package set and reuse its package functions where pypi2nix maintains separate package tree from nixpkgs. Bot approaches have their own pros and cons.”
@datakurre, could you comment on this pull request regarding documentation on mixing pip2nix python packages with nixpkgs python packages?
|
@adisbladis I'm also interested in hearing, what makes mixing with nixpkgs' Python package set unsafe? In my understanding it could be very fragile (prone to conflicting paths), but I'm not aware of safety issues. @ariutta Whenever pip2nix is used to extend nixpkgs' Python packages with a set that includes a package already in nixpkgs, it is very easy to get a result with multiple versions of the same package. That said, I've been trying to maintain pip2nix, because I have been using it for years to develop and build Python-based microservices with my "setup.nix"-project (and most of "setup.nix" is about aligning pip2nix-package set with nixpkgs set to prevent conflicts by duplicates). Here's an example of my projects that has recently used Nix based Python development environment with reasonable big package sets: https://github.com/robots-from-jupyter/robotkernel |
@worldofpeace, the feedback from @adisbladis and @datakurre suggests there is not a clear consensus for how to word the last sentence of my pull request. Would you prefer to just delete that sentence, or should we try to come up with a consensus? @adisbladis and @datakurre, does the following sound OK?
|
Most solutions can possibly have more than one version of a package in a closure. This can easily happen if something in nixpkgs overrides a dependency in a Take the following expression as an example: https://github.com/NixOS/nixpkgs/blob/ac4b42434a52ad4d0b728c9a59acae2a73df755e/pkgs/development/python-modules/poetry/default.nix.
This means that you cannot reasonably know which version of cleo this expression will use at runtime:
{ buildPythonApplication, cleo, poetry }:
buildPythonApplication {
propagatedBuildInputs = [
poetry
cleo
];
...
} The only safe way AFAIK to mix anything generated with nixpkgs is through I have personally been bitten by this and ended up using an outdated version of I'm guessing the cause of this is not enough awareness/understanding of these mechanics by maintainers. |
My recommendation: Either remove any section on using generated python sets together with python packages from nixpkgs or put a big fat blanket warning label that this is something to be avoided. |
That's also good with me @ariutta, you don't have to be even specific. (maybe an example could be useful to describe what that looks like) |
I can agree with the comments above too 👍 |
Hello, I'm a bot and I thank you in the name of the community for your contributions. Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do: If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list. If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past. If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments. Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel. |
Motivation for this change
Added pip2nix, nixpkgs-pytools and poetry2nix, because they are actively maintained and in nix-community. Removed python2nix, because it's been broken since 2017 and is still broken.
Things done
sandbox
innix.conf
on non-NixOS linux)nix-shell -p nix-review --run "nix-review wip"
./result/bin/
)nix path-info -S
before and after)Notify maintainers
cc @