@FRidh, thanks for your PR! By analyzing the annotation information on this pull request, we identified @domenkozar, @offlinehacker and @adnelson to be potential reviewers

Perhaps you could make the bin derivation just be another output of the env one, using multiple-outputs nix feature?

Indeed. python.buildEnv will need to get an extra parameter, outputs ? [ "out" ] then which will be "bin" in case of an application.

or could Python applications also create other output that we'd be interested in?

I couldn't figure out a way to give outputs to pkgs.buildEnv. Since it uses its own builder I don't think it has been implemented. Is that correct @vcunat ?

Example:

$ nix-shell -E 'with import <nixpkgs>{}; (pkgs.python35.buildEnv.override { extraLibs=[python35Packages.pytest]; outputs=["bin"];}).env'  -I nixpkgs=.
error: anonymous function at /home/freddy/Data/Data/Code/libraries/nixpkgs/pkgs/build-support/buildenv/default.nix:7:1 called with unexpected argument ‘outputs’, at /home/freddy/Data/Data/Code/libraries/nixpkgs/pkgs/development/interpreters/python/wrapper.nix:12:6
(use ‘--show-trace’ to show detailed location information)

Actually, using multiple-outputs won't work; we don't just want all of /bin, we only want the items that are in the package we're interested in. If we take /bin, we also get python, and then we can't install multiple Python applications simultaneously. @vcunat , you can ignore my question :)

@adnelson It can indeed use some cleanup. Note that the intention of this PR was to show how we can solve an issue, not really merging it as is. Implementing certain things takes a lot of time, so I rather get feedback first. Thanks for that.

In any case, buildEnv always produced a single output, at least around the time of the last NixCon.

I think this would be a really good way to avoid the more or less fragile PYTHONPATH solution. Are there any downsides?

What can we do about the missing multiple output support? Any new ideas?

Something like this is also on the todo list of triton: triton/triton#64

Are there any downsides?

Not that I can think of. I would prefer if there weren't any wrappers at all, but that would mean copying the contents of bin to the new env.

After some more thinking about this issue, I think PYTHONPATH isn't too bad either. The problem was just that we used prefix instead of set. However, if we use set, we break some other stuff, like the checkPhase. In #17749 I wrote that using set would break ipython, however, ipython would still be usable when creating an environment. Furthermore, and this is I think key to not using set PYTHONPATH, is that PYTHONPATH is an environment variable you might want to modify outside of Nix to add impure modules for whatever reason.

What can we do about the missing multiple output support? Any new ideas?

What do you mean with missing? Why do we need multiple outputs?

Are there any specific issues you're facing with the current implementation? That is, what Nixpkgs now uses.

Exactly! This is what I thought too.

Nope! I didn't read all coments carefully enough. Someone mentioned
multiple outputs a few comments ago and I thought that this is still a
missing piece.

@domenkozar What do you think about it?

What's the status here?

I still think this is the way forward, but this won't make it in before 17.09 though. I have no idea when I could continue on this so I'm closing this now.