Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildPython*: propagate setuptools #68314

Closed
wants to merge 1 commit into from

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Sep 8, 2019

This was accidentally removed in
7e28bf5d8181926e600a222cb70180519d09726. Longer-term, we should not
propagate setuptools though, and let derivations explicitly declare
whether they require setuptools during runtime.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @

This was accidentally removed in
7e28bf5d8181926e600a222cb70180519d09726. Longer-term, we should not
propagate setuptools though, and let derivations explicitly declare
whether they require setuptools during runtime.
@FRidh
Copy link
Member Author

FRidh commented Sep 8, 2019

If we don't encounter too many runtime issues I think we can do without this change.

@worldofpeace
Copy link
Contributor

Noticed that this broke fwupd (which breaks gnome3 tests).
It runs a script during build and we expected setuptools to be available

FAILED: libfwupd/fwupd.map
/build/fwupd-1.2.10/libfwupd/generate-version-script.py LIBFWUPD libfwupd/Fwupd-2.0.gir libfwupd/fwupd.map
Traceback (most recent call last):
  File "/build/fwupd-1.2.10/libfwupd/generate-version-script.py", line 11, in <module>
    from pkg_resources import parse_version
ModuleNotFoundError: No module named 'pkg_resources'

Adding setuptools would be proper, but I'm betting this assumption could be made often.

@FRidh
Copy link
Member Author

FRidh commented Sep 9, 2019

Yes, it's made more often, but I think we can still fix them on a case by case basis.

@lheckemann
Copy link
Member

Is this a bugfix which should be backported to 19.09 as well?

@FRidh FRidh added this to the 19.09 milestone Sep 10, 2019
@FRidh
Copy link
Member Author

FRidh commented Sep 10, 2019

For now I would like to avoid merging and backporting it and fix case-by-case. If it turns out to be too much, we can merge this and backport it.

@Ma27
Copy link
Member

Ma27 commented Sep 10, 2019

While testing 19.09 I realized that matrix-synapse is also affected by this. So I guess it would make sense to create a PR with a fix for matrix-synapse against release-19.09?

@FRidh
Copy link
Member Author

FRidh commented Sep 10, 2019

@Ma27 first master, then cherry-pick (and push yourself to) 19.09

Ma27 added a commit to Ma27/nixpkgs that referenced this pull request Sep 10, 2019
Currently, `setuptools` isn't propagated automatically to python
packages[1] which causes the following error when starting
`matrix-synapse`:

```
Traceback (most recent call last):
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/bin/.homeserver-wrapped", line 39, in <module>
     import synapse.config.logger
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/config/logger.py", line 27, in <module>
     from synapse.app import _base as appbase
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/app/__init__.py", line 18, in <module>
 E402
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/python_dependencies.py", line 19, in <module>
     from pkg_resources import (
 No module named 'pkg_resources'
```

[1] NixOS#68314
@Ma27 Ma27 mentioned this pull request Sep 10, 2019
10 tasks
worldofpeace pushed a commit that referenced this pull request Sep 10, 2019
Currently, `setuptools` isn't propagated automatically to python
packages[1] which causes the following error when starting
`matrix-synapse`:

```
Traceback (most recent call last):
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/bin/.homeserver-wrapped", line 39, in <module>
     import synapse.config.logger
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/config/logger.py", line 27, in <module>
     from synapse.app import _base as appbase
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/app/__init__.py", line 18, in <module>
 E402
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/python_dependencies.py", line 19, in <module>
     from pkg_resources import (
 No module named 'pkg_resources'
```

[1] #68314

(cherry picked from commit 58dc1e2)
ivan added a commit to ivan/nixpkgs that referenced this pull request Sep 11, 2019
This fixes:

Traceback (most recent call last):
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/bin/.snscrape-wrapped", line 11, in <module>
    sys.exit(main())
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 218, in main
    args = parse_args()
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 154, in parse_args
    import snscrape.version
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/version.py", line 1, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Related: NixOS#68314
ivan added a commit to ivan/nixpkgs that referenced this pull request Sep 11, 2019
This fixes:

Traceback (most recent call last):
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/bin/.snscrape-wrapped", line 11, in <module>
    sys.exit(main())
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 218, in main
    args = parse_args()
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 154, in parse_args
    import snscrape.version
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/version.py", line 1, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Related: NixOS#68314
(cherry picked from commit 7e7c98a)
@ivan ivan mentioned this pull request Sep 11, 2019
10 tasks
peti added a commit that referenced this pull request Sep 11, 2019
…puts

As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
#68314 for further details.

Fixes #68479.
peti added a commit that referenced this pull request Sep 11, 2019
…puts

As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
#68314 for further details.

Fixes #68479.

(cherry picked from commit b57f25a)
peti added a commit that referenced this pull request Sep 11, 2019
As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
#68314 for further details.
@FRidh
Copy link
Member Author

FRidh commented Oct 10, 2019

Yep, many packages have been fixed so let's close it.

@FRidh FRidh closed this Oct 10, 2019
@flokli
Copy link
Contributor

flokli commented Oct 10, 2019

@FRidh I still think something should be posted on discuss.python.org/c/packaging though ;-)

@FRidh
Copy link
Member Author

FRidh commented Oct 10, 2019

flokli added a commit to flokli/nixpkgs that referenced this pull request Oct 10, 2019
worldofpeace pushed a commit that referenced this pull request Oct 10, 2019
It is used during runtime.

cc #68314

(cherry picked from commit 06dc7cb)
jerith666 added a commit to jerith666/nixpkgs that referenced this pull request Oct 15, 2019
As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
NixOS#68314 for further details.
flokli added a commit to flokli/nixpkgs that referenced this pull request Oct 15, 2019
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
NixOS#68314 for further details.

(cherry picked from commit ce3ddb2)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
More NixOS#68314 related breakage.

(cherry picked from commit 01aa4bb)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
As a side-effect of f7e28bf, the build no longer propagated 'setuptools', which
is a run-time dependency. See NixOS#68314 for further details.

(cherry picked from commit 55bf3b4)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
Related: NixOS#68314

This fixes startup of anki, which currently shows this in a dialog:

Error during startup:
Traceback (most recent call last):
  File "/nix/store/0h395dwc6b80n5xg93p86ywaz6kpz6ck-anki-2.1.15/lib/python3.7/site-packages/aqt/main.py", line 46, in __init__
    self.setupAddons()
  File "/nix/store/0h395dwc6b80n5xg93p86ywaz6kpz6ck-anki-2.1.15/lib/python3.7/site-packages/aqt/main.py", line 657, in setupAddons
    import aqt.addons
  File "/nix/store/0h395dwc6b80n5xg93p86ywaz6kpz6ck-anki-2.1.15/lib/python3.7/site-packages/aqt/addons.py", line 9, in <module>
    import markdown
  File "/nix/store/knq8798kl0xzzr7ii4bchskg1c8mq6pj-python3.7-Markdown-3.1.1/lib/python3.7/site-packages/markdown/__init__.py", line 25, in <module>
    from .core import Markdown, markdown, markdownFromFile
  File "/nix/store/knq8798kl0xzzr7ii4bchskg1c8mq6pj-python3.7-Markdown-3.1.1/lib/python3.7/site-packages/markdown/core.py", line 29, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

(cherry picked from commit 2769d61)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
`setuptools` isn't propagated automatically anymore, see also NixOS#68314.

(cherry picked from commit 54752cd)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
NixOS#68314 for further details.

(cherry picked from commit 14854f2)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
…puts

As a side-effect of f7e28bf, the build
no longer propagated 'setuptools', which is a run-time dependency. See
NixOS#68314 for further details.

Fixes NixOS#68479.

(cherry picked from commit b57f25a)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
This fixes:

Traceback (most recent call last):
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/bin/.snscrape-wrapped", line 11, in <module>
    sys.exit(main())
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 218, in main
    args = parse_args()
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/cli.py", line 154, in parse_args
    import snscrape.version
  File "/nix/store/607z14x0spsz1lsh0fg9cbyc9lr038mi-python3.7-snscrape-0.3.0/lib/python3.7/site-packages/snscrape/version.py", line 1, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Related: NixOS#68314
(cherry picked from commit 7e7c98a)
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Oct 23, 2019
Currently, `setuptools` isn't propagated automatically to python
packages[1] which causes the following error when starting
`matrix-synapse`:

```
Traceback (most recent call last):
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/bin/.homeserver-wrapped", line 39, in <module>
     import synapse.config.logger
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/config/logger.py", line 27, in <module>
     from synapse.app import _base as appbase
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/app/__init__.py", line 18, in <module>
 E402
   File "/nix/store/xxkds7821mrahfx75az0sq3ryf69m612-matrix-synapse-1.3.1/lib/python3.7/site-packages/synapse/python_dependencies.py", line 19, in <module>
     from pkg_resources import (
 No module named 'pkg_resources'
```

[1] NixOS#68314

(cherry picked from commit 58dc1e2)
@tbenst tbenst mentioned this pull request Nov 25, 2019
10 tasks
rajivr added a commit to rajivr/tock that referenced this pull request Nov 25, 2019
Due to a change in Nix, `setuptools` is no longer available by default. 

Upstream issue [#68314](NixOS/nixpkgs#68314)

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
idontgetoutmuch pushed a commit to idontgetoutmuch/nixpkgs that referenced this pull request Jan 15, 2020
idontgetoutmuch pushed a commit to idontgetoutmuch/nixpkgs that referenced this pull request Jan 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants