Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: package set updates before branch-off #55757

Merged
merged 440 commits into from Feb 21, 2019
Merged

Python: package set updates before branch-off #55757

merged 440 commits into from Feb 21, 2019

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Feb 14, 2019

Motivation for this change

Update all packages before branching off.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member Author

FRidh commented Feb 14, 2019

@dotlambda flake8 update is included here

@dotlambda
Copy link
Member

dotlambda commented Feb 14, 2019

@FRidh The flake8 bump required bumping pyflakes, pycodestyle, entrypoints and configparser as well. I added some changed other than just bumping these packages so I replaced your commits by mine.

@dotlambda
Copy link
Member

I'm getting

$ nix-shell -I nixpkgs=. -p "python3.withPackages (ps: with ps; [ pykerberos ])" --run "python -c 'import kerberos'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /nix/store/77n1k65iv1ihk8j3jbz78rzl477p3qlr-python3-3.7.2-env/lib/python3.7/site-packages/kerberos.cpython-37m-x86_64-linux-gnu.so: undefined symbol: gss_nt_service_name

Any idea what's happening?
cc @catern

@catern
Copy link
Contributor

catern commented Feb 23, 2019

I believe that's caused by pykerberos having krb5 listed as a buildInput, when it should be a nativeBuildInput. I guess something was changed about buildInput handling in Python-land in this PR, since that worked before.

Fixed here: #56268

@timokau
Copy link
Member

timokau commented Mar 2, 2019

Is there any kind of policy on how to handle the fallout of breaking changes? I just spent quite some time bisecting the rpy2 failure to the strictDeps commit. I realize that it may not be realistic to expect the committer to fix all downstream issues, but I think doing nothing and hoping for maintainers / end-users to figure out the root cause of failures isn't optimal.

Maybe we could at least notify maintainers / post about changes with big breaking potential on nixos-weekly / post on discourse.

timokau added a commit to timokau/nixpkgs that referenced this pull request Mar 3, 2019
timokau added a commit to timokau/nixpkgs that referenced this pull request Mar 3, 2019
@timokau timokau mentioned this pull request Mar 3, 2019
10 tasks
timokau added a commit to timokau/nixpkgs that referenced this pull request Mar 4, 2019
More fallout from enabling strictDeps in NixOS#55757. `buildInputs` are not
accessible from the check phase.
@timokau timokau mentioned this pull request Mar 4, 2019
10 tasks
pull bot pushed a commit to evanjs/nixpkgs that referenced this pull request Mar 4, 2019
More fallout from enabling strictDeps in NixOS#55757. `buildInputs` are not
accessible from the check phase.
@catern
Copy link
Contributor

catern commented Mar 4, 2019

Uh, hmm, actually that pykerberos fix seems really weird. #56268 pykerberos links against krb5, so shouldn't krb5 be in buildInputs? But when krb5 is in buildInputs, the built binary doesn't properly link against krb5. But it does work if krb5 is in nativeBuildInputs. Huh?

@dotlambda
Copy link
Member

dotlambda commented Mar 4, 2019

Maybe the following would make more sense.

diff --git a/pkgs/development/python-modules/pykerberos/default.nix b/pkgs/development/python-modules/pykerberos/default.nix
index 994ba059f9e..ca1fda884d5 100644
--- a/pkgs/development/python-modules/pykerberos/default.nix
+++ b/pkgs/development/python-modules/pykerberos/default.nix
@@ -9,7 +9,9 @@ buildPythonPackage rec {
     sha256 = "0v47p840myqgc7hr4lir72xshcfpa0w8j9n077h3njpqyn6wlbag";
   };
 
-  nativeBuildInputs = [ krb5 ];
+  nativeBuildInputs = [ krb5 ]; # for krb5-config
+
+  buildInputs = [ krb5 ];
 
   # there are no tests
   doCheck = false;

I have no idea though why it works without buildInputs nor do I get what their code at https://github.com/02strich/pykerberos/blob/v1.2.1/setup.py#L38 is supposed to do. The except certainly doesn't trigger when krb5-config is not present. Maybe we should report that upstream.

EDIT: Removed a # that shouldn't have been there.

@FRidh
Copy link
Member Author

FRidh commented Mar 5, 2019

@dotlambda that's correct, it needs to be in both. strictDeps only handles the things on PATH, but it doesn't consider distinguishing whether the libraries need to be available for the host machine or not.

@dotlambda
Copy link
Member

that's correct, it needs to be in both. strictDeps only handles the things on PATH, but it doesn't consider distinguishing whether the libraries need to be available for the host machine or not.

I know, but nix-shell -I nixpkgs=. -p "python3.withPackages (ps: with ps; [ pykerberos ])" --run "python -c 'import kerberos'" works without having it in buildInputs. That's what I don't get. Maybe the library is only used at a later point.

@FRidh
Copy link
Member Author

FRidh commented Mar 5, 2019

When not cross-compiling, there is no difference between buildInputs and nativeBuildInputs.

@dotlambda
Copy link
Member

I pushed the fix in 82b3892.

pull bot pushed a commit to evanjs/nixpkgs that referenced this pull request Mar 5, 2019
@flokli
Copy link
Contributor

flokli commented Mar 6, 2019

The python: azure-mgmt-compute: 0.20.1 -> 4.4.0 bump also broke nixops.

@flokli
Copy link
Contributor

flokli commented Mar 6, 2019

backported the nixops fix (#56775) to 19.03 in 17a5b64.

@bendlas
Copy link
Contributor

bendlas commented Mar 7, 2019

The python: azure-mgmt-compute: 0.20.1 -> 4.4.0 bump also broke nixops.

azure-mgmt-compute doesn't build anymore at all, but there's a PR for that.

@FRidh? If you don't want to rebase #52550, you can always just pin the older version.

samueldr pushed a commit that referenced this pull request Mar 10, 2019
More fallout from enabling strictDeps in #55757. `buildInputs` are not
accessible from the check phase.

(cherry picked from commit 83dcf66)
timokau added a commit to timokau/nixpkgs that referenced this pull request Mar 10, 2019
See NixOS#54182 and
NixOS#55757.

(cherry picked from commit 9e37547)
timokau added a commit to timokau/nixpkgs that referenced this pull request Mar 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet