Ansible py3 #49764

sengaya · 2018-11-04T22:05:01Z

Motivation for this change

Switch to python3 for ansible and related packages
Remove ansible 2.4 as it does not support python 3 and is already out of support

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

This PR should make #44413 obsolete.
closes #44413

I guess I need to provide an update to the release notes for 19.03, but I would first like to get some feedback it this change has the chance to get merged.

cc @jgeerds @joamaki @c0bw3b

c0bw3b · 2018-11-04T22:42:25Z

+1 for the removal of ansible_2_4 since it has unpatched CVEs (probably).

@GrahamcOfBorg build ansible_2_7 ansible-lint

GrahamcOfBorg · 2018-11-04T22:45:20Z

Success on aarch64-linux (full log)

Attempted: ansible_2_7, ansible-lint

Partial log (click to expand)

wrapping `/nix/store/2shcbr5f4qjpkgblzngihk8j45s76kgv-python3.6-ansible-lint-3.4.23/bin/ansible-lint'...
running install tests
..........................................WARNING: Couldn't open /build/source/test/doesnotexist.yml - No such file or directory
............................................
----------------------------------------------------------------------
Ran 86 tests in 34.807s

OK
/nix/store/dp81rc3xigwfvq4g4i1ai26zx4cb27bx-python3.6-ansible-2.7.1
/nix/store/2shcbr5f4qjpkgblzngihk8j45s76kgv-python3.6-ansible-lint-3.4.23

GrahamcOfBorg · 2018-11-04T22:46:51Z

Success on x86_64-linux (full log)

Attempted: ansible_2_7, ansible-lint

Partial log (click to expand)

wrapping `/nix/store/3ls3v2ih5wqn7y72b2w92bafadcchqvy-python3.6-ansible-lint-3.4.23/bin/ansible-lint'...
running install tests
..........................................WARNING: Couldn't open /build/source/test/doesnotexist.yml - No such file or directory
............................................
----------------------------------------------------------------------
Ran 86 tests in 63.424s

OK
/nix/store/baynj3kdq6liw471115ajf62hxg4fmbr-python3.6-ansible-2.7.1
/nix/store/3ls3v2ih5wqn7y72b2w92bafadcchqvy-python3.6-ansible-lint-3.4.23

c0bw3b · 2018-11-04T22:58:06Z

At least it builds successfully with Python 3.6

FRidh · 2018-11-05T08:32:52Z

Note on staging-next we've switched to python3 = python37.

dotlambda · 2018-11-05T08:57:07Z

pkgs/development/python-modules/pytest-ansible/default.nix

@@ -4,13 +4,12 @@
 , ansible
 , pytest
 , mock
-, isPy3k
+, python3Packages


Please never do this for packages called from python-packages.nix!
Where is this package even used?

dotlambda · 2018-11-05T09:01:12Z

pkgs/development/tools/ansible-lint/default.nix

@@ -1,6 +1,6 @@
-{ stdenv, fetchFromGitHub, pythonPackages, ansible }:
+{ stdenv, fetchFromGitHub, python3Packages, ansible }:


You might want to replace python3Packages by python3 ...

dotlambda · 2018-11-05T09:02:04Z

pkgs/development/tools/ansible-lint/default.nix

@@ -11,9 +11,9 @@ pythonPackages.buildPythonPackage rec {
    sha256 = "0cnfgxh5m7alzm811hc95jigbca5vc1pf8fjazmsakmhdjyfbpb7";
  };

-  propagatedBuildInputs = with pythonPackages; [ pyyaml six ] ++ [ ansible ];
+  propagatedBuildInputs = with python3Packages; [ pyyaml six ] ++ [ ansible ];


... and ansible by ansible.override { python3 = python3; }.
That way, both are using the same set of python packages.

Mic92 · 2018-11-05T16:43:52Z

cc @Lassulus

dasJ · 2018-11-05T19:22:41Z

@Mic92 Shouldn't this go into the release notes? ansible_2_4 was dropped and the change to Python 3 introduced some potential incompatibilites (I see them myself)

Mic92 · 2018-11-06T01:55:30Z

I guess mentioning the python3 switch is worth a changelog line.
Dropping ansible 2.4 is something to be expected since we regularly remove unsupported versions (those that are marked end of life by upstream).

costrouc · 2018-11-07T13:05:21Z

Why are ansible and ansible-list not moved into python-modules and exposed as applications in all-packages for compatibility? Two packages pytest-ansible and ansible-lint already depend on ansible as a library. I am working on two packages ansible-kernel and ansible-runner that both have python 27 and 3.5+ support and depend on ansible as a module. Currently I run into an error because of python 2 -> 3 issues.

dotlambda · 2018-11-07T13:11:57Z

Why are ansible and ansible-list not moved into python-modules and exposed as applications in all-packages for compatibility?

Sounds like a sane thing to do.

costrouc · 2018-11-07T14:02:12Z

Added a note here that one of my PRs depend on this one moving to python-modules. There is a jupyter kernel for ansible that has support for python 2.7 and 3+. @sengaya let me know if you need any help!

dasJ · 2018-12-03T21:28:46Z

So what about this? I'd really love to have these features

c0bw3b · 2018-12-03T21:52:00Z

ansible and ansible-lint are applications, first and foremost.
They should stay visible in all-packages.nix

dotlambda · 2018-12-19T16:39:26Z

ansible and ansible-lint are applications, first and foremost.
They should stay visible in all-packages.nix

That's absolutely possible by adding e.g.

anisble = with python3.pkgs; toPythonApplication ansible;

to all-packages.nix.

dasJ · 2019-02-17T21:05:32Z

Bumping this up because I really need it (currently pinning ansible manually to use python 3).

sengaya added 2 commits November 4, 2018 22:11

ansible_2_4: remove

905e4c0

ansible: switch to python3

a802375

sengaya requested a review from FRidh as a code owner November 4, 2018 22:05

GrahamcOfBorg added 6.topic: python 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Nov 4, 2018

dotlambda requested changes Nov 5, 2018

View reviewed changes

Mic92 mentioned this pull request Nov 5, 2018

ansible-lint: explicitly use python2Packages #44413

Closed

9 tasks

costrouc mentioned this pull request Nov 7, 2018

Integration of kernels and extensions with jupyter #49807

Closed

12 tasks

costrouc mentioned this pull request Feb 24, 2019

Move ansible to python-modules and use toPythonApplication + ansible related packages #56323

Merged

10 tasks

dotlambda closed this in #56323 Mar 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ansible py3 #49764

Ansible py3 #49764

sengaya commented Nov 4, 2018 •

edited by dotlambda

c0bw3b commented Nov 4, 2018

GrahamcOfBorg commented Nov 4, 2018

GrahamcOfBorg commented Nov 4, 2018

c0bw3b commented Nov 4, 2018

FRidh commented Nov 5, 2018

dotlambda Nov 5, 2018

dotlambda Nov 5, 2018

dotlambda Nov 5, 2018

Mic92 commented Nov 5, 2018

dasJ commented Nov 5, 2018

Mic92 commented Nov 6, 2018 •

edited

costrouc commented Nov 7, 2018 •

edited

dotlambda commented Nov 7, 2018

costrouc commented Nov 7, 2018 •

edited

dasJ commented Dec 3, 2018

c0bw3b commented Dec 3, 2018 •

edited

dotlambda commented Dec 19, 2018

dasJ commented Feb 17, 2019

		@@ -1,6 +1,6 @@
		{ stdenv, fetchFromGitHub, pythonPackages, ansible }:
		{ stdenv, fetchFromGitHub, python3Packages, ansible }:

Ansible py3 #49764

Ansible py3 #49764

Conversation

sengaya commented Nov 4, 2018 • edited by dotlambda

Motivation for this change

Things done

c0bw3b commented Nov 4, 2018

GrahamcOfBorg commented Nov 4, 2018

GrahamcOfBorg commented Nov 4, 2018

c0bw3b commented Nov 4, 2018

FRidh commented Nov 5, 2018

dotlambda Nov 5, 2018

Choose a reason for hiding this comment

dotlambda Nov 5, 2018

Choose a reason for hiding this comment

dotlambda Nov 5, 2018

Choose a reason for hiding this comment

Mic92 commented Nov 5, 2018

dasJ commented Nov 5, 2018

Mic92 commented Nov 6, 2018 • edited

costrouc commented Nov 7, 2018 • edited

dotlambda commented Nov 7, 2018

costrouc commented Nov 7, 2018 • edited

dasJ commented Dec 3, 2018

c0bw3b commented Dec 3, 2018 • edited

dotlambda commented Dec 19, 2018

dasJ commented Feb 17, 2019

sengaya commented Nov 4, 2018 •

edited by dotlambda

Mic92 commented Nov 6, 2018 •

edited

costrouc commented Nov 7, 2018 •

edited

costrouc commented Nov 7, 2018 •

edited

c0bw3b commented Dec 3, 2018 •

edited