oslo-config: fix build #29602

makefu · 2017-09-20T14:45:03Z

Also updated the following dependencies:
keystoneauth1: 3.1.0 -> 3.2.0
needed to disable test due to circular dependencies towards oslo-config
oslo-i18n: 2.7.0 -> 3.18.0
update required to not have oslo-config as a build dependency
oslotest: 1.12.0 -> 2.18.0
os-client-config: 1.8.1 -> 1.28.0
needed to disable testing due to circular dependency with oslotest
mox3: 0.11.0 -> 0.23.0
for py36: needed to disable testing due to LOCALE flag ValueError
debtcollector: 0.9.0 -> 1.17.0

update:
oslo-service: 0.10.0 -> 1.26.0
needs to disable tests due to network errors when importing eventlet
for tests ( socket.getprotobyname('tcp') -> no such protocol )
eventlet: 0.17.4 -> 0.20.0
cannot update to 0.21.0 due to version pinning ( < 0.21.0 ) of oslo-service
monotonic: 0.4 -> 1.3
oslo-serialization: 1.10.0 -> 2.20.0
oslo-utils: 2.6.0 -> 3.29.0
oslo-concurrency: 2.7.0 -> 3.22.0
oslo-log: 1.12.1 -> 3.31.0
oslo-context: 0.7.0 -> 2.18.1
routes: 1.12.3 -> 2.4.1
webob: 1.4.1 -> 1.7.3

extra packages:
requestsexceptions: init at 1.3.0
removed hacking reference in test-dependencies.txt as it is not a test dependency

Motivation for this change

oslo-config broke (and never got fixed) with the update introduced in d58e1f1 . It got marked as broken 67651d8 and is now in the process of being deleted via the PR to remove all things openstack #29491 .
oslo-config however is a dependency for a number of different other packages. My main reason to have working tests it in nixpkgs is for sqlalchemy_migrate ( -> tempest-lib -> oslo-config ).

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

nlewo · 2017-09-20T16:47:31Z

pkgs/top-level/python-packages.nix

@@ -18722,6 +18725,31 @@ in {
      maintainer = maintainers.fridh;
    };
  };
+  requestsexceptions = buildPythonPackage rec {


Can you move this package to pkgs/development/python-modules ?

an specific reason not to keep it in python-packages ? it is in fact only a build dependency for os-client-config.

We want to move all packages out of this file to make maintenance easier. But it is a lot of labour work so it will take some time.

okay, i will rewrite the commit and put requestsexceptions into a separate file

nlewo · 2017-09-20T16:48:44Z

pkgs/top-level/python-packages.nix

+
+    propagatedBuildInputs = with self; [ pbr ];
+
+    # upstream hacking package is not required for testing


It seems no tests are executed: Ran 0 tests in 0.000s

The package does not provide any tests (yet). However the test will evaluate the python files for syntax and import errors

@nlewo as described in https://pypi.python.org/pypi/hacking/ the project is only for checking the openstack styleguide. These tests however are not of functional nature and therefore can be omitted. I think we should not start hot-fixing style-issues in packages for the sake of pleasing the pedantic style checker.

:)
It's fine. It was just an observation.

makefu · 2017-09-21T22:02:31Z

@nlewo @Mic92 ok i've rebased and now the new package resides in python-modules.

FRidh · 2017-09-23T06:43:19Z

Good that you are having a look at this. I strongly recommend you add yourself as maintainer to packages that are important to you, be it directly or indirectly.

FRidh · 2017-09-23T07:00:40Z

python.pkgs3.oslo-service fails. Packages that can now be build again, but fail, should either be fixed or marked as broken.

makefu · 2017-09-23T11:42:43Z

@FRidh is there a command which builds all the packages dependent on the packages i touched? i tried nox-review pr but it always crashes with

subprocess.CalledProcessError: Command '['git', 'checkout', '--quiet', '35f205a4b624bceca7c53b9c19ddac5f37a5ae4c']' returned non-zero exit status 1

I will add myself to the list of maintainers for the tools i need in nixpkgs 👍

makefu · 2017-09-23T13:13:26Z

ok, I've finished my updating frenzy (and soldered some cables while waiting for the build).

Is there a good way to test if all dependent pachages still build?

FRidh · 2017-09-23T13:26:13Z

@makefu sometimes you need to clear ~/.nox when that happens. I use my own script, niff, to determine what packages were changed.

You could run e.g. niff pr 29602 in which case it compares your branch to the branch the PR is made against. Therefore, don't forget to rebase.

To also build the changes

nix-shell -p niff # its available in unstable and 17.09
nix-build -k $(niff pr 29602 --output=attrs)

also updated the following dependencies: keystoneauth1: 3.1.0 -> 3.2.0 disabled tests which require oslo-config, oslo-test or requests-kerberos oslo-i18n: 2.7.0 -> 3.18.0 oslotest: 1.12.0 -> 2.18.0 os-client-config: 1.8.1 -> 1.28.0 needed to disable testing due to circular dependency with oslotest mox3: 0.11.0 -> 0.23.0 disable tests for py36 due to upstream bug debtcollector: 0.9.0 -> 1.17.0 tests enabled extra packages: requestsexceptions: init at 1.3.0

requires a later version of betamax, bumped to 0.8.0

oslo-service: needs to disable tests due to network errors when importing eventlet for tests ( socket.getprotobyname('tcp') -> no such protocol ) eventlet: 0.17.4 -> 0.20.0 cannot update to 0.21.0 due to version pinning ( < 0.21.0 ) of oslo-service monotonic: 0.4 -> 1.3 oslo-serialization: 1.10.0 -> 2.20.0 oslo-utils: 2.6.0 -> 3.29.0 oslo-concurrency: 2.7.0 -> 3.22.0 oslo-log: 1.12.1 -> 3.31.0 oslo-context: 0.7.0 -> 2.18.1 routes: 1.12.3 -> 2.4.1 webob: 1.4.1 -> 1.7.3 when updating i rewrote the package to use fetchPypi for making future updating easier

eventlet cannot be imported in any tests because it fails to be imported inside the sandboxed environment

nlewo · 2017-09-26T14:21:41Z

@Mic92 would it be possible to backport the oslo fix (commit 7251699) to 17.09 ?

Mic92 · 2017-09-26T15:42:33Z

@makefu if I backport oslo, what else I need to backport?

FRidh · 2017-09-26T16:27:21Z

17.09 currently: https://hydra.nixos.org/eval/1396332
17.09 with this merge commit: https://hydra.nixos.org/eval/1396335

Mic92 · 2017-09-26T16:36:29Z

So this is pyramid and something else?

FRidh · 2017-09-26T16:40:34Z

pkgs/top-level/python-packages.nix

@@ -7333,6 +7350,9 @@ in {
      webtest
      zope_component
      zope_interface
+      plaster
+      plaster-pastedeploy
+      hupper


why are these added to buildInputs? Seems to be needed at runtime (install_requires).

You are right, they should be included in the propagated buildinputs. ... should i prepare a PR or can this be done with a direct commit?

Adding hupper and plaster should fix all packages the update broke

copumpkin · 2017-11-20T13:26:39Z

See for example https://hydra.nixos.org/job/nixpkgs/trunk/python27Packages.hupper.x86_64-darwin/all?page=1, although it's unclear why it's failing. It just doesn't seem to notice changes... 😦

copumpkin · 2017-11-20T14:11:32Z

I fixed hupper in 73c30b6 so I think we're all set now.

makefu · 2017-11-20T15:01:10Z

@copumpkin thanks for the investigation and fix, i really appreciate it as i do not have any darwin builder to test 👍

copumpkin · 2017-11-20T15:27:22Z

No problem, thanks for the initial update 😄

makefu requested a review from FRidh as a code owner September 20, 2017 14:45

nlewo reviewed Sep 20, 2017

View reviewed changes

makefu force-pushed the pkgs/oslo-config/fix branch from 5ae8c52 to 9b23b75 Compare September 21, 2017 21:59

FRidh added the 6.topic: python label Sep 23, 2017

makefu force-pushed the pkgs/oslo-config/fix branch 2 times, most recently from 0ed997f to 242c313 Compare September 25, 2017 20:04

makefu and others added 9 commits September 25, 2017 23:23

keystoneauth1: enable tests

97644a9

requires a later version of betamax, bumped to 0.8.0

sqlalchemy-migrate: re-enable tests

42edc6f

oslo-config,sqlalchemy: add makefu to list of maintainers

d7c26f2

celery: disable eventlet tests

d00ba80

eventlet cannot be imported in any tests because it fails to be imported inside the sandboxed environment

pyramid: 1.7 -> 1.9.1

dcc36a4

celery: fix eventlet tests

1b628fb

gdal: fix path to pg_config

032c50d

makefu force-pushed the pkgs/oslo-config/fix branch from 488a5c8 to 032c50d Compare September 25, 2017 21:24

Mic92 merged commit 40a58db into NixOS:master Sep 26, 2017

FRidh reviewed Sep 26, 2017

View reviewed changes

makefu mentioned this pull request Sep 27, 2017

pyramid: fix propagated builds #29844

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

oslo-config: fix build #29602

oslo-config: fix build #29602

makefu commented Sep 20, 2017 •

edited

nlewo Sep 20, 2017

makefu Sep 21, 2017

Mic92 Sep 21, 2017

makefu Sep 21, 2017

nlewo Sep 20, 2017

makefu Sep 20, 2017

makefu Sep 21, 2017

nlewo Sep 21, 2017

makefu commented Sep 21, 2017

FRidh commented Sep 23, 2017

FRidh commented Sep 23, 2017

makefu commented Sep 23, 2017

makefu commented Sep 23, 2017

FRidh commented Sep 23, 2017

nlewo commented Sep 26, 2017

Mic92 commented Sep 26, 2017

FRidh commented Sep 26, 2017

Mic92 commented Sep 26, 2017

FRidh Sep 26, 2017

makefu Sep 26, 2017 •

edited

copumpkin commented Nov 20, 2017

copumpkin commented Nov 20, 2017

makefu commented Nov 20, 2017

copumpkin commented Nov 20, 2017


		propagatedBuildInputs = with self; [ pbr ];

		# upstream hacking package is not required for testing

oslo-config: fix build #29602

oslo-config: fix build #29602

Conversation

makefu commented Sep 20, 2017 • edited

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

makefu commented Sep 21, 2017

FRidh commented Sep 23, 2017

FRidh commented Sep 23, 2017

makefu commented Sep 23, 2017

makefu commented Sep 23, 2017

FRidh commented Sep 23, 2017

nlewo commented Sep 26, 2017

Mic92 commented Sep 26, 2017

FRidh commented Sep 26, 2017

Mic92 commented Sep 26, 2017

Choose a reason for hiding this comment

makefu Sep 26, 2017 • edited

Choose a reason for hiding this comment

copumpkin commented Nov 20, 2017

copumpkin commented Nov 20, 2017

makefu commented Nov 20, 2017

copumpkin commented Nov 20, 2017

makefu commented Sep 20, 2017 •

edited

makefu Sep 26, 2017 •

edited