python3Packages.qiskit: 0.19.6 -> 0.20.0 & update components #95713

drewrisinger · 2020-08-17T19:50:31Z

Motivation for this change

Updates python3Packages.qiskit to v0.20.0, as well as updates all components.
Some components require new packages, which were added (muparserx, pythonPackages.yfinance (which requires pythonPackages.multitasking)).

Things done

pkgs/development/libraries/muparserx/default.nix

jonringer

ohterwise LGTM

pkgs/development/python-modules/retworkx/default.nix

pkgs/development/python-modules/yfinance/default.nix

jonringer

also, retworkx in python-modules needs to be changed since it's not a normal buildPythonPackage.

original:

  retworkx = callPackage ../development/python-modules/retworkx { };

correct:

  retworkx = toPythonModule(callPackage ../development/python-modules/retworkx { });

pkgs/development/python-modules/retworkx/default.nix

jonringer · 2020-08-17T20:15:04Z

do you mind looking at #95527 as well?

Uses feedback from NixOS/nixpkgs#95713

drewrisinger · 2020-08-17T20:51:42Z

Addressed all comments. Building locally now. Only outstanding item IMHO is maybe running self-test on muparserx (WIP).

🤦 on changing retworkx = toPythonModule ..., did it in my nur-packages but forgot here... Fixed.

drewrisinger · 2020-08-18T02:32:08Z

OK. Worked out last few bugs.

pythonPackages.retworkx now properly works as python37Packages.retworkx, and is disabled for python27Packages.
muparserx is self-tested
all review comments addressed.
nix-review pr 95713 runs locally

jonringer

seeing some issues with aqua:

https://github.com/NixOS/nixpkgs/pull/95713
6 packages failed to build:
python37Packages.qasm2image python37Packages.qiskit python37Packages.qiskit-aqua python38Packages.qasm2image python38Packages.qiskit python38Packages.qiskit-aqua

20 packages built:
muparserx python27Packages.multitasking python27Packages.pylatexenc python27Packages.yfinance python37Packages.multitasking python37Packages.pylatexenc python37Packages.qiskit-aer python37Packages.qiskit-ibmq-provider python37Packages.qiskit-ignis python37Packages.qiskit-terra python37Packages.retworkx python37Packages.yfinance python38Packages.multitasking python38Packages.pylatexenc python38Packages.qiskit-aer python38Packages.qiskit-ibmq-provider python38Packages.qiskit-ignis python38Packages.qiskit-terra python38Packages.retworkx python38Packages.yfinance

the failure:

=================================== FAILURES ===================================
__________________________ TestOptimizers.test_p_bfgs __________________________

self = <test.aqua.test_optimizers.TestOptimizers testMethod=test_p_bfgs>

    def test_p_bfgs(self):
        """ parallel l_bfgs_b test """
        optimizer = P_BFGS(maxfun=1000)
        res = self._optimize(optimizer)
>       self.assertLessEqual(res[2], 10000)
E       AssertionError: 44250 not less than or equal to 10000

test/aqua/test_optimizers.py:69: AssertionError

drewrisinger · 2020-08-24T20:38:22Z

the failure:

=================================== FAILURES ===================================
__________________________ TestOptimizers.test_p_bfgs __________________________

self = <test.aqua.test_optimizers.TestOptimizers testMethod=test_p_bfgs>

    def test_p_bfgs(self):
        """ parallel l_bfgs_b test """
        optimizer = P_BFGS(maxfun=1000)
        res = self._optimize(optimizer)
>       self.assertLessEqual(res[2], 10000)
E       AssertionError: 44250 not less than or equal to 10000

test/aqua/test_optimizers.py:69: AssertionError

@jonringer this is probably another massively-parallel CPU issue... the test runs an optimizer that distributes across the cores (https://github.com/Qiskit/qiskit-aqua/blob/72a6fb3b40accfe18b77bfb8c811aa94a7afa443/qiskit/aqua/components/optimizers/p_bfgs.py#L90), which each run an optimization and the number of function evalutions are summed across the processes and assumed to be < 10000 in the test.

I'll open an upstream issue, but in the meantime I feel comfortable disabling this test to get this PR to pass since I've not had this issue locally or on the GitHub Actions build of my NUR repo.

EDIT: upstream issue. qiskit-community/qiskit-aqua#1214

Also compile from source using Rust. Now possible b/c pyo3/maturin work on Rust stable.

Dependency for python3Packages.yfinance

jonringer · 2020-08-24T20:57:06Z

Steps to reproduce the problem

    Get a Threadripper 3990X (or equivalent many-core CPU), running Linux

lol, yes, $4k cpu :)

drewrisinger · 2020-08-24T20:59:07Z

Steps to reproduce the problem

    Get a Threadripper 3990X (or equivalent many-core CPU), running Linux

lol, yes, $4k cpu :)

That is the CPU you tested it against, right?
And this package is distributed by IBM, I'm sure they have a few high-core-count systems like Summit that they could test this on if they really wanted :D

jonringer · 2020-08-24T21:04:04Z

That is the CPU you tested it against, right?

yes

I'm sure they have a few high-core-count systems like Summit that they could test this on if they really wanted :D

right, or AMD's rome 7742 also has 64/128 https://www.newegg.com/amd-epyc-7742-socket-sp3/p/N82E16819113581

drewrisinger · 2020-08-25T13:10:00Z

@jonringer pushed the patch upstream made. Hoping python3Packages.qiskit-aqua will build cleanly for you now.

jonringer

diff LGTM

https://github.com/NixOS/nixpkgs/pull/95713
26 packages built:
muparserx python27Packages.multitasking python27Packages.pylatexenc python27Packages.yfinance python37Packages.multitasking python37Packages.pylatexenc python37Packages.qasm2image python37Packages.qiskit python37Packages.qiskit-aer python37Packages.qiskit-aqua python37Packages.qiskit-ibmq-provider python37Packages.qiskit-ignis python37Packages.qiskit-terra python37Packages.retworkx python37Packages.yfinance python38Packages.multitasking python38Packages.pylatexenc python38Packages.qasm2image python38Packages.qiskit python38Packages.qiskit-aer python38Packages.qiskit-aqua python38Packages.qiskit-ibmq-provider python38Packages.qiskit-ignis python38Packages.qiskit-terra python38Packages.retworkx python38Packages.yfinance

cc @FRidh does the rust -> python retworkx changes look good to you?

jonringer · 2020-08-25T20:39:51Z

@jonringer pushed the patch upstream made. Hoping python3Packages.qiskit-aqua will build cleanly for you now.

I think i mentioned it somewhere else, but i had to drop the NIX_BUILD_CORES to 32 due to other packages having timing issues (bashCompeletion would fail in tests, some meson builds would try to link against objects that don't exist, etc)

drewrisinger · 2020-08-25T22:38:20Z

cc @FRidh does the rust -> python retworkx changes look good to you?

I based it on the @FRidh approved #91342 python3Packages.tokenizers.

drewrisinger requested review from FRidh and jonringer as code owners August 17, 2020 19:50

jonringer reviewed Aug 17, 2020

View reviewed changes

pkgs/development/libraries/muparserx/default.nix Outdated Show resolved Hide resolved

ofborg bot added the 6.topic: python label Aug 17, 2020

jonringer reviewed Aug 17, 2020

View reviewed changes

pkgs/development/python-modules/retworkx/default.nix Outdated Show resolved Hide resolved

pkgs/development/python-modules/retworkx/default.nix Outdated Show resolved Hide resolved

pkgs/development/python-modules/yfinance/default.nix Outdated Show resolved Hide resolved

jonringer reviewed Aug 17, 2020

View reviewed changes

pkgs/development/python-modules/retworkx/default.nix Outdated Show resolved Hide resolved

ofborg bot added the 8.has: package (new) label Aug 17, 2020

ofborg bot requested a review from pandaman64 August 17, 2020 20:19

ofborg bot added 11.by: package-maintainer 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Aug 17, 2020

drewrisinger added a commit to drewrisinger/nur-packages that referenced this pull request Aug 17, 2020

python3Packages.retworkx: fix mistakes

55d4a2d

Uses feedback from NixOS/nixpkgs#95713

drewrisinger mentioned this pull request Aug 17, 2020

python3Packages.qiskit-aer fails if you have too many cores #95527

Closed

drewrisinger force-pushed the dr-pr-python-qiskit-bump branch from f7c5df7 to 1c5d37c Compare August 17, 2020 20:44

drewrisinger force-pushed the dr-pr-python-qiskit-bump branch 4 times, most recently from 73610b2 to ed4dbb4 Compare August 17, 2020 22:17

drewrisinger mentioned this pull request Aug 18, 2020

Qiskit cleanup drewrisinger/nur-packages#17

Merged

jonringer reviewed Aug 21, 2020

View reviewed changes

drewrisinger mentioned this pull request Aug 24, 2020

test_p_bfgs fails on >60 core CPU qiskit-community/qiskit-aqua#1214

Closed

drewrisinger added 5 commits August 24, 2020 16:53

python3Packages.pylatexenc: 2.4 -> 2.7

30f819a

muparserx: init at 4.0.8

10c4fd2

python3Packages.retworkx: 0.3.4 -> 0.4.0

c0f2898

Also compile from source using Rust. Now possible b/c pyo3/maturin work on Rust stable.

python3Packages.qiskit-terra: 0.14.2 -> 0.15.1

4b01d21

python3Packages.multitasking: init at 0.0.9

4e6414d

Dependency for python3Packages.yfinance

drewrisinger added 4 commits August 24, 2020 16:53

python3Packages.yfinance: init at 0.1.54

83c1996

python3Packages.qiskit-aer: 0.5.2 -> 0.6.1

615ecd9

python3Packages.qiskit-ibmq-provider: 0.7.2 -> 0.8.0

739fc0e

python3Packages.qiskit-ignis: 0.3.3 -> 0.4.0

83d7ae8

drewrisinger force-pushed the dr-pr-python-qiskit-bump branch from ed4dbb4 to fd1ca04 Compare August 24, 2020 20:53

drewrisinger added 2 commits August 25, 2020 09:08

python3Packages.qiskit-aqua: 0.7.3 -> 0.7.5

cd48f59

python3Packages.qiskit: 0.19.6 -> 0.20.0

b0c0c50

drewrisinger force-pushed the dr-pr-python-qiskit-bump branch from fd1ca04 to b0c0c50 Compare August 25, 2020 13:08

jonringer approved these changes Aug 25, 2020

View reviewed changes

jonringer merged commit 4dee731 into NixOS:master Aug 25, 2020

drewrisinger deleted the dr-pr-python-qiskit-bump branch August 26, 2020 12:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python3Packages.qiskit: 0.19.6 -> 0.20.0 & update components #95713

python3Packages.qiskit: 0.19.6 -> 0.20.0 & update components #95713

drewrisinger commented Aug 17, 2020

jonringer left a comment

jonringer left a comment •

edited

jonringer commented Aug 17, 2020

drewrisinger commented Aug 17, 2020

drewrisinger commented Aug 18, 2020

jonringer left a comment

drewrisinger commented Aug 24, 2020 •

edited

jonringer commented Aug 24, 2020

drewrisinger commented Aug 24, 2020

jonringer commented Aug 24, 2020 •

edited

drewrisinger commented Aug 25, 2020

jonringer left a comment

jonringer commented Aug 25, 2020

drewrisinger commented Aug 25, 2020

python3Packages.qiskit: 0.19.6 -> 0.20.0 & update components #95713

python3Packages.qiskit: 0.19.6 -> 0.20.0 & update components #95713

Conversation

drewrisinger commented Aug 17, 2020

Motivation for this change

Things done

jonringer left a comment

Choose a reason for hiding this comment

jonringer left a comment • edited

Choose a reason for hiding this comment

jonringer commented Aug 17, 2020

drewrisinger commented Aug 17, 2020

drewrisinger commented Aug 18, 2020

jonringer left a comment

Choose a reason for hiding this comment

drewrisinger commented Aug 24, 2020 • edited

jonringer commented Aug 24, 2020

drewrisinger commented Aug 24, 2020

jonringer commented Aug 24, 2020 • edited

drewrisinger commented Aug 25, 2020

jonringer left a comment

Choose a reason for hiding this comment

jonringer commented Aug 25, 2020

drewrisinger commented Aug 25, 2020

jonringer left a comment •

edited

drewrisinger commented Aug 24, 2020 •

edited

jonringer commented Aug 24, 2020 •

edited