Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3Packages.spacy: 2.3.4 -> 2.3.5 #106731

Merged
merged 3 commits into from Dec 16, 2020
Merged

python3Packages.spacy: 2.3.4 -> 2.3.5 #106731

merged 3 commits into from Dec 16, 2020

Conversation

danieldk
Copy link
Contributor

Motivation for this change

python3Packagese.spacy: 2.3.4 -> 2.3.5

Changelog:

https://github.com/explosion/spacy/releases/tag/v2.3.5

Also reformat with nixpkgs-fmt and make passthru.tests an attrset.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested a review from aptlin December 12, 2020 10:49
drewrisinger
drewrisinger approved these changes Dec 12, 2020
View reviewed changes
Copy link
Contributor

@drewrisinger drewrisinger left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Diff mostly LGTM. see comments
  • Commits should probably be squashed
  • Builds via nix-review on x86-64:

Misspelled packages in the PR title.

https://github.com/NixOS/nixpkgs/pull/106731
6 packages built:
python37Packages.spacy python37Packages.textacy python38Packages.spacy python38Packages.textacy python39Packages.spacy python39Packages.textacy

pkgs/development/python-modules/spacy/default.nix
Comment on lines +59 to +63
--replace "blis>=0.4.0,<0.8.0" "blis>=0.4.0,<1.0" \
--replace "catalogue>=0.0.7,<1.1.0" "catalogue>=0.0.7,<3.0" \
--replace "plac>=0.9.6,<1.2.0" "plac>=0.9.6,<2.0" \
--replace "srsly>=1.0.2,<1.1.0" "srsly>=1.0.2,<3.0" \
--replace "thinc==7.4.1" "thinc>=7.4.1,<8"
--replace "thinc>=7.4.1,<7.5.0" "thinc>=7.4.1,<8"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're relaxing dependencies, would it make sense to just change them to e.g. blis vs blis>=0.4.0...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see the arguments on both sides. But as a maintainer, I'd prefer to check compatibility upstream if thinc (or any of the other dependencies) has a new major version, than possibly silently failing (since we cannot run the regular unit tests and instead rely on the single passthru.tests test).

Moreover, thinc should not be updated without taking spaCy into consideration anyway. spaCy is the only downstream package that uses it, so thinc is in nixpkgs for spaCy.

pkgs/development/python-modules/spacy/default.nix
'';

pythonImportsCheck = [ "spacy" ];

passthru.tests = callPackage ./annotation-test {};
passthru.tests.annotation = callPackage ./annotation-test { };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what this does, honestly. A comment might be helpful?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About what part? passtru.tests is documented in the nixpkgs manual. annotation is the name of the test, because it is an annotation test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more accurately, nix build commands can also build a set of derivations. This is just one way to organize many tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'd usually not seen it in this format, so I got confused

@drewrisinger drewrisinger self-requested a review December 12, 2020 17:15
drewrisinger
drewrisinger suggested changes Dec 12, 2020
View reviewed changes
Copy link
Contributor

@drewrisinger drewrisinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clicked wrong button, see comments above

drewrisinger
drewrisinger approved these changes Dec 14, 2020
View reviewed changes
Copy link
Contributor

@drewrisinger drewrisinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments addressed. Build was fine, commits haven't changed.

@danieldk
Copy link
Contributor Author

Comments addressed. Build was fine, commits haven't changed.

Sorry, I missed the comment about the spelling error. I’ll fix that tomorrow, thanks!

@danieldk danieldk changed the title python3Packagese.spacy: 2.3.4 -> 2.3.5 python3Packages.spacy: 2.3.4 -> 2.3.5 Dec 15, 2020
@danieldk
Copy link
Contributor Author

Comments addressed. Build was fine, commits haven't changed.

Sorry, I missed the comment about the spelling error. I’ll fix that tomorrow, thanks!

The typo is removed from the first commit message.

@danieldk danieldk merged commit 9631eb8 into NixOS:master Dec 16, 2020
@danieldk danieldk deleted the spacy-2.3.5 branch December 16, 2020 07:21
@jonringer
Copy link
Contributor

I get:

-x86_64-linux-gnu.so
  Setting up data for gemm. 1000 iters,  nO=384 nI=384 batch_size=2000
  Blis gemm...
  Total: 11032014.6484375
  5.20 seconds
  Numpy (lapack) gemm...
  BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
  BLAS : Program is Terminated. Because you tried to allocate too many memory regions.
  /nix/store/iab5r4liiwpijdvy5ri755wjf3pha2z4-setuptools-check-hook/nix-support/setup-hook: line 4:  9618 Segmentation fault      (core dumped) /nix/store/z2i5h9nvk51iism38d3ia5gf85w3nqyn-python3-3.7.9/bin/python3.7 nix_run_setup test
cannot build derivation '/nix/store/gbh4yrva40bqa7bs5xm5ymwfssgr2szc-python3.7-thinc-7.4.5.drv': 1 dependencies couldn't be built

but probably related to OpenMathLib/OpenBLAS#2995

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@drewrisinger drewrisinger drewrisinger approved these changes

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

@aptlin aptlin Awaiting requested review from aptlin

Assignees
No one assigned
Labels
6.topic: python 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 11.by: package-maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@danieldk @jonringer @drewrisinger