Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] sage: 8.9 -> 9.0, switch to python3 #70474

Closed
wants to merge 7 commits into from
Closed

Conversation

timokau
Copy link
Member

@timokau timokau commented Oct 5, 2019

Motivation for this change

Tracking the betas & release candidates for the next sage update.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @

@timokau timokau mentioned this pull request Nov 24, 2019
10 tasks
@timokau timokau changed the title [WIP] sage: 8.9 -> 9.0 [WIP] sage: 8.9 -> 9.0, switch to python3 Dec 4, 2019
@akirakyle
Copy link

I was wondering what the status of this is? It appears that as of Jan 1, 2020, sage 9.0 has been released running entirely on python 3 and I was hoping to use it.

Thanks!

@timokau
Copy link
Member Author

timokau commented Feb 27, 2020

Always good to see that somebody is interested in this :) The issue here is that upstream sage insists on maintaining python2 support for 9.0. As a result, they're stuck on outdated dependency versions and some work is needed to make it work with our newer (python3) versions.

The main culprit that was blocking me is sphinx: https://trac.sagemath.org/ticket/28856

Since I don't have too much time to devote to this right now and I don't agree with upstream sage's decision, I didn't continue working on it for now. So this is blocked until somebody picks it up, I find time and motivation or sage 9.1 is released and drops python2 support.

@akirakyle
Copy link

Ah I see, this really is a complicated piece of software to package! Thanks for updating me with what's keeping this from happening. I suppose it might be easiest to just wait for sage 9.1 and not have to deal with python2. For now I am making due with just SymPy which covers pretty much everything I need to do anyways so thanks for also maintaining that package.

@timokau
Copy link
Member Author

timokau commented Feb 28, 2020

Ah I see, this really is a complicated piece of software to package!

Yeah, unfortunately very complicated.

Thanks for updating me with what's keeping this from happening. I suppose it might be easiest to just wait for sage 9.1 and not have to deal with python2.

Yeah that might be the option I'll go with. Theoretically I could also package sage 9.0 with python2, but then we'd ship sage that is different from the "blessed" version and that would probably only add to the confusion.

For now I am making due with just SymPy which covers pretty much everything I need to do anyways so thanks for also maintaining that package.

You're welcome! Luckily that one is much easier to package.

@timokau timokau mentioned this pull request Mar 10, 2020
@matthuszagh
Copy link
Contributor

Per the discussion above, I thought it might be worth mentioning ticket 29141. Apparently, support for python2 will be removed in sage9.2. So, once 9.2 is released, we shouldn't experience the same issues upgrading sage.

@timokau
Copy link
Member Author

timokau commented Jun 4, 2020

Yes, and we probably don't have to wait that long either. The development branch dropped python2 support just recently. I expect the upstream effort to add compatibility with newer python libraries to pick up steam now. Once that happens, we can hopefully backport those changes to the current stable sage version.

@timokau timokau mentioned this pull request Jul 7, 2020
@Synthetica9
Copy link
Member

So this is blocked until somebody picks it up, I find time and motivation or sage 9.1 is released and drops python2 support.

@timokau Now that 9.1 is released, what is the status of this?

@timokau
Copy link
Member Author

timokau commented Jul 28, 2020

The update is likely possible now. It will likely be quite a bit of work though. So basically the status is that I didn't have the time yet, and don't know when I will. Feel free to take a shot at it :)

omasanori added a commit to omasanori/nixpkgs that referenced this pull request Oct 20, 2020
Some changes are derived from NixOS#70474
originally published by Timo Kaufmann.

Signed-off-by: Masanori Ogino <167209+omasanori@users.noreply.github.com>
@omasanori
Copy link
Contributor

Feel free to take a shot at it :)

So did I. #101116 😃
It seems not trivial, as expected, though.

@timokau
Copy link
Member Author

timokau commented Oct 20, 2020

Nice! Personally I'd suggest to skip 9.1 and just go straight to 9.2, which is in the release-candidate phase and should be released soon-ish. It has some additional patches for better compatibility. Then I'd go straight to python3, which should be easier at this point. Some of the current hacks can probably be removed.

As can be seen in this thread, I don't really have the time and motivation to be the main sage maintainer anymore. I might do another update to 9.2 to get it working again, but it would probably break again soon enough. I don't really use it myself anymore, and I think a maintainer with an actual use-case would be able to do a better job.

Are you or anybody else in this thread willing to help out? You could take over the main sage maintenance while I would take more of an advisory role.

Its a significant commitment. I have some hope that the maintenance burden has become a little easier since 8.9 (due to the python3 change and a bigger upstream emphasis on packaging), but keeping sage running still takes quite some effort. Too much effort for me alone.

@omasanori
Copy link
Contributor

First, thank you so much for your effort, @timokau!

While I am not a SageMath user as an author, I want to keep SageMath working, as some cryptographers use the software to describe and analyze algorithms. (According to a quick survey by Google site:eprint.iacr.org <software> queries, the popularity seems somewhere between Mathematica and Magma.)

I guess I cannot maintain the package alone since I am a novice Nix user and SageMath is so complex, but I am glad to become a co-maintainer with some help from experienced Nix developer(s).

@timokau
Copy link
Member Author

timokau commented Oct 20, 2020

Once you have a working base, most of the maintenance work is

  1. keeping an eye on test failures,
  2. noticing them early and using git-bisect to detect their source
  3. look for upstream tickets on trac.sagemath.org, open one if none exists.

Often the issue is either trivial or somebody has already fixed it and you just need to apply a fix. The sage-packaging mailing list can be helpful. Bonus points for following sage-release and/or sage-devel, but those are much higher volume.

Most of the time its just some small update breaking compatibility in a minor way. Sometimes the tests just need adjustment.

If somebody was willing to do at least (1) and (2) (pinging me with "sage test xyz was broken by nixpkgs commit asdf"), that would be a huge help. At that point we could find a solution together. Bisecting sage does take a bit of compute (depending on your resources, something in the order of one or two hours to build sagelib and run the test suite; much faster if you just run an individual test).

@yurrriq
Copy link
Member

yurrriq commented Oct 21, 2020

I have recently fallen in love with SageMath as a hobby user (and GAP convert), and am an intermediate/advanced Nix user. My main problem these days is time. I took a brief stab at packaging 9.1 the other day, but the lengthy build time was prohibitive. I can't imagine I'll stop studying maths in my free time, so I'm happy to help maintain this going forward. Is there anything in particular I can help with this weekend?

@omasanori
Copy link
Contributor

Welcome @yurrriq!
I have been working on identifying patches and workarounds that are no longer required for SageMath 9.2.rc3. Thanks to @timokau and upstream developers, I guess only a handful of them will be kept, unless I find new pitfalls.
I will publish my WIP branch, including patches to other packages for this moment due to build failures, this weekend so that you can review if you want.

@timokau
Copy link
Member Author

timokau commented Oct 22, 2020

Very glad to see progress here, a team of 2-3 people would be much more sustainable. One tip that is very useful for bisecting: Use

nix-build -E 'with (import ./. {}); sage.tests.override { files = [ "src/sage/interfaces/sagespawn.pyx" ]; }'

To only run the tests for a single file instead of the whole test suite.

@ryantm ryantm marked this pull request as draft October 23, 2020 03:05
@omasanori
Copy link
Contributor

Here is my progress on the upgrade: #101447
Nothing works correctly, but Cython and GCC seem happy with this, at least.

@timokau timokau mentioned this pull request Nov 6, 2020
10 tasks
@timokau timokau mentioned this pull request Nov 26, 2020
10 tasks
@timokau
Copy link
Member Author

timokau commented Dec 19, 2020

Superseded by #105615 🎉

@timokau timokau closed this Dec 19, 2020
Python 2 deprecation automation moved this from WIP to Done Dec 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

None yet

7 participants