Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pythonPackages.tensorflow 1.1 -> 1.3 #29229

Closed
wants to merge 1 commit into from
Closed

pythonPackages.tensorflow 1.1 -> 1.3 #29229

wants to merge 1 commit into from

Conversation

jyp
Copy link
Contributor

@jyp jyp commented Sep 11, 2017

Motivation for this change

Upstream update

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@jyp jyp requested a review from FRidh as a code owner September 11, 2017 09:25
@mention-bot
Copy link

@jyp, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edwtjo, @FRidh and @abbradar to be potential reviewers.

@akamaus
Copy link
Contributor

akamaus commented Sep 12, 2017

@jyp what happens to tensorboard after merging this? AFAIK it was separated from tensorflow-1.3. I'm afraid we need a separate expression for it.

@jyp
Copy link
Contributor Author

jyp commented Sep 12, 2017

@akamus: This is correct: this PR removes tensorboard. However, unfortunately, the dependencies of tensorboard do not build at the moment (see the commit). So the PR is not a regression.

@akamaus
Copy link
Contributor

akamaus commented Sep 12, 2017

@jyp If you're speaking about werkzeug, then there're good news :) Recently I added python3.6 support to tensorflow expression, it got merged 6abb29f. cffi works normally on 3.6 so now tensorflow-1.1 is buildable again. Maybe it's better to rebase your work against that commit?

@jyp
Copy link
Contributor Author

jyp commented Sep 12, 2017

@akamaus It does not look like your commit made it to master --- regardless I suppose that if werkzeug is working with python 3.6 then the best is to drop support for 3.5 and go for 3.6 and add tensorboard properly. I'll give it a shot.

@jyp
Copy link
Contributor Author

jyp commented Sep 12, 2017

In fact, tensorboard still won't build, because it depends on exactly htlm5lib 9999999 (that's seven nines), which does not build. If you know @akamaus where to complain to get tensorboard people to put decent dependencies on their lib then let me know.

I'll reiterate my observation that tensorflow works perfectly well without tensorboard. So I maintain that this PR is the way to go until tensorboard uses saner dependencies.

@akamaus
Copy link
Contributor

akamaus commented Sep 14, 2017

oh, I see now, https://github.com/tensorflow/tensorboard/blob/master/tensorboard/pip_package/setup.py

At first I thought we should file issue there... But look, nixpkgs (I looked at 17.09) have weird version too

  html5lib = buildPythonPackage (rec {
    version = "0.999999999";
    name = "html5lib-${version}";

    src = pkgs.fetchurl {
      url = "http://github.com/html5lib/html5lib-python/archive/${version}.tar.gz";
      sha256 = "09j6194f5mlnd5xwbavwvnndwl1x91jw74shxl6hcxjp4fxg3h05";
    };

Please, note, there are different number of nines ;)
I have no experience with python packaging, so I can't comment on this practice.

@jyp
Copy link
Contributor Author

jyp commented Sep 14, 2017

Well the terrible html5lib versioning scheme does not help with typos and misunderstandings of all kind. But the crux of the matter seems to be that tensorboard people are reluctant to upgrade to bleach 2.0.

tensorflow/tensorboard#427

And I'm not to thrilled at the idea of packaging essentially unmaintained packages.

So, to re-interate, because tensorboard is not an essential component of ten, I am dropping it until they eventually move to upgrade. (Also they claim that they don't even support python 3.6; see above issue).

@jyp jyp force-pushed the tf1.3 branch 3 times, most recently from 20ce68e to 5bbf1b1 Compare September 14, 2017 08:49
@abbradar
Copy link
Member

FWIW I have a working build with TensorBoard (which I made separately) but it doesn't support Python 3.6, so I guess your approach is better.

6abb29f has been merged recently which messes with your patch; you probably can just revert it and leave only 2.7 and 3.6.

@abbradar
Copy link
Member

Giving more thought to this, can you instead make your package support both 3.5 and 3.6 like it currently does on master? We'll then merge my tensorboard on top of your update, so people can have it with tensorboard on 3.5 and without on 3.6.

@jyp
Copy link
Contributor Author

jyp commented Sep 17, 2017

@abbradar I'm not too fond of having to track too many wheel versions, in honesty I won't have the time to test them. (Is there even a good way to automatically update the hashes when a new version comes out?)

If you have a working tensorboard please submit it as a separate PR on top of this one.

@abbradar
Copy link
Member

@jyp There are some projects to automate this but I haven't tried them yet.

For some reason I can't submit a pull request to you -- GitHub doesn't allow me to select your branch as a base fork. Please check out https://github.com/abbradar/nixpkgs/tree/tf1.3

@jyp
Copy link
Contributor Author

jyp commented Sep 19, 2017

@abbradar Thanks a lot; hopefully this PR will be merged soon (I suppose code owners are busy with the upcoming release right now) and then we can proceed with yours.

@FRidh any chance to merge this PR as it seems that we're reaching a consensus on the way forward?

@abbradar
Copy link
Member

@jyp Notice that currently this PR is broken because of commit adding Python 3.5 support on master! You have py3 and py36 attributes with py36 being old TensorFlow.

@jyp
Copy link
Contributor Author

jyp commented Sep 19, 2017

@abbradar I am pretty sure that I rebased it. ("This branch has no conflicts with the base branch")

sha256 = "0ahz9222rzqrk43lb9w4m351klkm6mlnnvw8xfqip28vbmymw90b";
py3 = {
url = tfurl "linux" "cpu" "cp36-cp36m-linux_x86_64";
sha256 = "1qm8lm2f6bf9d462ybgwrz0dn9i6cnisgwdvyq9ssmy2f1gp8hxk";
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jyp The problem is for example here, notice py3 and py36 are both in your result. Git still managed to merge it without conflicts ;)

@jyp
Copy link
Contributor Author

jyp commented Sep 19, 2017

Alright; the PR was not in fact broken but had some dead code. Fixed anyway :)

@abbradar
Copy link
Member

Yeah, I used too strong a word. I have tested this locally (not on GPUs) and it works for me. Will merge in several days unless anybody objects.

Thank you for your work!

@abbradar
Copy link
Member

Merged along with my support for 3.5 in 01d53df. Tested on a simple VAE, worked for me.

@jyp jyp closed this Oct 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants