[WIP] bazel, tensorflow: enable Darwin source build #37608

lukeadams · 2018-03-22T03:57:15Z

Motivation for this change

Tracking for WIP patches.

Need to break apart commits since some Bazel commits are mixed in with the tf commit
need to set LD to re-enable bazel tests on Darwin
also need to make the changes Darwin only since linux doesn't need these hacks. usually just lib.optionalString

Things done

With this PR tf is buildable for py27/py36, sidestepping the unicode mismatch when using a wheel for py27. This also enables tf c++ artifacts, but I haven't tested those yet.

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

matthewbauer · 2018-03-22T04:27:03Z

pkgs/development/tools/build-managers/bazel/default.nix

@@ -71,7 +92,7 @@ stdenv.mkDerivation rec {

  # Build the CPP and Java examples to verify that Bazel works.

-  doCheck = true;
+  doCheck = false;


This should be a conditional, I think.

I'll change this to (!stdenv.isDarwin) if I can't get tests working

matthewbauer · 2018-03-22T04:34:10Z

pkgs/development/python-modules/tensorflow/default.nix

+        cp test.sh cxx.sh
+        cp test.sh ld.sh
+
+        echo "(clang \"\$@\" ) || (clang++ \"\$@\" )" >> cc.sh


Have you tried just doing "clang++" for this? I think that clang++ should support both (although some of the Nix purity stuff may break this).

I initially tried this, but it caused failures when building a few dependencies (I think mostly Protobuf). I'll try to find a log so I can at least put a comment explaining it.

matthewbauer · 2018-03-22T04:34:44Z

Thanks for working on this! Tensorflow on Darwin would be great.

abbradar · 2018-03-22T09:33:36Z

pkgs/development/python-modules/tensorflow/default.nix

@@ -88,7 +92,7 @@ let

    hardeningDisable = [ "all" ];

-    bazelFlags = [ "--config=opt" ]
+    bazelFlags = [ "--config=opt" "--config=monolithic" ]


Please leave a comment on why is this necessary; let's make it optional for Darwin to avoid static dependencies as much as possible.

abbradar · 2018-03-22T09:34:42Z

pkgs/development/python-modules/tensorflow/default.nix

@@ -58,7 +61,8 @@ let
    nativeBuildInputs = [ swig which ];

    buildInputs = [ python jemalloc openmpi glibcLocales numpy ]
-      ++ lib.optionals cudaSupport [ cudatoolkit cudnn nvidia_x11 ];
+      ++ lib.optionals cudaSupport [ cudatoolkit cudnn nvidia_x11 ]


Mark stdenv.isDarwin && cudaSupport as broken.

Would it be a good idea to move the asserts at the top to meta.broken? (see #36229)

LnL7 · 2018-03-24T20:40:12Z

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-03-24T20:40:32Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-03-24T20:42:15Z

Failure on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

Target //src:bazel failed to build
[6 / 177] no action

INFO: Elapsed time: 5.008s, Critical Path: 0.07s
[6 / 177] no action

FAILED: Build did NOT complete successfully


ERROR: Could not build Bazel
builder for ‘/nix/store/qi5r9dr1xg5ryz972nkif7hwm7d434qr-bazel-0.10.1.drv’ failed with exit code 1
error: build of ‘/nix/store/qi5r9dr1xg5ryz972nkif7hwm7d434qr-bazel-0.10.1.drv’ failed

GrahamcOfBorg · 2018-03-24T20:58:16Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

    [Sched] Linking external/com_google_protobuf/js_embed [for host]
Target //src:bazel failed to build
�[32m[1,179 / 1,914]�[0m no action
�[32mINFO: �[0mElapsed time: 622.827s, Critical Path: 173.37s
�[32m[1,179 / 1,914]�[0m no action
�[31m�[1mFAILED:�[0m Build did NOT complete successfully
�[0m
ERROR: Could not build Bazel
builder for '/nix/store/nwzhb2hkqacflb2jw23f61c4a51539r8-bazel-0.10.1.drv' failed with exit code 1
�[31;1merror:�[0m build of '/nix/store/nwzhb2hkqacflb2jw23f61c4a51539r8-bazel-0.10.1.drv' failed

removes need to include jdk/jre in dependent packages

lukeadams · 2018-03-24T23:37:09Z

@abbradar I cleaned it up quite a bit and added conditionals on everything. Rebuilt py27 + py36 and both succeeded!

hacky method to solidify NIX_* env vars as Bazel does not pass these through when calling cc/cxx

bazel cc/cxx hacks on darwin

LnL7 · 2018-03-28T18:58:59Z

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-03-28T18:59:26Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-03-28T19:12:04Z

Success on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

//examples/java-native/src/test/java/com/example/myproject:hello         PASSED
 in 0.4s

Executed 2 out of 2 tests: 2 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
installing
post-installation fixup
patching script interpreter paths in /nix/store/6rdvkgs8dk2ypyi962chgsj1mldppvjw-bazel-0.10.1
checking for references to /tmp in /nix/store/6rdvkgs8dk2ypyi962chgsj1mldppvjw-bazel-0.10.1...
/nix/store/6rdvkgs8dk2ypyi962chgsj1mldppvjw-bazel-0.10.1

GrahamcOfBorg · 2018-03-28T19:30:18Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

    [Sched] Linking third_party/zlib/libzlib.a
Target //src:bazel failed to build
�[32m[687 / 952]�[0m no action
�[32mINFO: �[0mElapsed time: 311.403s, Critical Path: 182.49s
�[32m[687 / 952]�[0m no action
�[31m�[1mFAILED:�[0m Build did NOT complete successfully
�[0m
ERROR: Could not build Bazel
builder for '/nix/store/61nlfs2w8xv3r1ahgxx5nmjwi9c9brch-bazel-0.10.1.drv' failed with exit code 1
�[31;1merror:�[0m build of '/nix/store/61nlfs2w8xv3r1ahgxx5nmjwi9c9brch-bazel-0.10.1.drv' failed

LnL7 · 2018-03-28T19:42:35Z

Do you have the CLT/Xcode installed?

lukeadams · 2018-03-28T20:56:21Z

@LnL7 Ahh yep (just CLT) – good catch. Since I need it on my main machine, I can work it out Friday on a VM without it installed.

Sorry about that!

LnL7 · 2018-03-28T21:13:34Z

That might explain the missing framework, if you add it I can start another build.

lukeadams · 2018-04-03T20:44:57Z

@abbradar I'm not going to have time to finish this for the next month, so maybe you can merge the wheel as python3 only?

@LnL7 I added the framework, but there are other issues due to the way Bazel calls xcrun/xcbuild. It may be simple to patch, but I don't have time to play with it

LnL7 · 2018-04-03T20:47:37Z

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-04-03T20:48:17Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

lukeadams · 2018-04-03T20:58:26Z

@LnL7 The framework was in my staging branch; I just pushed it to this one

GrahamcOfBorg · 2018-04-03T20:59:12Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

    Linking third_party/protobuf/3.4.0/libprotobuf.a [for host]; 0s local
Target //src:bazel failed to build
�[32m[270 / 298]�[0m no action
�[32mINFO: �[0mElapsed time: 253.583s, Critical Path: 45.79s
�[32m[270 / 298]�[0m no action
�[31m�[1mFAILED:�[0m Build did NOT complete successfully
�[0m
ERROR: Could not build Bazel
builder for '/nix/store/nygjcxa5y2n0xascw1n913fjbfz874d7-bazel-0.11.1.drv' failed with exit code 1
�[31;1merror:�[0m build of '/nix/store/nygjcxa5y2n0xascw1n913fjbfz874d7-bazel-0.11.1.drv' failed

GrahamcOfBorg · 2018-04-03T21:02:17Z

Success on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

//examples/java-native/src/test/java/com/example/myproject:hello
PASSED in 0.4s

Executed 2 out of 2 tests: 2 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
installing
post-installation fixup
patching script interpreter paths in /nix/store/pmlkwlxlhh1qji7r71iy304m679n1pqk-bazel-0.11.1
checking for references to /tmp in /nix/store/pmlkwlxlhh1qji7r71iy304m679n1pqk-bazel-0.11.1...
/nix/store/pmlkwlxlhh1qji7r71iy304m679n1pqk-bazel-0.11.1

LnL7 · 2018-04-03T21:26:01Z

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-04-03T21:26:21Z

Success on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

/nix/store/pmlkwlxlhh1qji7r71iy304m679n1pqk-bazel-0.11.1

GrahamcOfBorg · 2018-04-03T21:26:29Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-04-03T21:56:29Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

ld: warning: object file (bazel-out/host/bin/src/main/tools/_objs/linux-sandbox/src/main/tools/dummy-sandbox.o) was built for newer OSX version (10.13) than being linked (10.10)
INFO: From Linking src/main/tools/build-runfiles [for host]:
ld: warning: object file (bazel-out/host/bin/src/main/tools/_objs/build-runfiles/src/main/tools/build-runfiles.o) was built for newer OSX version (10.13) than being linked (10.10)
[460 / 501] Compiling third_party/protobuf/3.4.0/src/google/protobuf/descriptor.cc [for host]; 45s local ... (8 actions running)
INFO: From Linking third_party/protobuf/3.4.0/libprotobuf.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/third_party/protobuf/3.4.0/libprotobuf.a(gzip_stream.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/third_party/protobuf/3.4.0/libprotobuf.a(error_listener.o) has no symbols
[503 / 543] Compiling external/com_google_protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc [for host]; 5s local ... (8 actions running)
building of '/nix/store/qq2nbfqzx0s02sz57x0kbgv939lnhj64-bazel-0.11.1.drv' timed out after 1800 seconds
�[31;1merror:�[0m build of '/nix/store/qq2nbfqzx0s02sz57x0kbgv939lnhj64-bazel-0.11.1.drv' failed

abbradar · 2018-04-05T12:00:49Z

@lukeadams No sweat! I've merged #37044 for now.

LnL7 · 2018-04-05T17:28:11Z

I was hoping the framework was the only issue, but I get an xcrun failure locally. 😕

lukeadams · 2018-05-14T02:03:39Z

Closed until I have time to work on this. Not sure when tbh :/

#40424 with full-paths.patch patch looks interesting for Darwin

lukeadams requested a review from FRidh as a code owner March 22, 2018 03:57

GrahamcOfBorg added the 6.topic: darwin Running or building packages on Darwin label Mar 22, 2018

lukeadams mentioned this pull request Mar 22, 2018

tensorflow: add a binary wheel release for Darwin #37044

Merged

8 tasks

GrahamcOfBorg added 6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100 labels Mar 22, 2018

matthewbauer reviewed Mar 22, 2018

View reviewed changes

abbradar reviewed Mar 22, 2018

View reviewed changes

matthewbauer added 2.status: work-in-progress and removed 2.status: work-in-progress labels Mar 22, 2018

bazel: add JAVA_HOME to wrapProgram

a52aaf8

removes need to include jdk/jre in dependent packages

lukeadams force-pushed the tf-darwin branch from 8572cfb to a987fdf Compare March 24, 2018 23:33

Luke Adams added 2 commits March 24, 2018 22:00

bazel: enable Darwin building

e398d60

hacky method to solidify NIX_* env vars as Bazel does not pass these through when calling cc/cxx

tensorflow: enable Darwin building

e215aef

bazel cc/cxx hacks on darwin

lukeadams force-pushed the tf-darwin branch from a987fdf to e215aef Compare March 25, 2018 03:00

lukeadams changed the title ~~[WIP] bazel, tensorflow: enable Darwin source build~~ bazel, tensorflow: enable Darwin source build Mar 25, 2018

bazel: add foundation.framework

7403394

lukeadams changed the title ~~bazel, tensorflow: enable Darwin source build~~ [WIP] bazel, tensorflow: enable Darwin source build Apr 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] bazel, tensorflow: enable Darwin source build #37608

[WIP] bazel, tensorflow: enable Darwin source build #37608

lukeadams commented Mar 22, 2018 •

edited

matthewbauer Mar 22, 2018

lukeadams Mar 24, 2018

matthewbauer Mar 22, 2018

lukeadams Mar 24, 2018 •

edited

matthewbauer commented Mar 22, 2018

abbradar Mar 22, 2018

abbradar Mar 22, 2018

lukeadams Mar 24, 2018

lukeadams Mar 25, 2018 •

edited

LnL7 commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

lukeadams commented Mar 24, 2018 •

edited

LnL7 commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

LnL7 commented Mar 28, 2018

lukeadams commented Mar 28, 2018

LnL7 commented Mar 28, 2018

lukeadams commented Apr 3, 2018

LnL7 commented Apr 3, 2018 •

edited

GrahamcOfBorg commented Apr 3, 2018

lukeadams commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

LnL7 commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

abbradar commented Apr 5, 2018

LnL7 commented Apr 5, 2018

lukeadams commented May 14, 2018

[WIP] bazel, tensorflow: enable Darwin source build #37608

[WIP] bazel, tensorflow: enable Darwin source build #37608

Conversation

lukeadams commented Mar 22, 2018 • edited

Motivation for this change

Things done

matthewbauer Mar 22, 2018

Choose a reason for hiding this comment

lukeadams Mar 24, 2018

Choose a reason for hiding this comment

matthewbauer Mar 22, 2018

Choose a reason for hiding this comment

lukeadams Mar 24, 2018 • edited

Choose a reason for hiding this comment

matthewbauer commented Mar 22, 2018

abbradar Mar 22, 2018

Choose a reason for hiding this comment

abbradar Mar 22, 2018

Choose a reason for hiding this comment

lukeadams Mar 24, 2018

Choose a reason for hiding this comment

lukeadams Mar 25, 2018 • edited

Choose a reason for hiding this comment

LnL7 commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

GrahamcOfBorg commented Mar 24, 2018

lukeadams commented Mar 24, 2018 • edited

LnL7 commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

GrahamcOfBorg commented Mar 28, 2018

LnL7 commented Mar 28, 2018

lukeadams commented Mar 28, 2018

LnL7 commented Mar 28, 2018

lukeadams commented Apr 3, 2018

LnL7 commented Apr 3, 2018 • edited

GrahamcOfBorg commented Apr 3, 2018

lukeadams commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

LnL7 commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

GrahamcOfBorg commented Apr 3, 2018

abbradar commented Apr 5, 2018

LnL7 commented Apr 5, 2018

lukeadams commented May 14, 2018

lukeadams commented Mar 22, 2018 •

edited

lukeadams Mar 24, 2018 •

edited

lukeadams Mar 25, 2018 •

edited

lukeadams commented Mar 24, 2018 •

edited

LnL7 commented Apr 3, 2018 •

edited