Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

julia: 1.0.0 -> 1.0.1 #48191

Merged
merged 2 commits into from Oct 22, 2018
Merged

julia: 1.0.0 -> 1.0.1 #48191

merged 2 commits into from Oct 22, 2018

Conversation

f--t
Copy link
Contributor

@f--t f--t commented Oct 11, 2018

Motivation for this change

updating julia to next version,
disabled tests as they fail regardless of update

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@fpletz
Copy link
Member

fpletz commented Oct 11, 2018

@GrahamcOfBorg build julia

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: julia

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: julia

Partial log (click to expand)

copying path '/nix/store/j51sisi1p4vdh15irrz3420mh69xs8z3-fftw-double-3.3.8' from 'https://cache.nixos.org'...
copying path '/nix/store/gvjsashnn5k3hcs951siclj6ayv5rx1p-fftw-single-3.3.8' from 'https://cache.nixos.org'...
copying path '/nix/store/6qakplf69vkv9zzfi06c00iqz0jky4cb-gmp-6.1.2' from 'https://cache.nixos.org'...
copying path '/nix/store/byybrsvlglbirfqhv6smwxxbydlqxzby-llvm-3.9.1-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/nggdvvhnmjcvbq8jvpxcrjp9365ya72v-openlibm-0.6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/bg4fzy60bj96k1ffy9wfb3zc4pz7kdc4-openspecfun-0.5.3' from 'https://cache.nixos.org'...
copying path '/nix/store/hc6n1ak3cvn62nx0xqbvd5mgxy9z56kj-suitesparse-4.4.4' from 'https://cache.nixos.org'...
copying path '/nix/store/x26d5qc5ykyj0j7ya7nfy4rjimk53177-llvm-3.9.1' from 'https://cache.nixos.org'...
copying path '/nix/store/4gplsgr81q13sdl0gvljk9i1856hfizs-julia-0.6.4' from 'https://cache.nixos.org'...
/nix/store/4gplsgr81q13sdl0gvljk9i1856hfizs-julia-0.6.4

@fpletz
Copy link
Member

fpletz commented Oct 11, 2018

@GrahamcOfBorg build julia_10

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: julia_10

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: julia

Partial log (click to expand)

   [6] collect(::Base.Generator{Array{Symbol,1},Base.Distributed.##153#154}) at ./array.jl:470
   [7] collect(::Base.Generator{Array{Symbol,1},Base.Distributed.##153#154}) at ./array.jl:471
  while loading /build/source/test/linalg/arnoldi.jl, in expression starting on line 130
ERROR: LoadError: Test run finished with errors
while loading /build/source/test/runtests.jl, in expression starting on line 29
make[1]: *** [Makefile:18: all] Error 1
make[1]: Leaving directory '/build/source/test'
make: *** [Makefile:603: testall] Error 2
builder for '/nix/store/49jhy1d3dnfi354i8bln1gg4nf5cciiw-julia-0.6.4.drv' failed with exit code 2
error: build of '/nix/store/49jhy1d3dnfi354i8bln1gg4nf5cciiw-julia-0.6.4.drv' failed

@GrahamcOfBorg
Copy link

Timed out, unknown build status on x86_64-darwin (full log)

Attempted: julia_10

Partial log (click to expand)

[ 74%] Building CXX object lib/Target/AMDGPU/CMakeFiles/LLVMAMDGPUCodeGen.dir/AMDGPUIntrinsicInfo.cpp.o
[ 74%] Building CXX object lib/Target/NVPTX/CMakeFiles/LLVMNVPTXCodeGen.dir/NVPTXUtilities.cpp.o
[ 76%] Building CXX object lib/Target/AMDGPU/CMakeFiles/LLVMAMDGPUCodeGen.dir/AMDGPUISelDAGToDAG.cpp.o
[ 76%] Building CXX object lib/Target/NVPTX/CMakeFiles/LLVMNVPTXCodeGen.dir/NVVMIntrRange.cpp.o
[ 76%] Building CXX object lib/Target/NVPTX/CMakeFiles/LLVMNVPTXCodeGen.dir/NVVMReflect.cpp.o
[ 76%] Linking CXX static library ../../libLLVMNVPTXCodeGen.a
[ 76%] Built target LLVMNVPTXCodeGen
[ 76%] Building CXX object lib/Target/AMDGPU/CMakeFiles/LLVMAMDGPUCodeGen.dir/AMDGPUISelLowering.cpp.o
building of '/nix/store/0h773nwcd83jbn23q42ma4yn855cj0w4-julia-1.0.1.drv' timed out after 1800 seconds
error: build of '/nix/store/0h773nwcd83jbn23q42ma4yn855cj0w4-julia-1.0.1.drv' failed

@GrahamcOfBorg
Copy link

Timed out, unknown build status on x86_64-linux (full log)

Attempted: julia_10

Partial log (click to expand)

[ 47%] Building CXX object lib/Transforms/Instrumentation/CMakeFiles/LLVMInstrumentation.dir/BoundsChecking.cpp.o
[ 47%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/LowerSwitch.cpp.o
[ 47%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/Mem2Reg.cpp.o
[ 47%] Building CXX object lib/Transforms/Instrumentation/CMakeFiles/LLVMInstrumentation.dir/DataFlowSanitizer.cpp.o
[ 47%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/MetaRenamer.cpp.o
[ 47%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/ModuleUtils.cpp.o
[ 47%] Building CXX object lib/Transforms/Instrumentation/CMakeFiles/LLVMInstrumentation.dir/GCOVProfiling.cpp.o
[ 47%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/NameAnonGlobals.cpp.o
building of '/nix/store/f19321jyd3043xjgl16aw4x3inx8c4nn-julia-1.0.1.drv' timed out after 1800 seconds
error: build of '/nix/store/f19321jyd3043xjgl16aw4x3inx8c4nn-julia-1.0.1.drv' failed

@f--t
Copy link
Contributor Author

f--t commented Oct 11, 2018

it seems the build failure is a known issue with Julia i.e. JuliaLang/julia#29221

@nalimilan
Copy link

By the way, you'll probably be interested in JuliaLang/julia#29592.

@f--t
Copy link
Contributor Author

f--t commented Oct 14, 2018

As it currently stands, there needs to be a lot of manual tweaking to get Julia work on Nixos. For instance, lots of .jl packages depend on Arpack.jl which in turned depend on libarpack. The .so bundled upstream doesn't work on nixos, so one needs to copy the libarpack.so's from nix store manually into the package's directory.

@nalimilan
Copy link

As it currently stands, there needs to be a lot of manual tweaking to get Julia work on Nixos. For instance, lots of .jl packages depend on Arpack.jl which in turned depend on libarpack. The .so bundled upstream doesn't work on nixos, so one needs to copy the libarpack.so's from nix store manually into the package's directory.

That's due to using an LP64 rather than an ILP64 BLAS. To fix that, you need to pass USE_BLAS64=1 Unfortunately, OpenBLAS packages are generally LP64, so you either need to build a custom OpenBLAS for Julia, or add an ILP64 package. This is what Fedora does for a few years. If you do that, also adding the 64_ suffix is a good idea to avoid conflicts between symbols if both libraries are loaded at the same time (then you can build Julia with OPENBLAS_SYMBOLSUFFIX=64_).

@markuskowa
Copy link
Member

openblasCompat provides the LP64 version of openblas. However, arpack is build against openblas with an ILP64 interface. Could that be a problem here?

@f--t
Copy link
Contributor Author

f--t commented Oct 21, 2018

@markuskowa
It seems there are two std test suits that currently don't pass which breaks the build, Sockets and Distributed. Failure in Distributed looks related to Sockets. If the breaking tests at Sockets are reordered such that they're first in the test file; all the tests seem to pass. I think the failing test all have something to do with DNS. I'm not sure why they don't fail upstream!

@7c6f434c
Copy link
Member

7c6f434c commented Oct 21, 2018 via email

@eli-schwartz
Copy link

eli-schwartz commented Oct 21, 2018 •
edited

Julia disables the socket tests in one of their CI configs, none of which use make test. In my distro package for Arch Linux I blindly assumed that that meant the tests weren't ever expected to work at all, so I disabled them too...

@7c6f434c
Copy link
Member

Do I understand correctly that the commits in this PR are currently only about correctly switching off the failing tests and not version bump?

@f--t
Copy link
Contributor Author

f--t commented Oct 21, 2018

the version bump, it appears to be already cherry picked, at commit 46372e5.

@7c6f434c 7c6f434c merged commit c7c3396 into NixOS:master Oct 22, 2018
@f--t f--t deleted the fix/julia-update branch October 20, 2019 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@f--t @fpletz @GrahamcOfBorg @nalimilan @markuskowa @7c6f434c @eli-schwartz