Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

binder and pybind11 packages #25065

Closed
wants to merge 2 commits into from
Closed

binder and pybind11 packages #25065

wants to merge 2 commits into from

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Apr 20, 2017

Motivation for this change
Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@mention-bot
Copy link

@FRidh, thanks for your PR! By analyzing the history of the files in this pull request, we identified @FRidh to be a potential reviewer.

@FRidh
Copy link
Member Author

FRidh commented Apr 20, 2017

@copumpkin if I'm correct you're the llvm/clang expert here, so maybe you have a solution.

I'm packaging binder, and the following are the installation instructions:

# Create build dir
mkdir $HOME/prefix && cd $HOME/prefix

# Clone  LLVM
git clone http://llvm.org/git/llvm.git llvm && cd llvm
git reset --hard origin/release_38

# Clone Clang
cd $HOME/prefix/llvm/tools
git clone http://llvm.org/git/clang.git clang
cd clang && git reset --hard origin/release_38


# Clone Clang extra tools
cd $HOME/prefix/llvm/tools/clang/tools
git clone http://llvm.org/git/clang-tools-extra.git extra

# Create symlink pointing to binder/src dir
ln -s $HOME/binder/binder $HOME/prefix/llvm/tools/clang/tools/extra/binder


# Create ``llvm/tools/clang/tools/extra/CMakeLists.txt`` file with content: ``add_subdirectory(binder)``
echo 'add_subdirectory(binder)' > $HOME/prefix/llvm/tools/clang/tools/extra/CMakeLists.txt

# Build Binder
mkdir $HOME/prefix/build && cd $HOME/prefix/build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_EH=1 -DLLVM_ENABLE_RTTI=ON .. && ninja

# At this point, if all above steps is succeseful, binder should be at
# $HOME/prefix/build/bin/binder

Nixpkgs provides clang and llvm, and clang is also built with the extra tools if I am correct.
Here, though, they use the extra tools repo to build binder. Could you suggest a better way to build binder than to build clang again?

If I just run cmake like I have in the current expression I get the following error:

$ nix-build -A python.pkgs.binder 
these derivations will be built:
  /nix/store/417h1a5zkmjg26bxjnkvlkilj4rkh7g5-python2.7-binder-2017-03-27.drv
building path(s) ‘/nix/store/2xa90i07yvzf3jrzakgh9iw20r95bgwj-python2.7-binder-2017-03-27’
unpacking sources
unpacking source archive /nix/store/n2zfx0pg979sjqfg6nwv8nyim36mfsdc-binder
source root is binder/binder
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file binder/binder/util.hpp
patching sources
configuring
fixing cmake files...
cmake flags: -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/2xa90i07yvzf3jrzakgh9iw20r95bgwj-python2.7-binder-2017-03-27/include -DCMAKE_INSTALL_LIBDIR=/nix/store/2xa90i07yvzf3jrzakgh9iw20r95bgwj-python2.7-binder-2017-03-27/lib -DCMAKE_INSTALL_NAME_DIR=/nix/store/2xa90i07yvzf3jrzakgh9iw20r95bgwj-python2.7-binder-2017-03-27/lib -DCMAKE_INSTALL_PREFIX=/nix/store/2xa90i07yvzf3jrzakgh9iw20r95bgwj-python2.7-binder-2017-03-27  
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /nix/store/dqfikdyxdrykwcbg1wp6nd725y0c7va4-gcc-wrapper-5.4.0/bin/gcc
-- Check for working C compiler: /nix/store/dqfikdyxdrykwcbg1wp6nd725y0c7va4-gcc-wrapper-5.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /nix/store/dqfikdyxdrykwcbg1wp6nd725y0c7va4-gcc-wrapper-5.4.0/bin/g++
-- Check for working CXX compiler: /nix/store/dqfikdyxdrykwcbg1wp6nd725y0c7va4-gcc-wrapper-5.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:8 (add_clang_executable):
  Unknown CMake command "add_clang_executable".


CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.7)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
See also "/tmp/nix-build-python2.7-binder-2017-03-27.drv-0/binder/binder/build/CMakeFiles/CMakeOutput.log".
builder for ‘/nix/store/417h1a5zkmjg26bxjnkvlkilj4rkh7g5-python2.7-binder-2017-03-27.drv’ failed with exit code 1
error: build of ‘/nix/store/417h1a5zkmjg26bxjnkvlkilj4rkh7g5-python2.7-binder-2017-03-27.drv’ failed

@copumpkin
Copy link
Member

I wouldn't call myself an expert, but I've dealt with it a fair amount. Judging from their instructions, they might actually want the source to clang as opposed to a precompiled library/binary? It seems odd, but you could probably just make sure to drop the source files (i.e., just refer to .src) in the places the binder build expects to find them. I don't know if they then just use their build system to tell clang to build from scratch (which would suck) or if they just need to pull out a file or two, but I'd probably start that way, see what it expects to do, and then perhaps try patching up the build to use a prebuilt clang if we can.

@FRidh
Copy link
Member Author

FRidh commented Apr 21, 2017 •
edited

Apparently building tooling for Clang is done like this according to the docs
https://clang.llvm.org/docs/LibASTMatchersTutorial.html

We need a Nix ninja :)

I think the easiest solution here for me is to override clang-tools-extra_src to use the binder source with the additional cmake rule. If more programs start doing this we might want to keep the build directory and use that as input.

@FRidh
Copy link
Member Author

FRidh commented Apr 21, 2017

I got binder to build, and it seems to be working. The next issue I have is having it find headers. One can pass -I for includes and such.

Ideally, I would like it to support NIX_CFLAGS_COMPILE, making it more useful in a Nix expression. Therefore, I used wrapCC. Unfortunately, that seems to unset -fexceptions again.

@FRidh
Copy link
Member Author

FRidh commented Aug 13, 2017

Closing because I'm not working on this anymore.

@FRidh FRidh closed this Aug 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
2.status: work-in-progress 6.topic: python 8.has: package (new)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@FRidh @mention-bot @copumpkin