You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
New package for infer, an open-source static analysis tool maintained by facebook.
12/16/18: This PR is NOT FINISHED yet. I'm putting it here to ask for some help with a couple issues I've run into trying to compile these tools.
Organization:
This PR currently contains three new packages:
infer-deps - the ocaml 4.06.1+flambda compiler and the opam requirements from the opam.lock file for infer
facebook-clang - a custom clang for infer (see facebook-clang-plugins) that facilitates plugins for clang-analyzer and clang-frontend
infer - the project itself, for analysis of Java, C, C++, and Objective C code
I did also add a source.nix for installing the precompiled binaries, but I assume that won't be liked upstream since we prefer binaries built on the nix build farm?
Problems:
As you can see from their install and getting started pages, they recommend running Infer inside a Docker container on Linux. There is a homebrew formula available for macOS, but on Linux they recommend using their precompiled binaries if not using Docker.
The problems I'm having with writing this package generally revolve around the custom clang:
macOS: clang build fails about 25% of the way through make with this (truncated) error message:
macOS dynamic lib issue
[ 26%] Linking CXX shared library ../../../../lib/clang/7.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/f8y5mdc0sjf5lx97j8xqnnrhsad6f3xw-libc++-5.0.2/lib/libc++.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/h7rxgvw954lmc3xvb0jxnjbdax92v107-libc++abi-5.0.2/lib/libc++abi.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/x7v2211y4864na7ggfiqjx1wwbq90gsc-Libsystem-osx-10.11.6/lib/libc.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/f8y5mdc0sjf5lx97j8xqnnrhsad6f3xw-libc++-5.0.2/lib/libc++.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/h7rxgvw954lmc3xvb0jxnjbdax92v107-libc++abi-5.0.2/lib/libc++abi.dylib
ld: warning: linking against a dylib which is not safe for use in application extensions: /nix/store/x7v2211y4864na7ggfiqjx1wwbq90gsc-Libsystem-osx-10.11.6/lib/libc.dylib
/nix/store/ykzmq1yyggy8c8gfvxqzrd292j1ld797-bash-4.4-p23/bin/bash: codesign: command not found
make[2]: *** [projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/build.make:731: lib/clang/7.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib] Error 127
make[2]: *** Deleting file 'lib/clang/7.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib'
make[1]: *** [CMakeFiles/Makefile2:25117: projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/l7nja9b7af0r292zilwb9rcy19mj5mq7-facebook-clang.drv' failed with exit code 2
I assume this might be the type of issue that could be fixed by fixDarwinDylibNames but I don't know enough about how that pkg works to use it here. I'm also wary of continuing the attempt on macOS before getting the Linux side fully working since I think I might run into similar errors later on.
linux: here, clang gets much further through compilation, down to the external tools. One of the tools required is compiler-rt, and building this tool fails here:
CMakeError.log from compiler-rt
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: /tmp/nix-build-facebook-clang.drv-0/llvm/build/./bin/clang
Build flags:
Id flags:
The output was:
1
impure path `/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o' used in link
clang-7.0: error: linker command failed with exit code 1 (use -v to see invocation)
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /tmp/nix-build-facebook-clang.drv-0/llvm/build/./bin/clang++
Build flags:
Id flags:
The output was:
1
impure path `/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o' used in link
clang-7.0: error: linker command failed with exit code 1 (use -v to see invocation)
Determining if the C compiler works failed with the following output:
Change Dir: /tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp
Run Build Command:"/nix/store/crsfyhfdllxvdrrkv8jhyk7i7ahyn124-gnumake-4.2.1/bin/make" "cmTC_dd491/fast"
make[3]: Entering directory '/tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp'
/nix/store/crsfyhfdllxvdrrkv8jhyk7i7ahyn124-gnumake-4.2.1/bin/make -f CMakeFiles/cmTC_dd491.dir/build.make CMakeFiles/cmTC_dd491.dir/build
make[4]: Entering directory '/tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_dd491.dir/testCCompiler.c.o
/tmp/nix-build-facebook-clang.drv-0/llvm/build/./bin/clang -o CMakeFiles/cmTC_dd491.dir/testCCompiler.c.o -c /tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_dd491
/nix/store/451ffnn773jrl8qmhjs71dqd6pkfx2rx-cmake-3.12.1/bin/cmake -E cmake_link_script CMakeFiles/cmTC_dd491.dir/link.txt --verbose=1
/tmp/nix-build-facebook-clang.drv-0/llvm/build/./bin/clang CMakeFiles/cmTC_dd491.dir/testCCompiler.c.o -o cmTC_dd491
impure path `/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../x86_64-linux-gnu/crt1.o' used in link
clang-7.0: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [CMakeFiles/cmTC_dd491.dir/build.make:87: cmTC_dd491] Error 1
make[4]: Leaving directory '/tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp'
make[3]: *** [Makefile:121: cmTC_dd491/fast] Error 2
make[3]: Leaving directory '/tmp/nix-build-facebook-clang.drv-0/llvm/build/tools/clang/runtime/compiler-rt-bins/CMakeFiles/CMakeTmp'
In the custom clang build script, this option is enabled with -DLLVM_BUILD_EXTERNAL_COMPILER_RT=On. I poked around in the nix expression for llvm_7 and the llvm.Packages.compiler-rt files, but I'm still not entirely sure how to fix an impure path being linked with nix. I expect the cmake config for compiler-rt is linking against the library path it shouldn't be, but adding gcc, gcc_multi, clang_5, glibc or llvm to buildInputs didn't seem to fix the issue. I tried not passing that option to facebook-clang, but if I do that then the infer build eventually fails with a similar issue:
Lines from infer/config.log when trying to build using custom clang WITHOUT compiler-rt
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
Those are the current issues with this project. Any advice/edits would be much appreciated. Also feel free tell me if I should reorganize the files - I don't really like that three new packages are being added instead of just one, but I don't know how to put these all in the same directory and make them "inferPackages" like how llvm does it.
TODO
get java analyzers working
get c analyzers (i.e. clang) working
at this point the checkPhase call for make test should work properly
condense infer-deps and facebook-clang into infer to one package to clean up
joining these will allow getting rid of the hacky postUnpack phases in infer-deps and facebook-clang to work with opam
add utop v2.1.0 for:
infer REPL
make config_tests, which runs when one of java or c analyzers are disabled
maybe add option to install precompiled bins?
Things done
Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
NixOS
macOS
other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Assured whether relevant documentation is up to date
impure path issue on linux - possible related issues:
explicitly pass glibc opencv: don't use default /lib64/ when searching for libs #7698 clang's cmake config does NOT accept -DGLIBC_LIBRARY option. in fact using [ gcc gcc_multi glibc ] as a linux-specific buildInput just prevents make from finding stdenv stuff
hope this callout is OK, but I'd like to @lovek323@raskin@dtzWill , the maintainers of the llvm package, to see if they have any ideas from their work writing the actual llvm pkg
Noting here - infer released 0.16.0 about a month ago. I'm leaving this PR up for now, but will try to get 0.16 working instead, since it looks like it may be easier to get working in package managers (upgrades to opam2, and clang 7.0.0, among other changes)
I guess you should use buildOcaml and buildDunePackage. opam-lock I saw several months ago in channel, search in history. It would be hard to implement build phase by opam (by several reasons: opam trying to download dependencies, but nix must to know about all the dependencies before the building start. And even if you have builded dependencies cache (opam's switch), it would be hard to reuse it, because ocamlc hardcode ablosute paths in ocaml bytecode, and opam do same in opam's switch and opam's root configuration. So if you would try this, use symlinks to mimic same paths at build phase).
@Pitometsu thanks for the advice. I'm trying an approach right now by just manually setting OPAMROOT to a subdir under $out but that doesn't seem like a good convention. I'll check out buildOcaml and buildDunePackage, especially now since infer 0.16+ has upgraded to opam 2.0.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation for this change
New package for infer, an open-source static analysis tool maintained by facebook.
12/16/18: This PR is NOT FINISHED yet. I'm putting it here to ask for some help with a couple issues I've run into trying to compile these tools.
Organization:
This PR currently contains three new packages:
infer-deps- the ocaml4.06.1+flambdacompiler and the opam requirements from theopam.lockfile for inferfacebook-clang- a custom clang for infer (see facebook-clang-plugins) that facilitates plugins forclang-analyzerandclang-frontendinfer- the project itself, for analysis of Java, C, C++, and Objective C codesource.nixfor installing the precompiled binaries, but I assume that won't be liked upstream since we prefer binaries built on the nix build farm?Problems:
As you can see from their install and getting started pages, they recommend running Infer inside a Docker container on Linux. There is a homebrew formula available for macOS, but on Linux they recommend using their precompiled binaries if not using Docker.
The problems I'm having with writing this package generally revolve around the custom clang:
makewith this (truncated) error message:macOS dynamic lib issue
I assume this might be the type of issue that could be fixed by
fixDarwinDylibNamesbut I don't know enough about how that pkg works to use it here. I'm also wary of continuing the attempt on macOS before getting the Linux side fully working since I think I might run into similar errors later on.compiler-rt, and building this tool fails here:CMakeError.log from
compiler-rtIn the custom clang build script, this option is enabled with
-DLLVM_BUILD_EXTERNAL_COMPILER_RT=On. I poked around in the nix expression forllvm_7and thellvm.Packages.compiler-rtfiles, but I'm still not entirely sure how to fix an impure path being linked with nix. I expect the cmake config for compiler-rt is linking against the library path it shouldn't be, but addinggcc,gcc_multi,clang_5,glibcorllvmtobuildInputsdidn't seem to fix the issue. I tried not passing that option to facebook-clang, but if I do that then theinferbuild eventually fails with a similar issue:Lines from
infer/config.logwhen trying to build using custom clang WITHOUT compiler-rtThose are the current issues with this project. Any advice/edits would be much appreciated. Also feel free tell me if I should reorganize the files - I don't really like that three new packages are being added instead of just one, but I don't know how to put these all in the same directory and make them "inferPackages" like how llvm does it.
TODO
checkPhasecall formake testshould work properlyinfer-depsandfacebook-clangintoinferto one package to clean uppostUnpackphases ininfer-depsandfacebook-clangto work with opammake config_tests, which runs when one of java or c analyzers are disabledThings done
sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)nix path-info -Sbefore and after)