Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glscopeclient looks in /usr/bin for plugins when installed system-wide and tries to dlopen every binary on the system #393

Closed
hansfbaier opened this issue Nov 27, 2021 · 11 comments
Labels
bug Something isn't working
Milestone

Comments

@hansfbaier
Copy link

$ glscopeclient
No platforms found, disabling OpenCL
: CommandLine Error: Option 'strip-debug' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
@hansfbaier
Copy link
Author

git commit 1f47d35

@hansfbaier
Copy link
Author

Backtrace:

#0  __GI_exit (status=1) at exit.c:138
#1  0x00007fffeea0a0be in llvm::report_fatal_error(llvm::Twine const&, bool) () at /usr/lib/x86_64-linux-gnu/libLLVM-6.0.so.1
#2  0x00007fffeea0a1ee in  () at /usr/lib/x86_64-linux-gnu/libLLVM-6.0.so.1
#3  0x00007fffee9fa5b1 in  () at /usr/lib/x86_64-linux-gnu/libLLVM-6.0.so.1
#4  0x00007fffee9fa77f in llvm::cl::Option::addArgument() () at /usr/lib/x86_64-linux-gnu/libLLVM-6.0.so.1
#5  0x00007ffff1fad796 in  () at /usr/bin/llvm-objcopy-6.0
#6  0x00007ffff7fe0b8a in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd868, env=env@entry=0x7fffffffd878) at dl-init.c:72
#7  0x00007ffff7fe0c91 in call_init (env=0x7fffffffd878, argv=0x7fffffffd868, argc=1, l=<optimized out>) at dl-init.c:30
#8  _dl_init (main_map=0x555555960ec0, argc=1, argv=0x7fffffffd868, env=0x7fffffffd878) at dl-init.c:119
#9  0x00007ffff670c915 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:182
#10 0x00007ffff7fe50bf in dl_open_worker (a=a@entry=0x7fffffffcb10) at dl-open.c:758
#11 0x00007ffff670c8b8 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#12 0x00007ffff7fe45fa in _dl_open (file=0x55555595fff0 "/usr/bin/llvm-objcopy-6.0", mode=-2147483646, caller_dlopen=<optimized out>, nsid=-2, argc=1, argv=0x7fffffffd868, env=0x7fffffffd878) at dl-open.c:837
#13 0x00007ffff556234c in dlopen_doit (a=a@entry=0x7fffffffcd30) at dlopen.c:66
#14 0x00007ffff670c8b8 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffccd0, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:208
#15 0x00007ffff670c983 in __GI__dl_catch_error (objname=0x555555824da0, errstring=0x555555824da8, mallocedp=0x555555824d98, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
#16 0x00007ffff5562b59 in _dlerror_run (operate=operate@entry=0x7ffff55622f0 <dlopen_doit>, args=args@entry=0x7fffffffcd30) at dlerror.c:170
#17 0x00007ffff55623da in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#18 0x00007ffff6f7c1ee in InitializePlugins() () at /home/user/src/scopehal-apps/lib/scopehal/scopehal.cpp:455
#19 0x000055555574f820 in main(int, char**) (argc=1, argv=0x7fffffffd868) at /home/user/src/scopehal-apps/src/glscopeclient/main.cpp:231

@azonenberg
Copy link
Collaborator

Very interesting!

What exact build config did you use - debug, release, sanitize? Have you tried compiling with gcc instead of LLVM? This sounds like it might be compiler specific.

@hansfbaier
Copy link
Author

I used:
cmake ../ -DCMAKE_BUILD_TYPE=DEBUG -DCMAKE_INSTALL_PREFIX=/usr
I don't have a choice of compiler here.
I think the LLVM issues come from me having AMDGPU-PRO drivers and OpenCL installed which seems to rely on LLVM.

@hansfbaier
Copy link
Author

hansfbaier commented Dec 18, 2021

A verbose make show it uses /usr/bin/c++, which is symlinked to /etc/alternatives/c++ which is symlinked to /usr/bin/g++
and that should be the GNU compiler.

@hansfbaier
Copy link
Author

cmake ../ -DCMAKE_BUILD_TYPE=DEBUG -DCMAKE_INSTALL_PREFIX=/usr
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'gtkmm-3.0'
-- Found gtkmm-3.0, version 3.24.2
-- Checking for module 'sigc++-2.0'
-- Found sigc++-2.0, version 2.10.2
-- Checking for module 'ffts'
-- Variable 'exec_prefix' not defined in '/usr/local/lib/pkgconfig/ffts.pc'
-- Found libffts: /usr/local/include/ffts, /usr/local/lib/libffts.so
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: /usr/lib/x86_64-linux-gnu/libOpenCL.so (found version "2.2")
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Checking for one of the modules 'clFFT>=2.0'
-- Found YAML: /usr/lib/x86_64-linux-gnu/libyaml-cpp.so
-- Found components for YAML
-- YAML_ROOT_DIR = /usr
-- YAML_INCLUDES = /usr/include
-- YAML_LIBRARIES = /usr/lib/x86_64-linux-gnu/libyaml-cpp.so
-- Found GLEW: /usr/include (found version "2.1.0")
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Found components for YAML
-- YAML_ROOT_DIR = /usr
-- YAML_INCLUDES = /usr/include
-- YAML_LIBRARIES = /usr/lib/x86_64-linux-gnu/libyaml-cpp.so
Skipping MSI package build; define WIXPATH to enable
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/src/scopehal-apps/build

@hansfbaier
Copy link
Author

it looks like the LLVM error comes from
#12 0x00007ffff7fe45fa in _dl_open (file=0x555555962040 "/usr/bin/llvm-objcopy-6.0", mode=-2147483646, caller_dlopen=, nsid=-2, argc=1, argv=0x7fffffffd6c8, env=0x7fffffffd6d8) at dl-open.c:837
837 dl-open.c: No such file or directory.

@hansfbaier
Copy link
Author

hansfbaier commented Dec 18, 2021

which is loaded from here:

#18 0x00007ffff6f7c1ee in InitializePlugins () at /home/user/src/scopehal-apps/lib/scopehal/scopehal.cpp:455
455				void* hlib = dlopen(fname.c_str(), RTLD_NOW);
(gdb) print fname
$2 = "/usr/bin/llvm-objcopy-6.0"

I wonder where that comes from.

@hansfbaier
Copy link
Author

hansfbaier commented Dec 18, 2021

OH! It seems to use /usr/bin as plugin search dir and then
open all the binaries there with dlopen.
That won't go well for sure.
I put in a print loop to list all plugin dirs and got:

DIR: /usr/lib/scopehal/plugins/
DIR: /usr/local/lib/scopehal/plugins/
DIR: /usr/bin
DIR: /home/jack/.scopehal/plugins

@hansfbaier
Copy link
Author

hansfbaier commented Dec 18, 2021

Got it!
I comented out the following lines at scopehal.cpp:429, and now glscopeclient starts!

	//current binary dir
	char selfPath[1024] = "";
	ssize_t readlinkReturn = readlink("/proc/self/exe", selfPath, (sizeof(selfPath) - 1) );
	if ( readlinkReturn > 0)
		search_dirs.push_back(dirname(selfPath));

That part seems to add /usr/bin (that's where glscopeclient is installed) to the plugin search path and load all executables there with dlopen

@azonenberg azonenberg added the bug Something isn't working label Dec 18, 2021
@azonenberg
Copy link
Collaborator

Yeah ok, it shouldn't be doing that. The trick is that we do want to use the build directory as the search path during development.

I guess a quick hack would be to only use the executable directory if it's not under /usr?

@azonenberg azonenberg changed the title starting glscopeclient fails (Option 'strip-debug' registered more than once) glscopeclient looks in /usr/bin for plugins when installed system-wide and tried to dlopen every binary on the system Dec 18, 2021
@azonenberg azonenberg changed the title glscopeclient looks in /usr/bin for plugins when installed system-wide and tried to dlopen every binary on the system glscopeclient looks in /usr/bin for plugins when installed system-wide and tries to dlopen every binary on the system Dec 18, 2021
@azonenberg azonenberg added this to the v0.1 milestone Dec 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants