rabbit hole of rpath and origin

a little background on dynamic linking

the job of dynamic linker and loader(ld.so) is to resolve the executable’s dependencies on shared libraries and to load the required ones at run-time. readelf .bin to find the NEDDED headers and search the DSO with a matching SONAME

  1. DT_RPATH in the ELF binary, unless DT_RUNPATH set.
  2. LD_LIBRARY_PATH entries, unless setuid/setgid
  3. DT_RUNPATH in ELF binary
  4. /etc/ld.so.cache entries, unless -z nodeflib given at link time
  5. /lib, /usr/lib unless -z nodeflib
  6. Done, “not found”.

however, exceptions(small print…)

  • step1 & 2 if DT_RPATH will ignored is DT_RUNPATH is set. then LD_LIBRARY_PATH will search first
  • step 3 LD_LIBRARY can be override by calling dynamic linker with option –library-path
  • step6 if executable is linked with ```-z`` so #6 is skipped.
  • if using origin flag, remember to pass -z origin to mark the obejct as requiring origin processing
  • LD_PRELOAD happen before all those.
  • LD_LIBRARY_PATH inherited by all processes generated by parent and it is considered evil

Encoding rpath/runpath in binaray

ELF binaries can contains supplemental path for DSO, those path must encoded at compile time and it is searched before system default path(which could overriden by LD_LIBRARY_PATH). so the problem of find a DSO is done by the dev build the app rather than the user that deploy the app. there are two way to do rpath

  1. set LD_RUN_PATH to the search path you want to encode
  2. for rpath: add linker flag -Wl, -rpath, /user/local/lib similar to -L, you could add multiple time, order doestn’t matters AND --enable-new-dtagsfor runpath
  3. using chrpath to change already-present RPATH
$ gcc program.c -lm -o program '-Wl,-rpath,$ORIGIN/lib'
$ ldd program | grep libm
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 // local lib can't found
$ mkdir lib && cp /lib/x86_64-linux-gnu/libm.so.6
$ ldd program | grep libm
        libm.so.6 => /home/longwei/lib/libm.so.6 //use local lib instead

after compiler, you need to change elf binaray in order to change the rpath install_name_tool or patchelf

tl;dr

  • rpath is NOT runpath, runpath is introduced around 2000 when rpath cause mass issue used by libtool via overriding LD_LIBRARY_PATH
  • RUNPATH happens after LD_LIBRARY_PATH, and set RUNPATH will cancel RPATH.
  • chrpath -d foo.so to suppress rpth => runpath.
  • runpath is not used for finding indirect library dependecies.

$ORIGIN one of the common issue with explicit set the runpath is the use of absolute path. it is not a problem for fix system libraries, but you can’t gurentee the lib will be searched on the same exactly path. this allow myapp to install anyway as long as they keep in same relative path. -Wl, -z origin

@rpath on mac may not what you think on mac @rpath (under Mac OS X’s dyld) looks similar to DT_RPATH (or just rpath), but behaves a little differently.

When you dynamically link to a library on Mac OS X, the linker stores the “install name” of the library inside your executable. The install name is something that comes from the dylib you’re linking against, and by default it is the absolute path of the linked file. You can change the install name by modifying the dylib after linking. install_name_tool -id @rpath/my.dylib my.dylib After your application has been linked, you can change what the application thinks the install name is for one of its dependent libraries install_name_tool -change old.dylib @rpath/new.dylib my_application the direct translation of rpath on mac is -Wl,-rpath,@executable_path

This rpath does not do anything by default. To make it take effect, the install name for the shared library has to start with @rpath/ — and the dynamic linker will then substitute each of the possible values for @rpath in order. This means that you’ll typically change the install name of the dylib (if it’s a dylib you built yourself) or change the install name inside the application.

What’s the best practice?

when shipping a app, linking to core libs like(libc | libm) is ok as long as the version is good enough. it’s good to enable user to tweak/adjusting library search path.

so there is basically two way, RPATH and LD_LIBRARY_PATH LD_LIBRARY_PATH is very simple, run a shell script to bootstrap the environment and then call the executable. it only takes an absolute path and it will leave RPATH untouched. the catch is LD_LIBRARY_PATH can break other things in very subtle way…

another way is is tell compiler the locations at link time by RPATH, it is stored in the elf executable, in the dynamic section. It can be a relative path.

here is the catch, RUNPATH is recommended over RPATH, and RPATH is deprecated, but RUNPATH is currently not supported by all systems…

  1. you could using relative path of your current path, but user may not invoke the execuatable in pwd
  2. then $ORIGIN variable to define where is the executable
  3. but that’s not all, because $ORIGIN will be escape as RIGIN…
unix:{
    # suppress the default RPATH if you wish
    QMAKE_LFLAGS_RPATH=
    # add your own with quoting gyrations to make sure $ORIGIN gets to the command line unexpanded
    QMAKE_LFLAGS += "-Wl,-rpath,\'\$$ORIGIN\../mylibs\'"
    #There is QMAKE_RPATHDIR, But it won't work with a path containing $ORIGIN
    QMAKE_RPATHDIR += $${QUOTE}$${DOLLAR}$${DOLLAR}ORIGIN$${QUOTE}
}

but there is more, you may also need to pass in -Wl,-z,origin because gcc think pass -rpath=$ORIGIN is not good enough hassle. Here is a example when

readelf -d foo | grep ORIGIN
0x000000000000000f (RPATH)    Library rpath: [$ORIGIN:QT_INSTALL_GCC:QT_INSTALL_GCC/lib]
0x000000006ffffffb (FLAGS_1)  Flags: ORIGIN

Commands for Shared Library Management & Debugging Problem

TODO

System default path

command

  1. ldconfig: updates the necessary links for the run time binding
  2. ldd: tekk what’s lib a given program needs to run
  3. ltrace: A library call tracer
  4. ld.so/ld-linux.so: dynamic linker/loader

important file

  1. /lib/ld-linux.so.* : Execution time linker/loader.
  2. /etc/ld.so.conf : File containing a list of colon, space, tab, newline, or comma separated directories in which to search for libraries.
  3. /etc/ld.so.cache : File containing an ordered list of libraries found in the directories specified in /etc/ld.so.conf. This file is not in human readable format, and is not intended to be edited. This file is created by ldconfig command.
  4. lib*.so.version : Shared libraries stores in /lib, /usr/lib, /usr/lib64, /lib64, /usr/local/lib directories.

supposed I installed a new library in~/lib then ldconfig -l ~/lib/foo.so to manually linking this DSO.

another way is to create a /etc/ld.so.conf/foo.conf (name don’t matters). then ldconfig to update the cache.

Written on February 11, 2016