Distributing shared libraries with the executable on Linux
gamedev, tech, tips
This is possibly something that’s obvious to every C or C++ developer out there, but I’ve spent ages trying to figure it out.
Background
The game I’m writing depends on libraries that deal with the graphics, user input, etc. These are usually distributed as shared/dynamic (*.so on Linux, *.dll on Windows).
What I wanted to do was to build an executable, put the libraries and the assets into the same directory, zip it and ship it. This is sort of the default way of distributing indie games. One could do installers, distro packages, etc., but every gamer is familiar with the download/unzip/play paradigm.
Problem is, when I compiled my project, the executable could not find the libraries even though they were right next to it (and I did pass the right path to the -L argument, which I assumed would be enough).
So this will set up a sample project that uses one such dynamic library, show how to compile it and discuss the various ways of having it load the dependencies.
If you only care about the quick answer: use the -rpath linker argument and pass the relative path to the directory with the shared libs.
Sample Roguelike project
Libtcod is a library designed to make writing roguelikes simpler. It works on Linux, ships only in the dynamic form and isn’t packaged in the major distros, so you can’t just yum install libtcod and be done with it.
Setup
Download libtcod and unpack it:
$ tar -xf libtcod-1.5.1-linux*.tar.gzCreate the directory for your roguelike project and copy the necessary stuff over:
$ mkdir -p roguelike/{include,lib}
$ cp libtcod-1.5.1/*.so roguelike/lib
$ cp -r libtcod-1.5.1/{include,terminal.png} roguelike/
$ cd roguelike
$ lsYou should see the include directory with a bunch of header files for libtcod, lib with all the dynamic libraries (libtcod.so and friends) and terminal.png which is the default and extremely ugly font libtcod uses.
The Code
Create a file called main.c in the roguelike directory with the basic initialisation code and a game loop:
#include <libtcod.h>
int main(int argv, char** argc) {
TCOD_key_t key;
TCOD_console_init_root(80, 50, "roguelike!", false, TCOD_RENDERER_SDL);
while(!TCOD_console_is_window_closed()) {
key = TCOD_console_check_for_keypress(TCOD_KEY_PRESSED);
if(key.vk == TCODK_ESCAPE) {
break;
}
TCOD_console_clear(0);
TCOD_console_put_char(0, 40, 25, '@', TCOD_BKGND_DEFAULT);
TCOD_console_flush();
}
return 0;
}All it does is create the game window, loop until it’s closed (either by pressing Esc or clicking the close window button) and display the @ glyph on the screen.
Building
Compile the sample:
$ gcc main.c -I include -L lib -ltcod -o roguelikeAnd run it:
$ ./roguelike
./roguelike: error while loading shared libraries: libtcod.so: cannot open shared object file: No such file or directoryLoading dynamic libraries
The libtcod.so library is dynamic (so stands for shared object). It isn’t included within the final executable. Instead, it will be loaded by the operating system when you run your app. But the system doesn’t know that it should look for it in the roguelike/lib/ directory.
There are three ways to fix this:
- Put the dynamic library where the OS does look
-
Use the
LD_LIBRARY_PATHenvironment variable - rpath, a.k.a. The Real Solution
1. Installing the libraries into the OS
The directories for dynamic libraries are specified in /etc/ld.so.conf.d/. You can see all the locations by running:
$ cat /etc/ld.so.conf.d/*.confMine lists a bunch of folders including /usr/lib/x86_64-linux-gnu/ which seems to contain most of the installed libraries.
So if you put your libtcod.so there, things should work:
$ sudo cp lib/libtcod.so /usr/lib/x86_64-linux-gnu/
$ ./roguelike
24 bits font.
key color : 0 0 0
character for ascii code 255 is colored
Hooray!
This is how the packages installed on your system work. The apps depend on dynamic libraries that are installed in one of the directories in ld.so.conf.d/.
2. LD_LIBRARY_PATH
If the LD_LIBRARY_PATH environment variable is present when you run the program (not when you compile it), the OS will look for the libraries there first. These override any other settings (such as the system-level paths from the first option and rpaths from the third).
As such, this puts the burden on the user of the program to specify the right library locations and gives them the power to screw things up (e.g. if they provide an incompatible library of the same name).
On the other hand, it also lets the savvy user to investigate and fix a dynamic library issue without messing up the system.
So to solve our problem (remember to sudo rm /usr/lib/x86_64-linux-gnu/libtcod.so if you followed the steps from the previous section), we set the environment variable to our lib directory:
$ ./roguelike
./roguelike: error while loading shared libraries: libtcod.so: cannot open shared object file: No such file or directory
$ LD_LIBRARY_PATH=lib ./roguelike
24 bits font.
key color : 0 0 0
character for ascii code 255 is coloredObviously, we don’t want our users to set LD_LIBRARY_PATH manually every time they run the program, but we can create a launcher or a simple wrapper shell script that does this for us.
This works but it isn’t ideal. The user has to run the wrapper instead of the executable, the OS runs two processes and we need to add one more step to our build pipeline. And if someone wants to package your game they have to undo all this work to comply with their distro’s guidelines.
Still, there are games that do this. The Linux port FTL: Faster Than Light for example.
But there is a better way!
3. rpath
The Linux binaries conform to ELF (Executable and Linkable Format). A part of this structure is the list of dynamic libraries the program needs to load. And more interestingly to us, they can also contain an additional list of library locations.
You can check the ELF header with the readelf tool:
$ readelf -d roguelike
Dynamic section at offset 0xe18 contains 25 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libtcod.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000c (INIT) 0x400700
0x000000000000000d (FINI) 0x4009f4
0x0000000000000019 (INIT_ARRAY) 0x600e00
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x600e08
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x400298
0x0000000000000005 (STRTAB) 0x400480
0x0000000000000006 (SYMTAB) 0x4002d0
0x000000000000000a (STRSZ) 327 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x601000
0x0000000000000002 (PLTRELSZ) 216 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x400628
0x0000000000000007 (RELA) 0x400610
0x0000000000000008 (RELASZ) 24 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x4005f0
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x4005c8
0x0000000000000000 (NULL) 0x0Turns out our roguelike depends on two dynamic libs: lidtcod and libc. The latter provides the C runtime and if it’s not on your system, how are you able to read this in the first place?
The custom library location section is called rpath and there are two ways of setting it:
-
Using the
LD_RUN_PATHenvironment variable when compiling the program:
$ LD_RUN_PATH=lib gcc main.c -I include -L lib -ltcod -o roguelike-
Passing the
-rpathargument to the linker:
$ gcc main.c -I include -L lib -Wl,-rpath lib -ltcod -o roguelike(note the -Wl, part which instructs GCC to pass the rest of the argument to the linker)
If you do readelf -d roguelike again, here’s what you’ll see:
Dynamic section at offset 0xe08 contains 26 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libtcod.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000f (RPATH) Library rpath: [lib]
0x000000000000000c (INIT) 0x4006c8
0x000000000000000d (FINI) 0x4009b4
0x0000000000000019 (INIT_ARRAY) 0x600df0
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x600df8
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x400298
0x0000000000000005 (STRTAB) 0x400468
0x0000000000000006 (SYMTAB) 0x4002d0
0x000000000000000a (STRSZ) 326 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x601000
0x0000000000000002 (PLTRELSZ) 192 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x400608
0x0000000000000007 (RELA) 0x4005f0
0x0000000000000008 (RELASZ) 24 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x4005d0
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x4005ae
0x0000000000000000 (NULL) 0x0Notice the Library rpath: [lib] section. It contains the path relative to the current directory.
You can make it relative to the directory the executable is in by using $ORIGIN:
$ LD_RUN_PATH='$ORIGIN/lib' gcc main.c -I include -L lib -ltcod -o roguelike(note the single quotes to prevent shell from trying to evaluate $ORIGIN)
Summary
Build your app with the LD_RUN_PATH environment variable. It will allow you to ship it along with the dynamic libraries it depends on. And if someone decides to package it, all they have to do is remove the lib directory and set the package dependencies right (provided the libs themselves are already packaged).
The same executable will work in both cases.
References
http://www.eyrie.org/~eagle/notes/rpath.html
This is the page that finally helped me understand the issue and find the solution. It goes into greater detail.
http://man7.org/linux/man-pages/man8/ld.so.8.html
The man page for the linker. Helped me to figure out how to pass the linker args via GCC. It also discussed LD_RUN_PATH and -rpath, but was too unintelligible for me to figure out that was the answer I was looking for.
