Creating portable binaries on Linux

April 29, 2010

Distributing applications on Linux is hard. Sure, with modern package management, installing software is easy. But if you are distributing an application, you probably need one Windows version, plus umpteen different versions for Linux. In this article, we'll create a dummy application that targets the following operating systems, which are commonly used in business environments:

Windows Server 2003
Windows XP
Red Hat Enterprise Linux 3
Red Hat Enterprise Linux 4
Ubuntu 6.06.2
Ubuntu 8.04
Ubuntu 9.10

As evidence that the problem is hard, try downloading Firefox. It fails to start on many of the above platforms, due to missing libraries.

The sample application

The sample application is called plookup (download source and all binaries). It runs from the command line and takes a hostname, looks it up and prints out the IP address. Ignoring the security flaws, it has several monkey wrenches thrown in that make it hard to port to different versions of linux:

It uses C++, which causes headaches when dynamically linking
It uses socket functions, which cause migraines when statically linking

Will distributing source code solve the problem?

In theory, distributing source code seems to be an easy way to get around the problem, assuming your end user 1) has administrative access, 2) can install a compiler 2) knows how to run a configure script, 3) has the technical knowledge to interpret the output of a configure script and download the appropriate dependencies, 4) has technical expertise to resolve conflicts in library versions.

In other words, it's a completely unreasonable solution for software written for normal human beings.

Building on Windows

With the appropriate build flags, you can produce software on Windows 7 that will run on all versions of windows since Windows 95. By defining WINVER and some other macros, you'll be warned at compile time if you're using a feature that will break your program on earlier versions.

CL /EHsc /Feplookup.exe /DWINVER=0x0400 /D_WIN32 /D_WIN32_WINDOWS=0 /D_WIN32_IE=0x0400 wsock32.lib *.cpp

We're done with Windows. On to Linux!

Static linking on Linux FAIL

Static linking has gained an undeserved reputation for being portable. But see what happens when you try to create a statically linked version of plookup:

steve@ubuntu:~/plookup$ g++ -static -static-libgcc -o plookup main.cpp
/tmp/ccMhUffR.o: In function `hostlookup(std::basic_string, std::allocator > const&, std::basic_string, std::allocator >&)':
main.cpp:(.text+0x15): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

It defeats the purpose. The warning isn't kidding, either. Your app might run fine for months, then all it takes is one update and it will crash. Even if you didn't use any socket functions, you might have other problems.

Note: If you statically link using the GNU compiler, then according to the L-GPL you have to also distribute your object files so the end-user can possibly re-link them to another version of the C libraries that they could have modified. Because your customers love hacking on strcat in their spare time.

Dynamic Linking

I built and tested the sample application on many different platforms. This table summarizes the results of the experiment.

	Build on Red Hat EL 3	Build on Red Hat EL 4	Build on Ubuntu 6.06.2 g++3.3	Build on Ubuntu 6.06.2 g++4.0	Build on Ubuntu 8.04	Build on Ubuntu 9.10
Runs on Red Hat EL 3?	Yes	No	Yes	No	No	No
Runs on Red Hat EL 4?	No	Yes	No	Yes	Yes	Yes
Runs on Ubuntu 6.06.2?	No	Yes	No	Yes	Yes	Yes
Runs on Ubuntu 8.04?	No	Yes	No	Yes	Yes	Yes
Runs on Ubuntu 9.10?	No	Yes	No	Yes	Yes	Yes

Analysis

There are two classes of systems. Those with libstdc++5.0 (Red Hat EL 3), and those with libstdc++6.0 (All others). If you build on a system with one version, your application will only run on systems which have that library.

Ubuntu 6.06.2 is a special case. It does not have libstdc++5.0 by default, but you can add it by installing and building with g++3.3. That makes Ubuntu 6.06.2 a great build environment for portable binaries.

Linux Standards Base

Linux Standards Base (LSB) has an excellent utility that will predict which versions of Linux your application won't run on. But I had some problems using the rest of their toolchain:

It's not clear what you have to download to use their toolchain.
It's not clear how to use their toolchain (Hint: It's a wrapper around gcc)
Their toolchain doesn't work with recent versions of gcc
Once I finally found a distribution that would work with the lsb tools, it did not produce a portable application.

I spent six hours on LSB one weekend and gave up.

Summary

To distribute binaries on Linux,

Dynamically link the standard libraries
Produce one binary using g++3.3 for older systems
Produce another binary using g++4.0 for newer systems

Hopefully in time the situation will improve. According to the FAQ on libstdc++ 3:

The GNU C/C++/FORTRAN/ compiler (gcc, g++, etc) is widely considered to be one of the leading compilers in the world. Its development has recently been taken over by the GCC team. All of the rapid development and near-legendary portability that are the hallmarks of an open-source project are being applied to libstdc++.