Static linking can be an attractive deployment method for native programs when targeting multiple, possibly older, Linux distributions. The goal is to produce an executable file which contains all the required third-party code, including the C run-time library. This executable is relocatable to any compatible¹ Linux distribution.

This gives more control over the versions of the compiler and of the library dependencies, although our executable will be larger and it could potentially have a larger memory footprint, due to it not being able to use shared libraries already loaded by other processes.

Dynamic linking

The dynamic linking approach usually involves compiling the application on the target system and packaging it using the target distribution’s native package management solution. For third-party dependencies, the shared libraries present in the system are used. The advantage of this is that one can rely on the target distributions’ package management system to perform the installation of our library dependencies and these shared libraries only need to be loaded once in memory if multiple running programs require them.

There are, however, some disadvantages. The target distribution will only provide specific versions of libraries. In the case of distributions with Long-Term-Support (LTS), such as Red Hat Enterprise Linux and its derivatives, the support cycle is longer than 10 years. During this time, system packages typically only receive bug fixes, not updates to newer versions. In this situation, One is locked into using old versions of libraries, unless they are bundled with the package.

When it comes to the compiler version to use, there is even less flexibility: it’s not always practical or possible to install additional compiler toolchains, and to use system C++ libraries, it’s necessary to use a version of the compiler which provides a specific ABI version. Modern C++ language standards may not be supported by the system compiler.

As an example, RHEL 5, released in 2007, will be supported until March 2017, with extended support until 2020. It comes with GCC 4.1.2 which only supports the C++98 (or C++03) standard. This version of GCC also does not have OpenMP support. Note that new versions of GCC are available in the Red Hat Developer Tools addon.

Limitations of static linking

One needs to keep in mind that there are some limitations to static linking, which can sometimes make this approach not viable:

• If the application contains multiple executables that depend on common libraries, each executable will include a copy of the library code, using more memory.
• There is no support for plugins loaded with dlopen. This system call does not seem to work in a statically linked executable.
• API/ABI changes at the system level could break the application.

Nonetheless, it represents a useful technique which in certain cases can greatly simplify deployment of native programs.

Static linking and containers

It is actually quite easy to set up a C++ project for building fully static, relocatable, executables. We will perform an experiment based on the following pieces of technology:

1. musl - an alternative C standard library that supports making fully statically linked executables. The resulting executable can be run on any other system running a recent (Linux kernel 2.6) distribution.

2. Alpine Linux - a lightweight GNU/Linux distribution based on musl. The default compiler toolchain of this distribtion is what we need to make the statically linked executable.

3. Docker - a toolbox for working with Linux containers. There are many uses for Docker and containers, and I won’t try to explain the technology here, but for the purposes of this experiment, Docker is used to run the compilation process in the specific environment required to statically link the executable.

I created a C++ test project that makes use of some C++11 and C++14 features - std::thread, generic lambdas and std::async:

#include <iostream>
#include <future>
#include <thread>

int main()
{
  // C++11 std::thread
  auto t1 = std::thread([] {
    std::cout << "Hello from thread!" << std::endl;
  });

  t1.join();

  const auto n = 1;

  // C++14 generic lambda
  auto f = [n] (auto i) {
    return n + i;
  };

  std::cout << "f(1) = " << f(1) << std::endl;

  // C++11 std::async
  auto result = std::async(f, 2);

  std::cout << "f(2) = " << result.get() << std::endl;

  return 0;
}

Compiling

We will be compiling the test code inside a Docker container. We will use the official Alpine Linux images as a starting point. On top of the base image, we only need to install the C++ development packages:

FROM alpine:3.4

RUN apk update && apk add --no-cache binutils cmake make libgcc musl-dev gcc g++

This minimal Docker image is available on Docker hub.

To compile the test project, we can use the following script:

#!/bin/sh

export WORKDIR=/opt/src
rm -rf build-musl

docker run --rm -it -v "${PWD}":"${WORKDIR}" radupopescu/musl-builder \
    sh -c "cd ${WORKDIR} && mkdir build-musl && cd build-musl && cmake -D CMAKE_EXE_LINKER_FLAGS=\"-static\" ../ && make"

The key point here is that the current directory on the host, the root of the test project, is mounted inside the container at /opt/src/.

If all goes well, the statically linked executable, musl_test_main, can be found in the build-musl/apps/ directory of the test project.

Testing

We can easily test the resulting executable by using the official CentOS Docker images. For example, we can run the executable on CentOS 5, 6 and 7:

docker run -it -v "${PWD}:/opt/src/" centos:5 sh -c "cd /opt/src/build-musl && pwd && ./apps/musl_test_main"
docker run -it -v "${PWD}:/opt/src/" centos:6 sh -c "cd /opt/src/build-musl && pwd && ./apps/musl_test_main"
docker run -it -v "${PWD}:/opt/src/" centos:7 sh -c "cd /opt/src/build-musl && pwd && ./apps/musl_test_main"

In each case, we expect the following output:

/opt/src/build-musl
musl_test version: 0.0.1
f(1) = 2
f(2) = 3

Closing remarks

This post provides a quick introduction to making fully statically linked executables using musl and Docker. It would be interesting to try the technique described here on a real-world C++ project with a complex code base and which uses third-party libraries.

Notes