I ran into linker errors similar to the screenshot below in November 2021. abi_linker_error After some close inspection, I realized this was a result of mixing 2 C++ standard libraries, namely libc++ and libstdc++. I was unkowningly 😱 compiling TVM with libc++’s headers while linking it against LLVM which was compiled with libstdc++’s headers and linked against libstdc++.

A Simple Example

Here is a simple example illustrating what’s happening.

Source Files

Given 2 C++ souce files:

#include <cstdio>
#include <string>

void print_all(const std::string& str) {
  for (auto idx = 0uz; auto i : str) {
    printf("%lu\t%c\n", ++idx, i);
  }
}

Side note:

The uz suffix was introduced to C++23 by P0330, and the init-statement in range-based for was introduced to C++20 by P0614.

main.cpp

#include <string>

void print_all(const std::string&);

int main() { print_all("Hello World!"); }

Object Files

We compile print_all.cpp with headers of libc++ and libstdc++ respectively:

clang++ -std=c++2b -O3 -c print_all.cpp -stdlib=libc++ -o print_all_libc++.o
clang++ -std=c++2b -O3 -c print_all.cpp -stdlib=libstdc++ -o print_all_libstdc++.o

And then compile main.cpp with libc++’s header files:

clang++ -std=c++2b -O3 -c main.cpp -stdlib=libc++ -o main-libc++.o

Symbol Tables

We can inspect symbols of each object file by running nm --demangle *.o.

main-libc++.o:
0000000000000000 r GCC_except_table0
                 U _Unwind_Resume
                 U print_all(std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> > const&)
                 U operator delete(void*)
                 U __gxx_personality_v0
0000000000000000 T main

print_all_libc++.o:
0000000000000000 T print_all(std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> > const&)
                 U printf

print_all_libstdc++.o:
0000000000000000 T print_all(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
                 U __gxx_personality_v0
                 U printf

The print_all of print_all_libc++.o and print_all_libstdc++.o have different function signatures hence different symbol names! We will get to those different std::basic_string<char, char_traits<char>, allocator<char> > in a moment.

Linking

Let’s first link main-libc++ and print_all_libc++.o together.

clang++ -stdlib=libc++ -fuse-ld=lld main-libc++.o print_all_libc++.o

Voila, the resulting executable works as intended.

Next, let’s see what would happen if we linked main-libc++ and print_all_libstdc++.o together.

❯ clang++ -stdlib=libc++ -fuse-ld=lld main-libc++.o print_all_libstdc++.o
ld.lld: error: undefined symbol: print_all(std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> > const&)
>>> referenced by main.cpp
>>>               main-libc++.o:(main)
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

It failed expectedly as the linker cannot find print_all(std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char> > const&) in print_all_libstdc++.o (see the nm output in the previous section). This is exactly the problem we came across!

Inline Namespace

It turns out that there is this thing called inline namespace which is sort of transparent to library users but not to compilers and linkers in that it is soldered into symbol names.

#include <cstdio>
#include <string>

namespace not_std {
inline namespace v1 {

void print_all(const std::string& str) {
  for (auto idx = 0uz; auto i : str) {
    printf("%lu\t%c\n", ++idx, i);
  }
}

} // namespace v1
} // namespace not_std

int main() {
  not_std::print_all("Hello World!");     // works
  not_std::v1::print_all("Hello World!"); // also works
}

std::string

Hence, when we write std::string, it is not what we think it is naively (type alias of std::basic_string<char>). Instead, for libc++, it is

std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>>

or

std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>

or for libstdc++

std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>

Use cases

Inline namespace can be used for library versioning. In the case of libc++, the inline namespace is used for ABI versioning and is used for nearly every name, contrary to libstdc++, which seems to use it only for differentiating SSO (small string optimization) string from the pre-C++11 one.

More on libc++ ABI versioning

Notice in the Symbol Table section, there are __2 following std:: in all libc++ symbols. It’s __1 by default but I configured and built libc++ with LIBCXX_ABI_VERSION set to 2 just for fun. The differences from ABI version 1 can be found here. Version 2 includes alternate string layout, which is also used for Apple arm64.

Conclusion

When faced with linker errors, we must be as patient as we can, as Yoda put it, “patience you must have, my young padawan”. Also, by no means do you ever want to mingle libc++ and libstdc++.