Why are only some of these instances of C ++ templates exported to the shared library?
I have a C ++ dynamic library (on macOS) that has a templated function with some explicit instances that are exported to a public API. The client code only sees the template declaration; they have no idea what's going on inside it and rely on these instances to be available at link time.
For some reason, only a few of these explicit instances become visible in the dynamic link library.
Here's a simple example:
// libtest.cpp
#define VISIBLE __attribute__((visibility("default")))
template<typename T> T foobar(T arg) {
return arg;
}
template int VISIBLE foobar(int);
template int* VISIBLE foobar(int*);
I would expect both instances to be visible, but not a pointer one:
$ clang++ -dynamiclib -O2 -Wall -Wextra -std=c++1z -stdlib=libc++ -fvisibility=hidden -fPIC libtest.cpp -o libtest.dylib
$ nm -gU libtest.dylib | c++filt
0000000000000f90 T int foobar<int>(int)
This test program is not linked because the pointer is missing:
// client.cpp
template<typename T> T foobar(T); // assume this was in the library header
int main() {
foobar<int>(1);
foobar<int*>(nullptr);
return 0;
}
$ clang++ -O2 -Wall -Wextra -std=c++1z -stdlib=libc++ -L. -ltest client.cpp -o client
Undefined symbols for architecture x86_64:
"int* foobar<int*>(int*)", referenced from:
_main in client-e4fe7d.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
There seems to be a definite relationship between types and visibility. If you change the return type to void
, they will all be visible (even if the template arguments are still pointers or whatever). Especially strange this exports:
template auto VISIBLE foobar(int) -> int;
template auto VISIBLE foobar(int*) -> int*;
This is mistake? Why does explicit syntactic sugar change behavior?
It works if I change the definition of the template to be visible, but it doesn't seem perfect because only some of those instances need to be exported ... and I still want to understand why this is happening anyway.
I am using Apple LLVM version 8.0.0 (clang-800.0.42.1).
source to share
Your problem is reproducible on linux:
$ clang++ --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
$ clang++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden \
-fPIC libtest.cpp -o libtest.so
$ nm -C libtest.so | grep foobar
0000000000000620 W int foobar<int>(int)
0000000000000630 t int* foobar<int*>(int*)
Pointerless overloading is weakly global, but pointer overloading is local.
The reason for this is hidden clan diagnostics by the __attribute__
syntax extension, which is after all an invention of GCC. If we compile g ++ instead we get:
$ g++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden -fPIC libtest.cpp -o libtest.so
libtest.cpp:9:36: warning: ‘visibility’ attribute ignored on non-class types [-Wattributes]
template int * VISIBLE foobar(int *);
^
Note that g ++ only ignores the visibility attribute on pointer overloading, and, just like clang - and consistent with this warning - it emits code with:
$ nm -C libtest.so | grep foobar
0000000000000610 W int foobar<int>(int)
0000000000000620 t int* foobar<int*>(int*)
It is clear that clang does the same, but does not tell us why.
The difference between overloads that satisfy g ++ with one and dissatisfaction with the other is the difference between int
and int *
. Based on this, we expect to g++
be satisfied with the change:
template int VISIBLE foobar(int);
//template int * VISIBLE foobar(int *);
template float VISIBLE foobar(float);
And so it is:
$ g++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden -fPIC libtest.cpp -o libtest.so
$ nm -C libtest.so | grep foobar
0000000000000650 W float foobar<float>(float)
0000000000000640 W int foobar<int>(int)
And so does clang:
$ clang++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden -fPIC libtest.cpp -o libtest.so
$ nm -C libtest.so | grep foobar
0000000000000660 W float foobar<float>(float)
0000000000000650 W int foobar<int>(int)
Both of these will do what you want for T
non-pointer-type overloads , but not T
pointer-type overloads .
However, you are faced with this, it is not a ban on dynamically visible functions that return pointers, not pointers. He could not escape notification if he
visibility
was similarly broken. This is just a ban on form types:
D __attribute__((visibility("...")))
where D
is a pointer or reference type, as opposed to form types:
E __attribute__((visibility("..."))) *
or
E __attribute__((visibility("..."))) &
where is E
not a pointer or a reference type. The difference between them:
- A (pointer or link that has visibility ...) to type
D
and
- A (pointer or reference to a type
E
) that has visibility ...
Cm:
$ cat demo.cpp
int xx ;
int __attribute__((visibility("default"))) * pvxx; // OK
int * __attribute__((visibility("default"))) vpxx; // Not OK
int __attribute__((visibility("default"))) & rvxx = xx; // OK,
int & __attribute__((visibility("default"))) vrxx = xx; // Not OK
$ g++ -shared -Wall -Wextra -std=c++1z -fvisibility=hidden -o libdemo.so demo.cpp
demo.cpp:3:46: warning: ‘visibility’ attribute ignored on non-class types [-Wattributes]
int * __attribute__((visibility("default"))) vpxx; // Not OK
^
demo.cpp:5:46: warning: ‘visibility’ attribute ignored on non-class types [-Wattributes]
int & __attribute__((visibility("default"))) vrxx = xx; // Not OK
^
$ nm -C libdemo.so | grep xx
0000000000201030 B pvxx
0000000000000620 R rvxx
0000000000201038 b vpxx
0000000000000628 r vrxx
0000000000201028 b xx
OK declarations become global symbols; Out of order, become local and only the first ones are dynamically visible:
nm -CD libdemo.so | grep xx
0000000000201030 B pvxx
0000000000000620 R rvxx
It is reasonable. We cannot expect the compiler to attribute globally dynamic visibility to a pointer or reference that may point to or refer to something that does not have global or dynamic visibility.
This sane behavior only seems to violate your purpose, because - as you now see:
template int VISIBLE foobar(int);
template int* VISIBLE foobar(int*);
does not mean what you thought. You considered that for this type U
,
template U VISIBLE foobar(U);
declares a template instance function that has default
visibility, accepting a type argument, U
and returning the same. In fact, it declares a template instance function that takes a type argument U
and returns a type:
U __attribute__((visibility("default")))
which is allowed for U
= int
but not allowed for U
= int *
.
To express your intent that testers template<typename T> T foobar(T arg)
should be dynamically visible functions, qualify the type of the template function with a visibility attribute. Per GCC documentation __attribute__
syntax - which admittedly doesn't say anything about templates - you have to make the function qualification attribute in a declaration other than its definition. So, complying with this you would revisit your code, for example:
// libtest.cpp
#define VISIBLE __attribute__((visibility("default")))
template<typename T> T foobar(T arg) VISIBLE;
template<typename T> T foobar(T arg) {
return arg;
}
template int foobar(int);
template int* foobar(int*);
g ++ no longer has any problems:
$ g++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden -fPIC libtest.cpp -o libtest.so
$ nm -CD libtest.so | grep foobar
0000000000000640 W int foobar<int>(int)
0000000000000650 W int* foobar<int*>(int*)
and both overloads are dynamically visible. The same goes for clang:
$ clang++ -shared -O2 -Wall -Wextra -std=c++1z -fvisibility=hidden -fPIC libtest.cpp -o libtest.so
$ nm -CD libtest.so | grep foobar
0000000000000650 W int foobar<int>(int)
0000000000000660 W int* foobar<int*>(int*)
If you're lucky you will have the same result with clang on Mac OS
source to share