How to include a shared C library in a Python package
I have a project depending on a shared library. To get it right from the start: The shared library is a pure C library, not a Python library. For simplicity reasons, I created a small demo project called pkgtest, which I will discuss.
So here's the thing to do: Run the Makefile to compile the library and place the compiled shared library (called libhello.so
here) where it can be accessed from within the Python package.
So far, I have assumed to run the makefile as a preset, copy the file libhello.so
to the packages directory, and add it to the package_data
script install parameter . Once installed, the shared library is then placed in a directory site-packages/pkgtest/
and can be accessed from the module.
The package directory is just as simple:
pkgtest/ src/ libhello.c libhello.h Makefile pkgtest/ __init__.py hello.py setup.py
My setup.py looks like this:
setup.py
import subprocess
from setuptools import setup
from distutils.command.install import install as _install
class install(_install):
def run(self):
subprocess.call(['make', 'clean', '-C', 'src'])
subprocess.call(['make', '-C', 'src'])
_install.run(self)
setup(
name='pkgtest',
version='0.0.1',
author='stefan',
packages=['pkgtest'],
package_data={'pkgtest': ['libhello.so']},
cmdclass={'install': install},
)
The Makefile actually builds the library and copies it to my python package directory.
CSI / Makefile
all: libhello.so
libhello.o: libhello.c
gcc -fPIC -Wall -g -c libhello.c
libhello.so: libhello.o
gcc -shared -fPIC -o libhello.so libhello.o
cp libhello.so ../pkgtest/libhello.so
clean:
rm -f *.o *.so
So everything hello.py
actually loads the library and calls a function hello
that prints out some text. But for completeness, I'll show the code here:
pkgtest / hello.py
import os import ctypes basedir = os.path.abspath(os.path.dirname(__file__)) libpath = os.path.join(basedir, 'libhello.so') dll = ctypes.CDLL(libpath) def say_hello(): dll.hello()
So this does work, but what I don't like about this approach is that the shared library lives in the Python package directory. I suppose it would be better to place it in some central library directory like / usr / lib /. But this requires root privileges during installation. Someone has some experience with this problem and would like to share a solution or a helpful idea. It would be great.
source to share
You can create a Python package that includes shared libraries and works with just about any linux distribution using manylinux .
The goal of the manylinux project is to provide a convenient way to distribute Python binaries as wheels on Linux. This effort sparked PEP 513 , which defines platform tags
manylinux1_x86_64
andmanylinux1_i686
.
General procedure:
- Build an external Python library and package inside one of the docker containers provided by the manylinux command. (see python-manylinux-demo )
- Run
auditwheel repair
to copy the external shared libraries your package depends on into the Python wheel, setting RPATH accordingly.
See .travis.yml
also build-wheels.sh
example python-manylinux-demo
for an example.
source to share