Numpy ctypes "dynamic module does not define init function" unless recompiled every time
sorry for another question about dynamic module does not define init function
. I moved on to older questions, but I didn't find one that describes my case in sufficient detail.
I have a C ++ library that needs to export multiple functions in python (like ~ 5 functions defined in a block extern "C" {}
). It works great when I recompile the library every time I import it. However, if I import it without recompiling it gives the errorImportError: dynamic module does not define init function (initmylib)
A very simplified example that reproduces the same behavior (error) looks like this:
C ++ library code in file mylib.cpp
#include <math.h>
// there are some internal C++ functions and classes
// which are not exported, but used in exported functions
extern "C"{
// one of the functions which should be accessible from python
void oscilator( double dt, int n, double * abuff, double * bbuff ){
double a = abuff[0];
double b = bbuff[0];
for (int i=1; i<n; i++){
a = a - b*dt;
b = b + a*dt;
abuff[i] = a;
bbuff[i] = b;
}
}
// there are also other functions but let keep this example simple
// int initmylib( ){ return 0; } // some junk ... if this makes ctypes happy ?
}
python warper mylib.py
C ++ libraries mylib.cpp
:
import numpy as np
from ctypes import c_int, c_double
import ctypes
import os
name='mylib'
ext='.so' # if compited on linux .so on windows .dll
def recompile(
LFLAGS="",
#FFLAGS="-Og -g -Wall"
FFLAGS="-std=c++11 -O3 -ffast-math -ftree-vectorize"
):
import os
print " ===== COMPILATION OF : "+name+".cpp"
print os.getcwd()
os.system("g++ "+FFLAGS+" -c -fPIC "+name+".cpp -o "+name+".o"+LFLAGS)
os.system("g++ "+FFLAGS+" -shared -Wl,-soname,"+name+ext+" -o "+name+ext+" "+name+".o"+LFLAGS)
# this will recompile the library if somebody delete it
if not os.path.exists("./"+name+ext ):
recompile()
lib = ctypes.CDLL("./"+name+ext )
array1d = np.ctypeslib.ndpointer(dtype=np.double, ndim=1, flags='CONTIGUOUS')
# void oscilator( double dt, int n, double * abuff, double * bbuff )
lib.oscilator.argtypes = [ c_double, c_int, array1d, array1d ]
lib.oscilator.restype = None
def oscilator( dt, a, b ):
n = len(a)
lib.oscilator( dt, n, a, b )
python program test.py
that importsmylib
import os
from pylab import *
from basUtils import *
# this will delete all compiled files of the library to force recompilation
def makeclean( ):
[ os.remove(f) for f in os.listdir(".") if f.endswith(".so") ]
[ os.remove(f) for f in os.listdir(".") if f.endswith(".o") ]
[ os.remove(f) for f in os.listdir(".") if f.endswith(".pyc") ]
# if I do makeclean() every time it works, if I do not it does not
#makeclean( )
import mylib
a=zeros(100)
b=zeros(100)
a[0] = 1
mylib.oscilator( 0.1, a, b )
plot( a )
plot( b )
show()
I also tried to make ctypes happy by adding some function int initmylib( ){ return 0; }
in mylib.cpp
, as you can see in the above code. however this will result in an errorSystemError: dynamic module not initialized properly
I don't have this problem when I compile the cos_doubles example from the lecture notes. However, this example only works if I only want to import one function with the same name as the library name. I want something more general.
source to share
Try to run import imp; print imp.find_module('mylib')[1]
. Are you surprised it chooses mylib.so over mylib.py? The interpreter expects mylib.so to be an extension, which for CPython 2.x must define an initialization function named initmylib
. To avoid accidentally trying to import a shared library, change the name to something like _mylib.so or mylib.1.0.so - or just save the file in a directory that isn't in sys.path
.
Note that Windows Extension Modules are DLLs, but with the extension .pyd instead of .dll. Therefore, import mylib
it will not try to load mylib.dll.
source to share