Numpy ctypes "dynamic module does not define init function" unless recompiled every time

sorry for another question about dynamic module does not define init function

. I moved on to older questions, but I didn't find one that describes my case in sufficient detail.

I have a C ++ library that needs to export multiple functions in python (like ~ 5 functions defined in a block extern "C" {}

). It works great when I recompile the library every time I import it. However, if I import it without recompiling it gives the errorImportError: dynamic module does not define init function (initmylib)

A very simplified example that reproduces the same behavior (error) looks like this:

C ++ library code in file mylib.cpp

#include <math.h>
// there are some internal C++ functions and classes 
// which are not exported, but used in exported functions
extern "C"{
// one of the functions which should be accessible from python
void oscilator( double dt, int n, double * abuff, double * bbuff ){
    double a = abuff[0];
    double b = bbuff[0];
    for (int i=1; i<n; i++){
        a = a - b*dt;
        b = b + a*dt;
        abuff[i] = a;
        bbuff[i] = b;
    }
}
// there are also other functions but let keep this example simple
// int initmylib( ){ return 0; } // some junk ... if this makes ctypes happy ?
}

      

python warper mylib.py

C ++ libraries mylib.cpp

:

import numpy as np
from   ctypes import c_int, c_double
import ctypes
import os

name='mylib'
ext='.so'  # if compited on linux .so on windows .dll

def recompile( 
        LFLAGS="",
        #FFLAGS="-Og -g -Wall"
        FFLAGS="-std=c++11 -O3 -ffast-math -ftree-vectorize"
    ):
    import os
    print " ===== COMPILATION OF : "+name+".cpp"
    print  os.getcwd()
    os.system("g++ "+FFLAGS+" -c -fPIC "+name+".cpp -o "+name+".o"+LFLAGS)
    os.system("g++ "+FFLAGS+" -shared -Wl,-soname,"+name+ext+" -o "+name+ext+" "+name+".o"+LFLAGS)

# this will recompile the library if somebody delete it
if not os.path.exists("./"+name+ext ):
    recompile()

lib    = ctypes.CDLL("./"+name+ext )

array1d = np.ctypeslib.ndpointer(dtype=np.double, ndim=1, flags='CONTIGUOUS')

# void oscilator( double dt, int n, double * abuff, double * bbuff )
lib.oscilator.argtypes = [ c_double, c_int, array1d, array1d ]
lib.oscilator.restype  = None
def oscilator( dt, a, b ):
    n  = len(a)
    lib.oscilator( dt, n, a, b )

      

python program test.py

that importsmylib

import os
from pylab import *
from basUtils import * 

# this will delete all compiled files of the library to force recompilation
def makeclean( ):
    [ os.remove(f) for f in os.listdir(".") if f.endswith(".so") ]
    [ os.remove(f) for f in os.listdir(".") if f.endswith(".o") ]
    [ os.remove(f) for f in os.listdir(".") if f.endswith(".pyc") ]

# if I do makeclean() every time it works, if I do not it does not
#makeclean( )

import mylib

a=zeros(100)
b=zeros(100)
a[0] = 1

mylib.oscilator( 0.1, a, b )

plot( a )
plot( b )

show()

      

I also tried to make ctypes happy by adding some function int initmylib( ){ return 0; }

in mylib.cpp

, as you can see in the above code. however this will result in an errorSystemError: dynamic module not initialized properly

I don't have this problem when I compile the cos_doubles example from the lecture notes. However, this example only works if I only want to import one function with the same name as the library name. I want something more general.

+3


source to share


1 answer


Try to run import imp; print imp.find_module('mylib')[1]

. Are you surprised it chooses mylib.so over mylib.py? The interpreter expects mylib.so to be an extension, which for CPython 2.x must define an initialization function named initmylib

. To avoid accidentally trying to import a shared library, change the name to something like _mylib.so or mylib.1.0.so - or just save the file in a directory that isn't in sys.path

.



Note that Windows Extension Modules are DLLs, but with the extension .pyd instead of .dll. Therefore, import mylib

it will not try to load mylib.dll.

+1


source







All Articles