Why can't MATLAB import this python library?

Question

Why can't MATLAB import this python library?

I want to use http://www.losonczylab.org/sima/1.2/ from MATLAB.

I can run it from python.

I can import examples of dependencies into MATLAB.

$ /opt/local/bin/python 
Python 2.7.9 (default, Dec 13 2014, 15:13:49) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin 
Type "help", "copyright", "credits" or "license" for more information. 
>>> from sima import sequence 
>>> sequence 
<module 'sima.sequence' from '/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/sequence.pyc'> 
>>> from sima import ImagingDataset 
>>> ImagingDataset 
<class 'sima.imaging.ImagingDataset'>

in MATLAB:

>> pyversion /opt/local/bin/python 
>> pyversion 

version: '2.7' 
executable: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Cont...' 
library: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib' 
home: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7' 
isloaded: 0 

>> import py.picos.tools.available_solvers 
>> available_solvers 

ans = 

Python list with no properties. 

['cvxopt', 'mosek7'] 

>> import py.numpy.empty 
>> empty(0) 

ans = 

Python ndarray with properties: 

T: [1x1 py.numpy.ndarray] 
base: [1x1 py.NoneType] 
ctypes: [1x1 py.numpy.core._internal._ctypes] 
data: [1x0 py.buffer] 
dtype: [1x1 py.numpy.dtype] 
flags: [1x1 py.numpy.flagsobj] 
flat: [1x1 py.numpy.flatiter] 
imag: [1x1 py.numpy.ndarray] 
itemsize: 8 
nbytes: 0 
ndim: 1 
real: [1x1 py.numpy.ndarray] 
shape: [1x1 py.tuple] 
size: 0 
strides: [1x1 py.tuple] 

[] 

>> import py.sima.sequence 
Error using import 
Import argument 'py.sima.sequence' cannot be found or cannot be imported. 

>> import py.sima.Sequence 
Error using import 
Import argument 'py.sima.Sequence' cannot be found or cannot be imported. 

>> import py.sima.Sequence.create 
Error using import 
Import argument 'py.sima.Sequence.create' cannot be found or cannot be imported. 

>> import py.sima.sequence.create 
Error using import 
Import argument 'py.sima.sequence.create' cannot be found or cannot be imported. 

>> import py.sima.sequence 
Error using import 
Import argument 'py.sima.sequence' cannot be found or cannot be imported. 

>> import py.sima.Sequence 
Error using import 
Import argument 'py.sima.Sequence' cannot be found or cannot be imported. 

>> import py.sima.ImagingDataset 
Error using import 
Import argument 'py.sima.ImagingDataset' cannot be found or cannot be imported. 

>> import py.sima.imagingdataset 
Error using import 
Import argument 'py.sima.imagingdataset' cannot be found or cannot be imported. 

>> import py.sima 
Error using import 
Import argument 'py.sima' cannot be found or cannot be imported.

How can I figure out why you can't import? sima

sima

is on the way:

>> pp = py.pprint.PrettyPrinter

pp = 

  Python PrettyPrinter with no properties.

    <pprint.PrettyPrinter instance at 0x129425bd8>

>> pp.pprint(py.sys.path)
['',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/readline',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
 '/Users/eflister/Library/Python/2.7/lib/python/site-packages',
 '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages']

I came up with the following tedious method of finding import problems - is there a better way?

define the module as in http://www.mathworks.com/help/matlab/matlab_external/call-user-defined-custom-module.html
copy on import from source files in the library that MATLAB won't load
comment out the suspicious import until the MATLAB call to the module succeeds
rename the module and redo (after successful loading MATLAB should reboot to see the new changes)

I only had to do this 29 times to isolate the following problematic imports in sima

. :)

from PIL import Image

from skimage.measure import approximate_polygon
from skimage.measure import find_contours
from skimage.filter import threshold_otsu
from skimage import transform as tf

from sklearn.decomposition import FastICA

import scipy.io
from scipy.stats import uniform
from scipy.linalg import inv
from scipy.sparse.linalg import eigsh, eigs
from scipy.sparse.linalg import inv
from scipy import ndimage
from scipy.ndimage import measurements

import cv2

# note the following are OK:
from scipy.linalg import eig, eigh, norm
from scipy import sparse
from multiprocessing import Pool
import multiprocessing.util as util

running this code in python contains no import exceptions:

lines = """
import h5py
from PIL import Image

from skimage.measure import approximate_polygon
from skimage.measure import find_contours
from skimage.filter import threshold_otsu
from skimage import transform as tf

from sklearn.decomposition import FastICA

import scipy.io
from scipy.stats import uniform
from scipy.linalg import inv
from scipy.sparse.linalg import eigsh, eigs
from scipy.sparse.linalg import inv
from scipy import ndimage
from scipy.ndimage import measurements

import cv2
"""

def tryImport(i):
    try:
        exec(i)
    except ImportError as e:
        print i
        print e
        print

map(tryImport, filter(lambda x: len(x) > 0, lines.splitlines()))

but in MATLAB it gives the result:

import h5py
cannot import name _errors

from PIL import Image
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL/_imaging.so, 2): Library not loaded: /opt/local/lib/libtiff.5.dylib
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL/_imaging.so
  Reason: Incompatible library version: _imaging.so requires version 8.0.0 or later, but libtiff.5.dylib provides version 6.0.0

from skimage.measure import approximate_polygon
cannot import name _ni_support

from skimage.measure import find_contours
cannot import name _ni_support

from skimage.filter import threshold_otsu
cannot import name _ni_support

from skimage import transform as tf
cannot import name hough_ellipse

from sklearn.decomposition import FastICA
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so, 2): Symbol not found: __gfortran_transfer_character_write
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so
  Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so

import scipy.io
cannot import name doccer

from scipy.stats import uniform
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so, 2): Symbol not found: __gfortran_stop_numeric_f08
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
  Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so

from scipy.sparse.linalg import eigsh, eigs
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so, 2): Symbol not found: __gfortran_transfer_array_write
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
  Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so

from scipy.sparse.linalg import inv
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so, 2): Symbol not found: __gfortran_transfer_array_write
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
  Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so

from scipy import ndimage
cannot import name _ni_support

from scipy.ndimage import measurements
cannot import name _ni_support

import cv2
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cv2.so, 2): Library not loaded: /opt/local/lib/libtiff.5.dylib
  Referenced from: /opt/local/lib/libopencv_highgui.2.4.dylib
  Reason: Incompatible library version: libopencv_highgui.2.4.dylib requires version 8.0.0 or later, but libtiff.5.dylib provides version 6.0.0

i found this method shows errors from matlab:

>> py.importlib.import_module('sima')
Python Error: cannot import name hough_ellipse

is there any hope for a sequel here?

I don't really understand the errors, as far as I can guess, some of them are:

matlab libtiff.5.dylib provides version 6.0.0, but PIL and opencv require> = 8.0.0

matlab libgfortran.3.dylib doesn't work for scipy _lbfgsb.so, _ufuncs.so, _arpack.so

would there be some way to compile for MATLAB libraries? scipy/PIL/opencv

Would it hurt a lot? :)

I found the following 3rd party python-from-matlab bridges:

http://algoholic.eu/matpy/ - numpy.ndarray integration
https://github.com/kw/pymex - numpy.ndarray integration, less new development but also looks good
https://github.com/pv/pythoncall - even older, looks less polished
https://code.google.com/p/python-matlab-wormholes/ - also old, only ndarray, bidirectional over TCP

are there others? which opinions / experiences are most promising for this scenario? I guess matte ...

pymex was no good

Mother is not suitable:

>> py_shell
py> import sima
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
    from sima.imaging import ImagingDataset
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 26, in <module>
    import h5py
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/__init__.py", line 23, in <module>
    from . import _conv
  File "h5py/h5t.pxd", line 14, in init h5py._conv (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/_conv.c:6958)
  File "h5py/h5t.pyx", line 139, in init h5py.h5t (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:20282)
  File "h5py/h5t.pyx", line 73, in h5py.h5t.lockid (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:2562)
  File "h5py/h5t.pyx", line 42, in h5py.h5t.typewrap (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:2196)
RuntimeError: Interface initialization failed (Not a datatype object)
py> import sima
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
    from sima.imaging import ImagingDataset
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
    import sima.misc
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
    from skimage import transform as tf
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
    from ._hough_transform import (hough_ellipse, hough_line,
  File "skimage/transform/_hough_transform.pyx", line 13, in init skimage.transform._hough_transform (skimage/transform/_hough_transform.c:22321)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/draw/__init__.py", line 2, in <module>
    from .draw3d import ellipsoid, ellipsoid_stats
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/draw/draw3d.py", line 3, in <module>
    from scipy.special import (ellipkinc as ellip_F, ellipeinc as ellip_E)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so, 2): Symbol not found: __gfortran_stop_numeric_f08
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
  Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
py> import sima
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
    from sima.imaging import ImagingDataset
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
    import sima.misc
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
    from skimage import transform as tf
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
    from ._hough_transform import (hough_ellipse, hough_line,
ImportError: cannot import name hough_ellipse
py> import sima
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
    from sima.imaging import ImagingDataset
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
    import sima.misc
  File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
    from skimage import transform as tf
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
    from ._hough_transform import (hough_ellipse, hough_line,
ImportError: cannot import name hough_ellipse
py> import picos.tools
py> print picos.tools.available_solvers()
['cvxopt', 'mosek7']

I'm starting to think that wormholing in a separate process sounds good ...

but what about using LD_PRELOAD

to run matlab with updated libgfortran and libtiff?

the versions I want are located:

/opt/local/lib/libgcc/libgfortran.3.dylib 
/opt/local/lib/libtiff.5.dylib

from a new matlab session, the following didn't help.

>> setenv('DYLD_LIBRARY_PATH',['/opt/local/lib:' getenv('DYLD_LIBRARY_PATH')])
>> getenv('DYLD_LIBRARY_PATH')

ans =

/opt/local/lib:/Applications/MATLAB_R2015a.app/sys/os/maci64:/Applications/MATLAB_R2015a.app/bin/maci64/../../Contents/MacOS:/Applications/MATLAB_R2015a.app/bin/maci64:/Applications/MATLAB_R2015a.app/extern/lib/maci64:/Applications/MATLAB_R2015a.app/runtime/maci64:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./native_threads:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./server:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./lib/jli

but!!!

~$ DYLD_INSERT_LIBRARIES=/opt/local/lib/libgcc/libgfortran.3.dylib:/opt/local/lib/libtiff.5.dylib /Applications/MATLAB_R2015a.app/bin/matlab

solves all import problems !!!

>> import py.sima
Error using import
Import argument 'py.sima' cannot be found or cannot be imported.

>> import py.sima.Sequence % succeeds!
>> import py.sima.ImagingDataset % succeeds!
>> x = py.sima.ImagingDataset([1,2],'~')
Python Error: ImagingDataset objects must be initialized with a list of sequences.

>> x = py.numpy.random.random([2,3,4])

x = 

  Python ndarray with properties:

           T: [1x1 py.numpy.ndarray]
        base: [1x1 py.NoneType]
      ctypes: [1x1 py.numpy.core._internal._ctypes]
        data: [1x192 py.buffer]
       dtype: [1x1 py.numpy.dtype]
       flags: [1x1 py.numpy.flagsobj]
        flat: [1x1 py.numpy.flatiter]
        imag: [1x1 py.numpy.ndarray]
    itemsize: 8
      nbytes: 192
        ndim: 3
        real: [1x1 py.numpy.ndarray]
       shape: [1x3 py.tuple]
        size: 24
     strides: [1x3 py.tuple]

    [[[ 0.4158531   0.18912596  0.20743597  0.92631168]
      [ 0.0371785   0.48093953  0.88221637  0.85687334]
      [ 0.2267605   0.79464779  0.97438662  0.47791076]]

     [[ 0.05602249  0.06596678  0.05691491  0.02706325]
      [ 0.28477145  0.74781928  0.56700236  0.14500885]
      [ 0.87999554  0.78932748  0.24473039  0.18383564]]]

>> x = py.sima.Sequence.create('ndarray',x)
The class "py.sima.Sequence" is undefined.
Perhaps Java is not running.

>> py.sima.segment.STICA()

ans = 

  Python STICA with no properties.

    <sima.segment.stica.STICA object at 0x138e15890>

>> py.sima.segment.PlaneCA1PC()

ans = 

  Python PlaneCA1PC with no properties.

    <sima.segment.ca1pc.PlaneCA1PC object at 0x138e158d0>

we are finally SO CLOSED. but why can't i run sima.Sequence.create()

! ??!

+3

python linker matlab shared-libraries dyld

user1441998 02 june 15 at 19:42

source to share

2 answers

A: MATLAB has a set of explicit limits that it supports / `NOT`

maintains code `python`

and classes

Despite the fact that both desires and desires to make it work together, it is recognized, MATLAB

does not support python

without restrictions.

MATLAB explicitly lists functions it doesn't support . One of them is support for object names starting with an underscore character.

Many of these error messages that you posted confirm this fact.

More details about UN-SUPPORTED functions can be checked in →> Python Support Limitations

Going through the list shows how huge the budget to pay will be if you want to redesign / redesign all packages that do not fit these UN supported features.

What to do with it?

If you are really heavily dependent on this two-tier MATLAB / python, you may be interested in a different architecture on how to get things to work.

Try to integrate inter-process messaging via et al and start using distributed processing using your own (worker) modules on each side. ZeroMQ

It would be much faster than trial / error imports which will never fly.

+3

user3666197 04 june 15 at 0:51

source to share

user1441998 · Accepted Answer · 2015-06-05T00:01:32+0000

Combined with, DYLD_INSERT_LIBRARIES

we cannot touch sima.Sequence

directly in MATLAB because it has leading underscore attributes, which MATLAB doesn't allow as @ user3666197 pointed out.

but we can call python functions that use sima.Sequence

without problem:

https://github.com/losonczylab/sima/issues/183#issuecomment-109097816

finally, sweet success!

Application:
in Windows, the problem is not with libgfortran/libtiff

, but with correct use msvcr

.

Windows uses python msvcr90.dll

, but MATLAB uses msvcr100.dll

.

in slender, it

CDLL(find_library('c'))

CDLL('C:\\Program Files\\MATLAB\\R2015a\\bin\\win64\\msvcr100.dll')

all works.
This answer seems to be a method that will determine this path automatically, appropriate for where you are invoking slenderness.

Why can't MATLAB import this python library?

A: MATLAB has a set of explicit limits that it supports / NOT (adsbygoogle = window.adsbygoogle || []).push({}); maintains code python (adsbygoogle = window.adsbygoogle || []).push({}); and classes

What to do with it?

More articles:

A: MATLAB has a set of explicit limits that it supports / `NOT`

maintains code `python`

and classes