Numpy: vectorization for multiple values
Imagine you have an RGB image and you want to process every pixel:
import numpy as np
image = np.zeros((1024, 1024, 3))
def rgb_to_something(rgb):
pass
vfunc = np.vectorize(rgb_to_something)
vfunc(image)
vfunc
should now get each RGB value. The problem is numpy aligns the array and the function gets r0, g0, b0, r1, g1, b1, ...
when it should get
rgb0, rgb1, rgb2, ...
. Can this be done somehow?
http://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html
Maybe by pre-converting the numpy array to some special data type?
For example (doesn't work of course):
image = image.astype(np.float32)
import ctypes
RGB = ctypes.c_float * 3
image.astype(RGB)
ValueError: shape mismatch: objects cannot be broadcast to a single shape
Update: The main goal here is efficiency. A non-integrated version might look something like this:
import numpy as np
image = np.zeros((1024, 1024, 3))
shape = image.shape[0:2]
image = image.reshape((-1, 3))
def rgb_to_something((r, g, b)):
return r + g + b
transformed_image = np.array([rgb_to_something(rgb) for rgb in image]).reshape(shape)
source to share
The easiest way to solve this problem is to pass the entire array to a function and use the vectorized idioms in it. In particular, yours rgb_to_something
can also be written
def rgb_to_something(pixels):
return pixels.sum(axis=1)
which is about 15x faster than your version:
In [16]: %timeit np.array([old_rgb_to_something(rgb) for rgb in image]).reshape(shape)
1 loops, best of 3: 3.03 s per loop
In [19]: %timeit image.sum(axis=1).reshape(shape)
1 loops, best of 3: 192 ms per loop
The problem with this np.vectorize
is that when applied to large arrays, it inevitably requires a lot of Python function overhead.
source to share
You can use Numexpr for some cases. For example:
import numpy as np
import numexpr
rgb = np.random.rand(3,1000,1000)
r,g,b = rgb
In this case, numexpr is 5 times faster than even a numpy "vector" expression. But not all functions can be written this way.
%timeit r*2+g*3/b
10 loops, best of 3: 20.8 ms per loop
%timeit numexpr.evaluate("(r*2+g*3) / b")
100 loops, best of 3: 4.2 ms per loop
source to share