Numpy: vectorization for multiple values

Imagine you have an RGB image and you want to process every pixel:

import numpy as np
image = np.zeros((1024, 1024, 3))

def rgb_to_something(rgb):

vfunc = np.vectorize(rgb_to_something)



should now get each RGB value. The problem is numpy aligns the array and the function gets r0, g0, b0, r1, g1, b1, ...

when it should get rgb0, rgb1, rgb2, ...

. Can this be done somehow?

Maybe by pre-converting the numpy array to some special data type?

For example (doesn't work of course):

image = image.astype(np.float32)
import ctypes
RGB = ctypes.c_float * 3
ValueError: shape mismatch: objects cannot be broadcast to a single shape


Update: The main goal here is efficiency. A non-integrated version might look something like this:

import numpy as np
image = np.zeros((1024, 1024, 3))
shape = image.shape[0:2]
image = image.reshape((-1, 3))
def rgb_to_something((r, g, b)):
    return r + g + b
transformed_image = np.array([rgb_to_something(rgb) for rgb in image]).reshape(shape)



source to share

2 answers

The easiest way to solve this problem is to pass the entire array to a function and use the vectorized idioms in it. In particular, yours rgb_to_something

can also be written

def rgb_to_something(pixels):
    return pixels.sum(axis=1)


which is about 15x faster than your version:

In [16]: %timeit np.array([old_rgb_to_something(rgb) for rgb in image]).reshape(shape)
1 loops, best of 3: 3.03 s per loop

In [19]: %timeit image.sum(axis=1).reshape(shape)
1 loops, best of 3: 192 ms per loop


The problem with this np.vectorize

is that when applied to large arrays, it inevitably requires a lot of Python function overhead.



You can use Numexpr for some cases. For example:

import numpy as np
import numexpr
rgb = np.random.rand(3,1000,1000)
r,g,b = rgb


In this case, numexpr is 5 times faster than even a numpy "vector" expression. But not all functions can be written this way.

%timeit r*2+g*3/b
10 loops, best of 3: 20.8 ms per loop

%timeit numexpr.evaluate("(r*2+g*3) / b")
100 loops, best of 3: 4.2 ms per loop




All Articles