Iterating over 2d arrays contained in a 3d array in Python
I have seen many questions requiring a faster way to iterate over each element of a 2d array, but I have not found a good method for iterating over a 3-dimensional array to apply the function to each 2d array, For example:
from scipy.fftpack import dct
X = np.arange(10000, dtype=np.float32).reshape(-1,4,4)
np.array(map(dct, X))
Here, I am looping through each 2d array contained in a 3D measurement array (625,4,4)
and applying DCT (Discrete Cosine Transform) to each array 4X4
. I was wondering if there is a better method for doing this.
thank
source to share
Numpy functions:
In this case, since it dct
is a function numpy
, it has built-in functionality to apply to a specific axis. Almost all numpy functions operate on full arrays or can be used to operate on a specific axis (row or column).
So, just using a parameter axis
to a function dct
:
dct( X, axis=2)
you will get an equivalent result:
>>> ( dct(X, axis=2) == np.array(map(dct, X)) ).all()
True
which is also> 35 times faster than using the function map
in our matrix case (625,4,4)
:
%timeit dct(X, axis=2)
1000 loops, best of 3: 157 µs per loop
%timeit np.array(map(dct, X))
100 loops, best of 3: 5.76 ms per loop
Common Python functions:
In other cases, you can vectorize
use the python function using np.vectorize or np.frompyfunc . For example, if you have a demo function that does a scalar operation:
def foo(x): # gives an error if passed in an array
return x**2
>>> X = np.arange(8, dtype=np.float32).reshape(-1,2,2)
>>> foo_arr = np.vectorize( foo)
>>> foo_arr(X)
array([[[ 0., 1.],
[ 4., 9.]],
[[ 16., 25.],
[ 36., 49.]]])
The discussion here might also be helpful to you. That being said, vectorizing your non-numpy function doesn't actually make it any faster, though.
source to share