Extract 2d array elements with indices from another 2d array
I have a 2d numpy array: data.shape == (n, 8) and another ind.shape = (n, 4). The ind array is the same length as the data and contains indices such as [4,3,0,6]. How can I create another array with shape == (n, 4) containing elements from data specified by indices from ind? My actual arrays are quite long (form [0]), so the loop is slow. There must be a better way than loops?
import numpy as np
# Example data
data = np.array([[ 0.44180102, -0.05941365, 2.1482739 , -0.56875081, -1.45400572,
-1.44391254, -0.33710766, -0.44214518],
[ 0.79506417, -2.46156966, -0.09929341, -1.07347179, 1.03986533,
-0.45745476, 0.58853107, -1.08565425],
[ 1.40348682, -1.43396403, 0.8267174 , -1.54812358, -1.05854445,
0.15789466, -0.0666025 , 0.29058816]])
ind = np.array([[3, 4, 1, 5],
[4, 7, 0, 1],
[5, 1, 3, 6]])
# This is the part I want to vectorize:
out = np.zeros(ind.shape)
for i in range(ind.shape[0]):
out[i,:] = data[i,ind[i,:]]
# This should be good
assert np.all(out == np.array([[-0.56875081, -1.45400572, -0.05941365, -1.44391254],
[ 1.03986533, -1.08565425, 0.79506417, -2.46156966],
[ 0.15789466, -1.43396403, -1.54812358, -0.0666025 ]]))
source to share
This can be easily done if we index into the raveled array data
:
out = data.ravel()[ind.ravel() + np.repeat(range(0, 8*ind.shape[0], 8), ind.shape[1])].reshape(ind.shape)
Explanation
It might be easier to understand if it is broken down into three steps:
indices = ind.ravel() + np.repeat(range(0, 8*ind.shape[0], 8), ind.shape[1])
out = data.ravel()[indices]
out = out.reshape(ind.shape)
ind
has information about the items from data
which we want. Unfortunately, this is expressed in two-dimensional terms. The first line above converts them to indices
1-D raveled data
. The second line above selects these elements from the raveled array data
. The third line restores the 2D shape to out
. 2-D indices presented ind
are converted to ind indices
have indices
source to share
What you want is something like this:
import numpy as np
data = np.array([[ 0.4, -0.1, 2.1, -0.6, -1.5, -1.4, -0.3, -0.4],
[ 0.8, -2.5, -0.1, -1.1, 1. , -0.5, 0.6, -1.1],
[ 1.4, -1.4, 0.8, -1.5, -1.1, 0.2, -0.1, 0.3]])
expected = np.array([[-0.6, -1.5, -0.1, -1.4],
[ 1. , -1.1, 0.8, -2.5],
[ 0.2, -1.4, -1.5, -0.1]])
indI = np.array([[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2]])
indJ = np.array([[3, 4, 1, 5],
[4, 7, 0, 1],
[5, 1, 3, 6]])
out = data[indI, indJ]
assert np.all(out == expected)
Note that indI
both indJ
have the same shape and that
out[i, j] == data[indI[i, j], indJ[i, j]]
for all i
and j
.
You may have noticed that it is indI
very repetitive. Because of numpy broadcasting magic, one can simply indI
:
indI = np.array([[0],
[1],
[2]])
You can create such an array in indI
several ways, here is my favorite:
a, b = indJ.shape indI, _ = np.ogrid[:a, :0] out = data[indI, indJ]
source to share