Scipy matrix normalizing matrix string
I want to normalize each row of a sparse Scipy matrix obtained from a web-oriented graph.
import networkx as nx
import numpy as np
G=nx.random_geometric_graph(10,0.3)
M=nx.to_scipy_sparse_matrix(G, nodelist=G.nodes())
from __future__ import division
print(M[3])
(0, 1) 1
(0, 5) 1
print(M[3].multiply(1/M[3].sum()))
(0, 1) 0.5
(0, 5) 0.5
This is ok, I normalize as usual and works as desired. But if I write:
>>> M[3]=M[3].multiply(1/M[3].sum())
>>> M[3]
<1x10 sparse matrix of type '<type 'numpy.int64'>'
with 10 stored elements in Compressed Sparse Row format>
(0, 0) 0
(0, 1) 0
(0, 2) 0
(0, 3) 0
(0, 4) 0
(0, 5) 0
(0, 6) 0
(0, 7) 0
(0, 8) 0
(0, 9) 0
I just need to iterate over each row and normalize this sparse scipy matrix. How would you do it? Thanks to
source to share
Here's how to do it (from networkx.pagerank_scipy). It uses scipy linear algebra functions instead of iterating over each line. It will probably be faster for large graphs.
In [42]: G=nx.random_geometric_graph(5,0.5)
In [43]: M=nx.to_scipy_sparse_matrix(G, nodelist=G.nodes(), dtype=float)
In [44]: M.todense()
Out[44]:
matrix([[ 0., 1., 0., 1., 1.],
[ 1., 0., 0., 0., 1.],
[ 0., 0., 0., 1., 1.],
[ 1., 0., 1., 0., 1.],
[ 1., 1., 1., 1., 0.]])
In [45]: S = scipy.array(M.sum(axis=1)).flatten()
In [46]: S[S != 0] = 1.0 / S[S != 0]
In [47]: Q = scipy.sparse.spdiags(S.T, 0, *M.shape, format='csr')
In [48]: (Q*M).todense()
Out[48]:
matrix([[ 0. , 0.33333333, 0. , 0.33333333, 0.33333333],
[ 0.5 , 0. , 0. , 0. , 0.5 ],
[ 0. , 0. , 0. , 0.5 , 0.5 ],
[ 0.33333333, 0. , 0.33333333, 0. , 0.33333333],
[ 0.25 , 0.25 , 0.25 , 0.25 , 0. ]])
source to share
Reason for which
print(M[3].multiply(1/M[3].sum()))
gives the expected results and
M[3]=M[3].multiply(1/M[3].sum())
It creates zeros, because M
is an array <type 'numpy.int64'>
. As long as we do not try to return the norm to M
, this is not a problem.
If a M.A
array([[0, 1, 0, 1, 1],
[1, 0, 0, 0, 1],
[0, 0, 0, 1, 1],
[1, 0, 1, 0, 1],
[1, 1, 1, 1, 0]], dtype=int32)
the sum of the columns: Msum = M.sum(axis=1)
. Tightly
matrix([[3],
[2],
[2],
[3],
[4]], dtype=int32)
as well as its inverse:
Mnorm = 1/Msum matrix([[ 0.33333333], [ 0.5 ], [ 0.5 ], [ 0.33333333], [ 0.25 ]])
M.multiply(Mnorm)
is dense (only the method is installed M.multiply
). But make the rate thinned and the product is also thinned
M1 = M.multiply(sparse.csr_matrix(Mnorm))
<5x5 sparse matrix of type '<class 'numpy.float64'>'
with 14 stored elements in Compressed Sparse Row format>
M1.A
array([[ 0. , 0.33333333, 0. , 0.33333333, 0.33333333],
[ 0.5 , 0. , 0. , 0. , 0.5 ],
[ 0. , 0. , 0. , 0.5 , 0.5 ],
[ 0.33333333, 0. , 0.33333333, 0. , 0.33333333],
[ 0.25 , 0.25 , 0.25 , 0.25 , 0. ]])
Equivalent dense operation numpy
:
A = M.A A/np.sum(A, axis=1, keepdims=True)
source to share