Gradient descent vectorization

Question

Gradient descent vectorization

I have implemented this gradient descent in Numpy:

def gradientDescent(X, y, theta, alpha, iterations):
    m = len(y)

    for i in range(iterations):
        h = np.dot(X,theta)
        loss = h-y
        theta = theta - (alpha/m)*np.dot(X.T, loss) #update theta

    return theta

While other parts of the code are fully vectorized here, there is also a for loop that I find impossible to eliminate; especially requiring theta update at every step, I don't see how I could vectorize it or write it in a more efficient way.

thanks for the help

+3

vectorization numpy gradient-descent

Luigi Tiburzi 06 oct. 14 at 14:47

source to share

1 answer

Nir friedman · Accepted Answer · 2014-10-06T15:20:02+0000

You cannot vectorize the for loop because each iteration updates the state. Vectorization is primarily used when the calculation can be done in such a way that each iteration calculates an independent result (in a sense).

Gradient descent vectorization

More articles: