Linear gradient regression in Java

This is a bit of a long shot, but I am wondering if anyone can take a look at this. Am I doing batch gradient descent for linear regression correctly? It gives the expected answers for one independent and interception, but not for multiple independent variables.

/**
 * (using Colt Matrix library)
 * @param alpha Learning Rate
 * @param thetas Current Thetas
 * @param independent 
 * @param dependent
 * @return new Thetas
 */
public DoubleMatrix1D descent(double         alpha,
                              DoubleMatrix1D thetas,
                              DoubleMatrix2D independent,
                              DoubleMatrix1D dependent ) {
    Algebra algebra     = new Algebra();

    // ALPHA*(1/M) in one.
    double  modifier    = alpha / (double)independent.rows();

    //I think this can just skip the transpose of theta.
    //This is the result of every Xi run through the theta (hypothesis fn)
    //So each Xj feature is multiplied by its Theata, to get the results of the hypothesis
    DoubleMatrix1D hypothesies = algebra.mult( independent, thetas );

    //hypothesis - Y  
    //Now we have for each Xi, the difference between predictect by the hypothesis and the actual Yi
    hypothesies.assign(dependent, Functions.minus);

    //Transpose Examples(MxN) to NxM so we can matrix multiply by hypothesis Nx1
    DoubleMatrix2D transposed = algebra.transpose(independent);

    DoubleMatrix1D deltas     = algebra.mult(transposed, hypothesies );


    // Scale the deltas by 1/m and learning rate alhpa.  (alpha/m)
    deltas.assign(Functions.mult(modifier));

    //Theta = Theta - Deltas
    thetas.assign( deltas, Functions.minus );

    return( thetas );
}

      

+3


source to share


1 answer


There is nothing wrong with your implementation and based on your comment, the problem is in collinearity

what you are calling on creation x2

. This is problematic in regression estimation.

To test your algorithm, you can create two independent columns of random numbers. Select a value w0

, w1

and w2

i.e. coefficients for intercept

, x1

and, x2

respectively. Calculate the dependent value y

.



Then see if your stochastic / batch gradient can reconstruct the values w0

, w1

andw2

+1


source







All Articles