Logistic regression - calculating cost function returns incorrect results
I just started taking Andrew Ng's Machine Learning course on Coursera. The topic of the third week is logistic regression, so I am trying to implement the following cost function.
The hypothesis is defined as:
where g is a sigmoidal function:
This is what my function looks like at the moment:
function [J, grad] = costFunction(theta, X, y)
m = length(y); % number of training examples
S = 0;
J = 0;
for i=1:m
Yi = y(i);
Xi = X(i,:);
H = sigmoid(transpose(theta).*Xi);
S = S + ((-Yi)*log(H)-((1-Yi)*log(1-H)));
end
J = S/m;
end
Considering the following values
X = [magic(3) ; magic(3)];
y = [1 0 1 0 1 0]';
[j g] = costFunction([0 1 0]', X, y)
j returns 0.6931 2.6067 0.6931 although the result should be j = 2.6067. I guess there is a problem with Xi, but I just can't see the error.
I would be very grateful if someone could point me in the right direction.
source to share
You have to apply the sigmoid function to the dot product of your parameter vector (theta) and the input vector ( Xi
which in this case is a string vector). So you must change
H = sigmoid(transpose(theta).*Xi);
to
H = sigmoid(theta' * Xi'); % or sigmoid(Xi * theta)
Of course, you need to make sure the input offset 1 is added to your inputs (line 1s to X
).
Then think about how you can vectorize this whole operation so that it can be written without any loops. Thus, it will be significantly faster.
source to share