Neuroph: Multi Layer Perceptron Backpropagation Not Working
This question is related to the Neuroph Java library.
I have the following program which creates a multi-layer perceptron containing one hidden layer of 20 nodes. The investigated function is x ^ 2. The backpropagation learning rule is used. However, as you can see from the output, the program doesn't seem to work. The result is always 1. Is there any bug in my program?
Program
import org.neuroph.core.NeuralNetwork;
import org.neuroph.core.data.DataSet;
import org.neuroph.nnet.MultiLayerPerceptron;
import org.neuroph.nnet.learning.BackPropagation;
import org.neuroph.util.TransferFunctionType;
public class SquareNeuralNetwork {
public static void main(String[] args) {
NeuralNetwork neuralNetwork = new MultiLayerPerceptron(TransferFunctionType.SIGMOID, 1, 20, 1);
DataSet trainingSet = new DataSet(1, 1);
for (int i = 1; i <= 100; i++) {
trainingSet.addRow(new double[]{i}, new double[]{i * i});
}
BackPropagation backPropagation = new BackPropagation();
backPropagation.setMaxIterations(10);
neuralNetwork.learn(trainingSet, backPropagation);
for (int i = 1; i <= 100; i++) {
neuralNetwork.setInput(i);
neuralNetwork.calculate();
double output = neuralNetwork.getOutput()[0];
System.out.println(i + " - " + output);
}
}
}
Output
1 - 1.0
2 - 1.0
3 - 1.0
4 - 1.0
5 - 1.0
6 - 1.0
7 - 1.0
8 - 1.0
9 - 1.0
10 - 1.0
11 - 1.0
12 - 1.0
source to share
Sigmoid
Sigmoid activation function values in the range:
You seem to be trying to teach a sigmoid function to output values from 1 to 10000, which is not possible. Thus, the best fitness a network can achieve is always output 1.
An alternative approach
You can still teach the neural network to model an exponential function if you rewrite the function to be 1 / x ^ 2 rather than x ^ 2, as this will change the output range to [0, 1] when x> = 1 When using the network after After completing the training, you need to divide 1 / output to get the exponential curve you planned.
I modeled a network with 20 hidden nodes and one hidden layer as a proof of concept:
source to share