Clever Geek Handbook

How to interpret a neural network that won't overload?

I am doing some experiments on different classification datasets using WEKA's MultilayerPerceptron implementation. I expected to be able to observe overfitting as the number of train iterations (epochs) increased. However, even though the number of epochs is quite large (15k), I have not seen this yet. How should I interpret this? Please note that I do not achieve 100% accuracy on trains or test suites, so this is not the case, the problem is too simple.

Some ideas I came up with are as follows:

  • I just didn't wait long enough
  • My network is not sophisticated enough to resell
  • My data doesn't actually contain noise, but is not descriptive enough for the objective function.
  • I am not using the evaluation class correctly in WEKA
  • My test dataset leaked into my train set (I'm 99% sure it doesn't)

I do the following after each epoch (I modified the MultilayerPerceptron to have an "EpochListener", but no other changes other than this:

    Evaluation eval = new Evaluation(train);
    eval.evaluateModel(ann, train);
    System.out.println(eval.pctCorrect());
    eval.evaluateModel(ann, test);
    System.out.println(eval.pctCorrect());

The accuracy of the train seems to plateau, and I never see the accuracy of the test start to decrease significantly.

+3


source to share


3 answers


Wouldn't it be the simplest plausible explanation not that your training and test sets are balanced and the feedback is stable so nothing changes? In other words, you have agreed on the optimal outcome you can get for this problem with this data and this technology.



A simple test would be to drastically cut the training data and see if that would lead to overfitting (which would show almost 100% accuracy on a reduced set of workouts, disgusting accuracy on the test set).

+1


source


Can you describe your network and data a bit? How many dimensions are your data? How many hidden layers, how many nodes are in your network?



My initial thought is that if you have a fairly simple dataset, with a good amount of data, and a fairly simple network, your network simply won't have a sufficient alternative hypothesis to resell.

+2


source


As Taylor Pabus pointed out, it can be difficult to understand without understanding the parameters of Neural Notwork.

Have you adjusted the Learning Level, Impulse, and Epoch parameters? Perhaps adding more hidden neural layers could also help in getting the neural network to recycle if that number is low. The activation function and the number of training runs can also help.

0


source






More articles:


All Articles