Clever Geek Handbook

How to implement regularization in pybrain

Anyone can give an example of coding an implementation of a regularization method in pybrain? I am trying to prevent overriding in my data and am currently looking for a method like early stopping etc. to do this. Thank!

+3


source to share


3 answers


There is a variable weight decay which is the L2 regulation in pybrain. Also, I would use early stop as a combination with the term weight loss.

The following is how you determine weight breakdown.



trainer = RPropMinusTrainer(net, dataset=trndata, verbose=True, weightdecay=0.01)
0


source


Below is no L1 / L2 regularization. But it can be used to prevent overfitting by stopping early.

From the trainer documentation,



trainUntilConvergence (dataset = None, maxEpochs = None, verbose = None, continueEpochs = 10, validationProportion = 0.25)

Tune the module on the dataset until it converges.

Return the module with the parameters that gave the minimum validation error.

If no dataset is specified, the dataset is transferred during training using initialization. validationProportion is a relation dataset that is used for a validation dataset.

If maxEpochs is given, at most many epochs are learned. Each time the validation error reaches its minimum value, try to continue epops to find the best one.

If you are using the default parameters, you have already enabled the 75:25 split as a workout set against a validation dataset. The validation dataset is used for EARLY STOP.

0


source


Regularization means changing the cost function. The custom choices in PyBrain do affect the cost function - for example, choosing whether the layers are linear or sigmoid - but the cost function itself is not directly affected.

However , elsewhere on StackOverflow , someone claims that L2 regularization is possible with the weightdecay parameter. (The L2 norm sums the squares of the differences in each coordinate, while the L1 norm sums their positive absolute differences.)

0


source






More articles:


All Articles