Clever Geek Handbook

Keras.fit () reinitializes weights

I have a model trained using model.fit()

and using model.save()

to save it to a physical file. Now I have another dataset on which I want to resume training with the saved model. But I found that every challenge is fit()

seen as a new learning curve. This means that this is a reinitialization of the weights that were previously generated and saved.

When I called fit()

with epochs 0 then I don't see the weight reset issue. But I definitely want to try with epochs> 0.

Am I missing something or is this a Keras issue.

Keras version: 2.0.3

Thank.

+3


source to share


2 answers


In fact, the call case fit

looks like this:



  • The masses are not reset - your model will have exactly the same weights as before the call fit

    - until the optimization algorithm changes them during the first batch, of course.

  • The model's reset state is the scenario you probably ran into. All optimizer states and hidden model states (especially in case rnn

    ) are reset. This is the only thing that has changed. If you want to keep these values ​​as well (especially in many cases this is the state of the optimizer), you can use a method train_on_batch

    that does not affect any state of the model at all.

+4


source


The correspondence call must not reinitialize the balance.



You write that you are using a new dataset - if this dataset has different statistics, it can easily cause the network to quickly lose precision. If so, try a very low learning rate, or set Trainer = False for early layers during the first few epochs.

0


source






More articles:


All Articles