Behavior of dropout layers during testing / training phase
According to the drop-down layers of Keras documentation, different types of behavior are displayed during the training and testing phase:
Note that if your model has different training behavior and (for example, if it uses dropout, BatchNormalization, etc.), you need to pass the training phase flag to your function:
Unfortunately, nobody talks about the real differences. Why does the drop behave differently during the testing phase? I expect the layer to set a certain number of neurons to 0. Why should this behavior depend on the training / testing phase?
source to share
Dropout is used during the training phase to reduce the likelihood of overfitting. As you say, this layer deactivates certain neurons. The model will become more insensitive to the weights of other nodes. Basically with a dropdown layer, the trained model will be average across many thinned models. Check out more detailed explanation here
However, when you apply your trained model, you want to use the full power of the model. You want to use all the neurons in the trained (average) network to get the best accuracy.
source to share