Model with BatchNormalization: stagnant dough loss
I've written a neural network using Keras. It contains BatchNormalization layers.
When I trained him with help model.fit
, everything was fine. When you train with tensorflow as described here , training is fine, but the check step always gives very low performance and it saturates quickly (accuracy is 5%, 10%, 40%, 40%, 40% .., the loss is stagnant too ).
I need to use tensorflow because it provides a lot of flexibility regarding the monitoring part of the training.
I strongly suspect it has something to do with the BN layers and / or how I calculate the test characteristics (see below)
feed_dict = {x: X_valid,
batch_size_placeholder: X_valid.shape[0],
K.learning_phase(): 0,
beta: self.warm_up_schedule(global_step)
}
if self.weights is not None:
feed_dict[weights] = self.weights
acc = accuracy.eval(feed_dict=feed_dict)
Is there anything special if you are calculating the validation accuracy of a model containing Keras BatchNormalizatin layers?
Thank you in advance!
source to share
Actually I found out about training
the level method argument __call__
BatchNormalization
So what can you do by instantiating the layer:
x = Input((dim1, dim2))
h = Dense(dim3)(x)
h = BatchNormalization()(h, training=K.learning_phase())
And when evaluating performance in a validation set:
feed_dict = {x: X_valid, batch_size_placeholder: X_valid.shape[0], K.learning_phase(): 0, beta: self.warm_up_schedule(global_step) } acc = accuracy.eval(feed_dict=feed_dict) summary_ = merged.eval(feed_dict=feed_dict) test_writer.add_summary(summary_, global_step)
source to share