Is Bias required in the Colvolution Layer?
I am creating a CNN + ensemble model for image classification using Tensorflow in Python. I searched for dog and cat images on google images. Then changed them to 126 * 126 pixels and grayscale, add label 0 to dog, 1 to cat. CNN has 5 conv layer and 2 fc layers. HE, PReLU, max-pooling, drop-out, Adam are used in the model. When I finished setting the parameters, I added "Early Stop", the model learned the 65-70th epoch, completing the accuracy of 92.5 ~ 92.7%. At the end of the training, I want to change my CNN model to VGG network, I checked my CNN parameter, in shock I found that I had not added Bias to the conv layer. The 2 fc layer had an offset, but the 5 conv layer had no offset. So I added Bias to 5 conv layer BUT my model couldn't learn. The cost is increased to infinity.
Is the offset not necessary at the deep convolution level?
source to share
How did you add the offset to the convolutional layer? There are two ways to do this: Linked offsets , which share one offset per core, and Unbound offsets , which use one offset per core and output. Also read this .
Regarding your question whether they are needed, the answer is no . Bias in convolutional layers increases the capacity of your model, making it theoretically capable of representing more complex data. If your model, however, already has the ability to do this, they are not needed.
An example is this implementation of the ResNet 152 level architecture, where the convolution levels are not offset. Instead, an offset is added at subsequent levels of batch normalization.
source to share