How does the data shape change during Conv2D and Dense in Keras?
As the name says. This code only works with:
x = Flatten()(x)
Between convolutional layer and dense layer.
import numpy as np
import keras
from keras.models import Sequential, Model
from keras.layers import Dense, Dropout, Flatten, Input
from keras.layers import Conv2D, MaxPooling2D
from keras.optimizers import SGD
# Generate dummy data
x_train = np.random.random((100, 100, 100, 3))
y_train = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
#Build Model
input_layer = Input(shape=(100, 100, 3))
x = Conv2D(32, (3, 3), activation='relu')(input_layer)
x = Dense(256, activation='relu')(x)
x = Dense(10, activation='softmax')(x)
model = Model(inputs=[input_layer],outputs=[x])
#compile network
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd)
#train network
model.fit(x_train, y_train, batch_size=32, epochs=10)
Otherwise I get this error:
Traceback (most recent call last):
File "/home/michael/practice_example.py", line 44, in <module>
model.fit(x_train, y_train, batch_size=32, epochs=10)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1435, in fit
batch_size=batch_size)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1315, in _standardize_user_data
exception_prefix='target')
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 127, in _standardize_input_data
str(array.shape))
ValueError: Error when checking target: expected dense_2 to have 4 dimensions, but got array with shape (100, 10)
Why is the output 4 dimensions without a layer flatten()
?
source to share
According to keras doc,
Conv2D Withdrawal Form
4D tensor with shape: (samples, filters, new_rows, new_cols) if data_format = 'channels_first' or 4D tensor with shape: (samples, new_rows, new_cols, filters) if data_format = 'channels_last'. rows and column values may be changed due to padding.
Since you are using the channels_last
layer's output form will be:
# shape=(100, 100, 100, 3)
x = Conv2D(32, (3, 3), activation='relu')(input_layer)
# shape=(100, row, col, 32)
x = Flatten()(x)
# shape=(100, row*col*32)
x = Dense(256, activation='relu')(x)
# shape=(100, 256)
x = Dense(10, activation='softmax')(x)
# shape=(100, 10)
Explanation of the error (edited, thanks to @Marcin)
Binding a 4D tensor (shape = (100, row, col, 32)) to a 2D (shape = (100, 256)) using a layer Dense
will still constitute a 4D tensor (shape = (100, row, col, 256 )), which is not what you want.
# shape=(100, 100, 100, 3)
x = Conv2D(32, (3, 3), activation='relu')(input_layer)
# shape=(100, row, col, 32)
x = Dense(256, activation='relu')(x)
# shape=(100, row, col, 256)
x = Dense(10, activation='softmax')(x)
# shape=(100, row, col, 10)
And the error will happen when there is a mismatch between the output 4D tensor and the target 2D tensor.
For this you need a layer Flatten
to move it from 4D to 2D.
Link
source to share
From the Dense
documentation it can be read that if the input to Dense
has more than two dimensions - it applies only to the last one - and all other dimensions are saved:
# shape=(100, 100, 100, 3)
x = Conv2D(32, (3, 3), activation='relu')(input_layer)
# shape=(100, row, col, 32)
x = Dense(256, activation='relu')(x)
# shape=(100, row, col, 256)
x = Dense(10, activation='softmax')(x)
# shape=(100, row, col, 10)
This is why the target is expected 4d
.
source to share