Input form and Conv1d in Keras
The first layer of my neural network looks like this:
model.add(Conv1D(filters=40,
kernel_size=25,
input_shape=x_train.shape[1:],
activation='relu',
kernel_regularizer=regularizers.l2(5e-6),
strides=1))
if my input form (600,10)
i get (None, 576, 40)
as output form
if my input form (6000,1)
i get (None, 5976, 40)
as output form
so my question is what exactly is going on here? is this the first example just ignoring 90% of the input?
source to share
This is not "ignoring" 90% of the input, the problem is that if you do a one-dimensional convolution with a kernel of size K over an input of size X, the result of the convolution will be size X - K + 1. If you want the output to be the same size, like the entrance, you need to expand or "stuff" your data. There are several strategies for this, such as adding zeros, replicating the value at the ends, or wrapping. Keras' Convolution1D
has a parameter padding
that you can set to "valid"
(default, no padding), "same"
(add zeros on both sides of the input to get the same output size as the input) and "causal"
(padding with zeros only on one end, idea taken from WaveNet ).
Update
About questions in your comments. So you say your input (600, 10)
. This I assume is the size of one example and you have a batch of examples with size (N, 600, 10)
. In terms of the convolution operation, this means that you have examples N
, each of which has a length of at most 600
(this "length" can be time or something else, it is just the dimension along which the convolution works), and in each of these 600
points you have size vectors 10
. Each of these vectors is considered an atomic pattern with features 10
(eg price, height, size, etc.) or, as it is sometimes called in the context of convolution, "channels" (from the RGB channels used in convolution of a 2D image).
The point is that convolution has a kernel size and multiple output channels, which is a parameter filters
in Keras. In your example, what the convolution does is take every possible piece of 25 contiguous 10-vectors and create one 40-vector for each of them (which is for every example in the batch, of course). This way you go from having 10 functions or channels in your input to 40 after convolution. It is not that it only uses one of the 10 elements in the last dimension, using all of them to create an output.
If the value of the dimensions in your input is not an interpretation of the convolution, or if the operation it performs is not what you expected, you may need to either change the shape of the input or use a different layer.
source to share