Keras multiple exits
question about keras regression with multiple outputs:
Could you please explain the difference between this network:
two inputs â two outputs
input = Input(shape=(2,), name='bla')
hidden = Dense(hidden, activation='tanh', name='bla')(input)
output = Dense(2, activation='tanh', name='bla')(hidden)
and: two separate inputs â two separate outputs:
input = Input(shape=(2,), name='speed_input')
hidden = Dense(hidden_dim, activation='tanh', name='hidden')(input)
output = Dense(1, activation='tanh', name='bla')(hidden)
input_2 = Input(shape=(1,), name='angle_input')
hidden_2 = Dense(hidden_dim, activation='tanh', name='hidden')(input_2)
output_2 = Dense(1, activation='tanh', name='bla')(hidden_2)
model = Model(inputs=[speed_input, angle_input], outputs=[speed_output, angle_output])
They behave very similarly. Another, when I completely separate them, then the two networks behave as they should.
And is it okay for two single output networks to behave much more comprehensibly than larger ones with two outputs, I didn't think the difference could be as huge as I experienced.
Many thanks:)
source to share
It has to do with how neural networks work. In your first model, each hidden neuron receives 2 inputs (since this is the "Dense" layer, the input is propagated to each neuron). In your second model, you have twice as many neurons, but each one receives only speed_input
or angle_input
and only works with that data, not all of the data.
So, if speed_input
u angle_input
is 2 completely unrelated attributes, you will probably see better performance from separating the two models, since neurons don't get what is basically noise (they don't know your outputs match your inputs, they can only try optimize the loss function). Essentially, you are creating two separate models.
But in most cases, you want to pass the appropriate attributes to the model, which are combined to draw the forecast. Thus, splitting the model would then not make sense, since you are simply capturing the information you need.
source to share