Keras: how concatenation over a subset of inputs
I am training a neural network using Keras and Theano where the input data is in a format like:
[
[Situation features],
[Option 1 features],
[Option 2 features],
]
I want to train the model to predict how often each option will be selected, forcing the model to learn how to evaluate each option, and how the situation makes the scoring differences more or less important.
My model looks like this:
option_inputs = [Input(shape=(NUM_FEATURES,), name='situation_input'), Input(shape=(NUM_FEATURES,), name='option_input_0'), Input(shape=(NUM_FEATURES,), name='option_input_1')] situation_input_processing = Dense(5, activation='relu', name='situation_input_processing') option_input_processing = Dense(20, activation='relu', name='option_input_processing') diversity_neuron = Dense(1, activation='softplus', name='diversity_neuron') scoring_neuron = Dense(1, activation='linear', name='scoring_neuron') diversity_output = diversity_neuron(situation_input_processing(journey_inputs[0])) scoring_outputs = [scoring_neuron(option_input_processing(option_input)) for option_input in option_inputs[1:2]] logit_outputs = [Multiply()([diversity_output, scoring_output]) for scoring_output in scoring_outputs] probability_outputs = Activation('softmax')(keras.layers.concatenate(logit_outputs, axis=-1)) model = Model(inputs=option_inputs, outputs=probability_outputs)
When I try to get it, probability_outputs
I get the error:
ValueError icon:
Concatenate
must be called on the input list
It looks like the error is caused because logit_outputs
not built, iterating through all 3 collections of input functions, only out of 2 of them.
Any idea how to get around this issue?
Once the model is trained, I want to observe the outputs diversity_neuron
and scoring_neuron
to find out how to extrapolate the scoring for an arbitrary number of parameters and understand what discourse is.
source to share
I made these changes to fix the problem:
- I include the specifics of the situation at the beginning of each option list of functions
- I added a layer that can filter the situation of the function from the input options. This is done manually by the weight of the non-trained layer.
- Then I can iterate over all the inputs in any network path.
The final code looks like this:
option_inputs = [Input(shape=(NUM_FEATURES,), name='option_input_0'),
Input(shape=(NUM_FEATURES,), name='option_input_1')]
situation_input_filtering = Dense(NUM_SITUATION_FEATURES, activation='linear', name='situation_input_filtering')
situation_input_filtering.trainable = False
situation_input_processing = Dense(5, activation='relu', name='situation_input_processing')
option_input_processing = Dense(20, activation='relu', name='option_input_processing')
diversity_neuron = Dense(1, activation='sigmoid', name='diversity_neuron')
scoring_neuron = Dense(1, activation='linear', name='scoring_neuron')
diversity_outputs = [diversity_neuron(situation_input_processing(situation_input_filtering(option_input))) for
option_input in option_inputs]
scoring_outputs = [scoring_neuron(option_input_processing(option_input)) for option_input in option_inputs]
logit_outputs = [Multiply()([diversity_output, scoring_output]) for diversity_output, scoring_output in
zip(diversity_outputs, scoring_outputs)]
combined = keras.layers.concatenate(logit_outputs, axis=-1)
probability_outputs = Activation('softmax')(combined)
model = Model(inputs=option_inputs, outputs=probability_outputs)
model.compile(optimizer='adadelta', loss='categorical_crossentropy', metrics=['accuracy'])
mask_weights = np.zeros([NUM_FEATURES, NUM_SITUATION_FEATURES])
for i in xrange(NUM_situation_FEATURES):
mask_weights[i, i] = 1.0
for layer in model.layers:
if layer.name == 'situation_input_filtering':
layer.set_weights([mask_weights, np.zeros(NUM_SITUATION_FEATURES)])
source to share