Clever Geek Handbook

How to implement persistent neuron in Keras?

I have the following neural network in Python / Keras:

input_img = Input(shape=(784,))

encoded = Dense(1000, activation='relu')(input_img)  # L1
encoded = Dense(500, activation='relu')(encoded)     # L2
encoded = Dense(250, activation='relu')(encoded)     # L3
encoded = Dense(2, activation='relu')(encoded)       # L4

decoded = Dense(20, activation='relu')(encoded)      # L5
decoded = Dense(400, activation='relu')(decoded)     # L6
decoded = Dense(100, activation='relu')(decoded)     # L7
decoded = Dense(10, activation='softmax')(decoded)   # L8

mymodel = Model(input_img, decoded)

I would like to make one neuron in each layer 4 ~ 7 be constant 1 (to implement the bias term), that is, it has no input, has a fixed value of 1, and is fully connected to the next layer. Is there an easy way to do this? Many thanks!

+3


source to share


1 answer


You can create constant input tensors:

constant_values = np.ones(shape)
constant = Input(tensor=K.variable(constant_values))


With that said, your use case (offset) sounds like you just have to use use_bias=True

which is the default as @gionni noted.

+3


source






More articles:


All Articles