I'm trying to set a different activation function for each hidden unit in a layer. Is this possible in Keras with 'Concatenate'?


If I get the point, you can use a similar code like the following:

from keras.layers import merge, Convolution2D, MaxPooling2D, Input

input = Input(shape=(256, 256, 3))

seq1 = Dense(1, activation = 'relu')(input)
seq2 = Dense(1, activation = 'sigmoid')(input)
seq3 = Dense(1, activation = 'tanh')(input)

acum = merge([seq1, seq2, seq3], mode='concat', concat_axis=1)

Depending on your task, specify concat_axis.

  • $\begingroup$ how should I set "concat_axis"? I want to have all units in one layer. I read the documentation, but I didn't get much from it. $\endgroup$ – P.Joseph Dec 23 '17 at 21:04
  • $\begingroup$ suppose you have [[1, 2], [3, 4]] and [[98, 99], [100, 101]] if your axis is 0 the rows get concatenated, if 1, the columns get concatenated. $\endgroup$ – Media Dec 23 '17 at 21:09
  • $\begingroup$ ok, but what does the default value that is -1 mean in this case? $\endgroup$ – P.Joseph Dec 24 '17 at 9:36
  • $\begingroup$ I don't have access to its documentation right now, but if you are familiar with numpy.reshap, whenever you specify -1 it does a default operation, which in this case it unrolls the array to reshape that as a vector if you set it as the first input if I remember. $\endgroup$ – Media Dec 24 '17 at 12:09
  • $\begingroup$ well, the code works fine but I'm still not sure that it is doing what I want $\endgroup$ – P.Joseph Dec 25 '17 at 22:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.