Is it possible to implement mutiple softmaxes in the last layer in Keras? So the sum of Nodes 1-4 = 1; 5-8 = 1; etc.
Should I go for a different network design?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
I would use the functional interface.
Something like this:
from keras.layers import Activation, Input, Dense from keras.models import Model from keras.layers.merge import Concatenate input_ = Input(shape=input_shape) x = input_ x1 = Dense(4, x) x2 = Dense(4, x) x3 = Dense(4, x) x1 = Activation('softmax')(x1) x2 = Activation('softmax')(x2) x3 = Activation('softmax')(x3) x = Concatenate([x1, x2, x3]) model = Model(inputs=input_, outputs=x)
It is possible just implement your own softmax function. You can split a tensor to parts, then compute softmax separately per part and concatenate tensor parts:
def custom_softmax(t): sh = K.shape(t) partial_sm =  for i in range(sh // 4): partial_sm.append(K.softmax(t[:, i*4:(i+1)*4])) return K.concatenate(partial_sm)
concatenate without axis argument concatenate through last axis (in our case axis=1).
Then you can include this activation function in a hidden layer or add it to a graph.
You also need to define a new cost function.