Keras: Custom output layer for multiple multi-class classifications

Question

Hello, I’m quite new to machine learning and I want to build my first custom layer in Keras, using Python. I want to use a dataset of 103 dimensions to do classification task. The last fully connected layer of the model has 103 neurons (represented by 13 dots in the image). Groups of five dimensions of the former layer should be connected to three neurons of the output layer, so there will be 20 classifications. The neurons of the output layer represent "True" ("T" in the image), "indifferent" ("?") and "False" ("F"). The remaining three don’t need connections to the output layer.

How can I build this layer? And how can I make sure, that each of the 20 groups with three neurons gives probabilities that add up to 1? Can I apply the softmax activation function to each of the groups, for example?

Edit – This is my solution:

# define input and hidden layers. append them to list by calling the new layer with the last layer in the list
    self.layers: list = [keras.layers.Input(shape=self.neurons)]
    [self.layers.append(keras.layers.Dense(self.neurons, activation=self.activation_hidden_layers)(self.layers[-1])) for _ in range(num_hidden_layers)]
    self.layers.append(keras.layers.Dense(self.neurons - self.dims_to_leave_out, activation=activation_hidden_layers)(self.layers[-1]))

    # define multi-output layer by slicing the neurons from the last hidden layer
    self.outputs: list = []
    index_start: int = 0
    for i in range(int((self.neurons - self.dims_to_leave_out)/self.neurons_per_output_layer)):
        index_end: int = index_start + self.neurons_per_output_layer
        self.outputs.append(keras.layers.Dense(self.output_dims_per_output_layer, activation=self.activation_output_layers)(self.layers[-1][:, index_start:index_end]))
        index_start = index_end

Answer 1

Functional API allows you to design more complicated models, including multi-output models. Check the documentation to see how you can connect specific neurons to others of your choice. You should be able to make custom layers from scratch. Once you build distinct output layers, probabilities within each can be set just as usual by using softmax activation.