I have several models that classify the input (word embedding) into several classes. My problem is that I need to train these models separately and need to merge the output of these models together to get a label.
For simplicity assume only two models:
Model 1: predicts A, B or C
Model 2: predicts D or E
Then, I need to classify the input X so that I get the joint probability over A, B, C, D and E.
I have tried this code in Keras/Python, in which I define neural networks and remove the softmax activation function after training:
inputs = tf.keras.Input(shape=dim_in)
x = layers.Dense(dim_out)(inputs)
outputs = layers.Activation(activation="softmax")(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='keras_model')
model_without_softmax = tf.keras.Model(inputs=model.inputs[0], outputs=model.layers[-2].output, name='keras_model', trainable=False)
model.compile(optimizer=tf.keras.optimizers.Nadam(),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['sparse_categorical_accuracy'])
Then I get final prediction with softmax over joint arrays of predictions:
prediction = softmax(np.append(prediction_1, prediction_2))
It wasn't working because the output values from the neural networks were on a different scale when classifying into 2 or 3 classes. Thus, I tried dividing into a number of classes but without success:
prediction = softmax(np.append(prediction_1/len(idx2label_d1), prediction_2/len(idx2label_d2)))
Can you point me to right direction for the general formula of merging the neural network's outputs together?