I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be very accurate, they should just help the network to generalize better.

I thought of using the "Mean absolute error" for the first and the "mean squared error" for the other labels.

How can I integrate this in Keras?

  • $\begingroup$ You should probably change the definition of your problem to a one class or binary classification problem, since you are not interested in the other labels. $\endgroup$
    – Erwan
    Nov 28 '19 at 14:20
  • $\begingroup$ I think you could do something like that model.compile(loss = [loss1,loss2], loss_weights = [l1,l2], ...). Where by loss1, and loss2 you use loss metrics described there keras.io/losses. According to thread stackoverflow.com/questions/49404309/… final loss measure will be sum of each loss function for every output. $\endgroup$
    – maksylon
    Nov 28 '19 at 14:21
  • $\begingroup$ May i forgot to mention: i'ts a Regression Task. + For the others Labels i'd prefer to use "mean Squared Error" @maksylon-> i cant find the Part you are referring to. (Upper/loser half? Caption?) + (Would If have to add the Loss function selected for every single Label? This would Take alot of space)) If this works ,your answer Souls be very useful :) $\endgroup$
    – Benoid
    Nov 28 '19 at 14:23
  • $\begingroup$ If only that first label matters, why not just classify everything as that first label? $\endgroup$
    – Dave
    Sep 1 '21 at 14:55

Using mean absolute error might damage your output. You must do some regularization. One thing you can do would be to assign custom weights for each class metric.

Though in general, I would strongly advise against doing that. If this class is so important do separate model in one-vs-all fashion. Be aware you'll be overfitted.


What you could do is make two models. Since, the first class is the most important one, you could first do a binary classification (is it the first class or not), after you finished you train your model with data that wasn't the first class and classify it on 13 labels.


You can make a model with multiple (two) different outputs, the first output should predict your most important label, and the second dense layer ( in Keras) should predict others. something like below code in last layers:

    outputs1 = keras.layers.Dense(1, activation='sigmoid', 
    outputs2 = keras.layers.Dense(13, activation='softmax', 

    model = keras.Model(inputs=[inputs], outputs=

    model.compile(loss= ['fisrt loss function', 'second loss 

then control the importance of each loss function with weights.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.