I am building a CNN model for age classification. Assuming age of a person is between 1-100, my last Linear Layer contains 100 output neuron.

Now i want to find an appropriate loss function for this classification problem.

I dont wish to use Regression

My Observations:

  • I cannot use MSE or BCE loss because they only works element wise so unsuitable as if actual age is 25 then there will be same loss for predicted age 26 and 50. (Prediction will be 100 element vector as last layer has 100 neurons.)
  • I found about Hinge Loss and Cosine Proximity Loss. But i dont think they can be used in this type of classifiaction either because they are only finding similarity between two vectors without giving any importance or weight to nearby actual-predicted pairs (ex actual age 25 and predicted age 26 should have a very low loss)

Can anyone suggest me a suitable loss function (Preferrably in Pytorch) for this classification problem?


Lets say I want a Loss Function (L(predicted, actual)) such that (Assuming for 5 class classification)

let actual = [0,0,1,0,0]

L([0,0,1,0,0], actual) < L([0,1,0,0,0], actual) < L([1,0,0,0,0], actual)
L([0,0,1,0,0], actual) < L([0,0,0,1,0], actual) < L([0,0,0,0,1], actual) 
  • $\begingroup$ 1. What is your activation function for the output layer? 2. If you are making it a 100-Class multi-class then you must arrange dataset in similar variance e.g 12-15(wild guess) for every age i.e. 1-100. $\endgroup$ – 10xAI Sep 11 '20 at 12:29
  • $\begingroup$ Categorical cross-entropy would fit your problem of multi-class classification $\endgroup$ – yudhiesh Sep 11 '20 at 14:52
  • $\begingroup$ @yuRa Categoricat cross entropy does not care about the proximity or distance between actual & predicted vectors. See the Edit $\endgroup$ – Abhishek Aggarwal Sep 11 '20 at 17:49
  • $\begingroup$ @10xAI so for multi-class if if the actual age is 12, then i have to encode it in the form of 96 zeros and 4 ones from (12-15 indices) ? $\endgroup$ – Abhishek Aggarwal Sep 11 '20 at 18:02

If your network outputs a vector $x \in \{0,1\}^N$ with $N=100$ and $\sum_{i = 1}^{N} x_{i}= 1$, you could consider weights $\mathbf{W}=(w_{i}) \in \mathbb{R}^{N}$ with $w_{i} :=i$ for $i \in \{1,\ldots,100\}$.

Then, for prediction vector $x$ and ground-truth vector $y$, you could use the loss function $L(x,y):= || \mathbf{W}^{T}x - \mathbf{W}^{T}y || = || \mathbf{W}^{T}(x-y)||$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.