3
$\begingroup$

I am building a CNN model for age classification. Assuming age of a person is between 1-100, my last Linear Layer contains 100 output neuron.

Now i want to find an appropriate loss function for this classification problem.

I dont wish to use Regression

My Observations:

  • I cannot use MSE or BCE loss because they only works element wise so unsuitable as if actual age is 25 then there will be same loss for predicted age 26 and 50. (Prediction will be 100 element vector as last layer has 100 neurons.)
  • I found about Hinge Loss and Cosine Proximity Loss. But i dont think they can be used in this type of classifiaction either because they are only finding similarity between two vectors without giving any importance or weight to nearby actual-predicted pairs (ex actual age 25 and predicted age 26 should have a very low loss)

Can anyone suggest me a suitable loss function (Preferrably in Pytorch) for this classification problem?

Edit

Lets say I want a Loss Function (L(predicted, actual)) such that (Assuming for 5 class classification)

let actual = [0,0,1,0,0]

L([0,0,1,0,0], actual) < L([0,1,0,0,0], actual) < L([1,0,0,0,0], actual)
L([0,0,1,0,0], actual) < L([0,0,0,1,0], actual) < L([0,0,0,0,1], actual) 
$\endgroup$
5
  • $\begingroup$ 1. What is your activation function for the output layer? 2. If you are making it a 100-Class multi-class then you must arrange dataset in similar variance e.g 12-15(wild guess) for every age i.e. 1-100. $\endgroup$
    – 10xAI
    Sep 11, 2020 at 12:29
  • $\begingroup$ Categorical cross-entropy would fit your problem of multi-class classification $\endgroup$
    – yudhiesh
    Sep 11, 2020 at 14:52
  • $\begingroup$ @yuRa Categoricat cross entropy does not care about the proximity or distance between actual & predicted vectors. See the Edit $\endgroup$ Sep 11, 2020 at 17:49
  • $\begingroup$ @10xAI so for multi-class if if the actual age is 12, then i have to encode it in the form of 96 zeros and 4 ones from (12-15 indices) ? $\endgroup$ Sep 11, 2020 at 18:02
  • $\begingroup$ What you're doing is not quite classification but ordinal regression. There is no issue with this when you work with an ordinal logistic regression such as rms::orm in R. Your CNN is just an extension of that idea, same as how a CNN for MNIST digit classification is an extension of (multinomial) logistic regression. $\endgroup$
    – Dave
    Nov 19, 2021 at 14:52

2 Answers 2

1
$\begingroup$

What you're doing is not quite classification but ordinal regression. There is no issue with this when you work with an ordinal logistic regression such as rms::orm in R.

Your CNN is just an extension of that idea, same as how a CNN for MNIST digit classification is an extension of (multinomial) logistic regression.

Consequently, treat your problem as an ordinal regression.

There do see to be some ideas to shoehorn your model into the usual classification framework, though I have not thought about this enough to decide if this is statistically sound or just an interesting trick.

Or you can just consider this a regression problem and use one of the usual regression loss functions like MSE or MAE.

$\endgroup$
0
$\begingroup$

If your network outputs a vector $x \in \{0,1\}^N$ with $N=100$ and $\sum_{i = 1}^{N} x_{i}= 1$, you could consider weights $\mathbf{W}=(w_{i}) \in \mathbb{R}^{N}$ with $w_{i} :=i$ for $i \in \{1,\ldots,100\}$.

Then, for prediction vector $x$ and ground-truth vector $y$, you could use the loss function $L(x,y):= || \mathbf{W}^{T}x - \mathbf{W}^{T}y || = || \mathbf{W}^{T}(x-y)||$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.