1
$\begingroup$

Suppose we have a linear regression model that predicts an item’s price. If the item’s prediction is 8 USD and the actual value is 10 USD, then it is clear that the error is pow(10-8, 2)=4. But how is the error calculated when there are more than two classes?

For example, the model that predicts the numbers trained on the MNIST dataset. In this case we have 10 labels that mark the actual values - range between 0-9. But if we use the sigmoid function for the activation then the range for possible predicted values is between 0-1. Right? How can we compare these values if they are on a different scale? For example, the sigmoid function outputs the values 0.5 and we have to compare it to 3?

enter image description here

Improve this question
$\endgroup$
1
$\begingroup$

The problem here is that you are comparing a regression problem with a classification problem. Classifying MNIST digits is treated as a classification task because it is assumed that predicting the wrong digit is always bad, regardless of how "close" you are to the right answer. One could even argue that predicting a 9 instead of a 3 is a better guess than predicting 5 for instance.

If you want really want to compare the values, I would simply pick the class with the highest value and use this as the regression prediction.

Improve this answer
$\endgroup$
  • $\begingroup$ I just want to understand how the error calculation works in classification model. If the number to predict is 3 and the maximum value that the activation function outputs is 1 then how do You make sure that the prediction is right or wrong? $\endgroup$ – Jane Mänd Mar 2 '20 at 9:21
  • $\begingroup$ You one-hot encode the classes, so you essentially provide a prediction for each class, and then you pick the one with the highest "score" $\endgroup$ – Valentin Calomme Mar 2 '20 at 12:44
  • $\begingroup$ If we have more than two classes then we can't use one-hot encoding? Right? Is it correct that then the values for dependant variables (actual values) are initialized this way? * the first dependant variable = 0 * the last dependant variable = number_of_classes-1 And if the number to predict is 3 then the y value in the following logistic regression cost function formula is also 3? i.stack.imgur.com/zgdnk.png $\endgroup$ – Jane Mänd Mar 2 '20 at 14:20
  • 1
    $\begingroup$ Of course, you can use one-hot encoding if you have more than two classes, that is exactly what it is intended to do. If you do it and then you essentially treat your problem as many binary classifiers. If the answer is 3 then your "y" becomes [0, 0, 0, 1, 0, 0, 0, 0, 0, 0] for instance. You can calculate the distance from your prediction to the target by calculating the distance between both vectors $\endgroup$ – Valentin Calomme Mar 2 '20 at 14:27
  • 1
    $\begingroup$ Thank You! Now I understand :) $\endgroup$ – Jane Mänd Mar 3 '20 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.