1
$\begingroup$

I want to evaluate the performance of my linear regression model. I have the true values of y (y-true). I am thinking of two way for evaluation but not sure which one is correct.

Let's assume that we have 2 samples and each sample has two outputs as following:

y_true = [[0.5, 0.5],[0.6, 0.3]]
y_pred = [[0.3, 0.7],[0.9, 0.1]]

- Approach#1 :

One way to calculate the sum of the difference between the actual and predicted for each vector and then average all, as follows:

sum_diff_Vector(1) = abs( 0.5 - 0.3 ) + abs( 0.5 - 0.7 ) = 0.4

sum_diff_Vector(2) = abs( 0.6 - 0.9 ) + abs( 0.3 - 0.1 ) = 0.5

Then avg ( sum_diff_Vector(1) , sum_diff_Vector(2) ) = 0.45

- Approach#2 :

Another way to use the mean absolute error provided by sklearn.metrics in python. The thing with this metric, as opposed to the previous method, it calculates the mean absolute error for each output over all samples independently and then average all of them, as follows:

MAE_OUTPUT(1) = abs(( 0.5 - 0.3 ) + ( 0.6 - 0.9 )) / 2 = 0.25

MAE_OUTPUT(1) = abs(( 0.5 - 0.7 ) + ( 0.3 - 0.1 )) /2 = 0.2

Then avg ( MAE_OUTPUT(1) , MAE_OUTPUT(1) ) = 0.225

Which way is correct and I should use ? please advise?

$\endgroup$
0
$\begingroup$

The only difference is in your example is that you divide by an additional two, because you take the mean per vector instead of the sum. Correctness does not play here because for comparison between different models the only difference is a constant factor and for interpretability it depends on the problem you are solving.

The mean absolute error punishes mistakes linearly while the mean squared error punishes larger mistakes more heavily. This means this depends a bit on what you want to measure, based on the problem you are solving. Next to proper evaluation you could use this same measure to change the KPI you are optimizing directly with a different loss function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.