1
$\begingroup$

I am trying to build a regressor for a dataset which gives info about students' school performance and the probability of getting admitted in the University of their choice.

The first 5 observations look like this :

    GRE_Score  TOEFL_Score  Uni_Rating  LOR   CGPA  Research Chance_of_admit
0      337         118           4      4.5   9.65     1          0.92
1      324         107           4      4.5   8.87     1          0.76
2      316         104           3      3.5   8.00     1          0.72
3      322         110           3      2.5   8.67     1          0.80
4      314         103           2      3.0   8.21     0          0.65

I have build the following regressors so far : linear regressor, knn regressor and a recurrent neural network. ( I will try a few more later. )

So far, in order to chose among my models, I used the "score" method on the test set for the first two regressors ( it returns the $R^2$ for each one of them ) and the "evaluate" method on the test set for the network ( it returns the MeanSquaredError ).

So, keeping in mind that $R^2$ and MeanSquaredError have different formulas, how can I compare my network with the other two models ??

Any help is much appreciated.

$\endgroup$
2
$\begingroup$

If you up to a predictive model, you look for a model which performs well on the test set and the metric of interest is the mean squared error which indicates by how much you fail to predict $y$ on average. So don't use $R^2$. Just compare all models based on MSE.

$\endgroup$
2
  • $\begingroup$ I replaced OLS regressor with Ridge regressor.. Ok it is clear that I should use MSE to choose among my final models.. I have one more question now, regarding the tuning of the models : The tuning of the network depends on the MSE.. However, when tuning the Ridge and the kNN regressors, the "cross_val_score" method uses the R^2 coefficient to choose among the models.. Is it ok ? Or should I also change the tuning so that it depends also on the MSE ? $\endgroup$ – batman Jan 18 '20 at 14:18
  • 1
    $\begingroup$ I would tune on MSE $\endgroup$ – Peter Jan 18 '20 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.