0
$\begingroup$

Is there any thumb of rule which scoring function should be used for e.g. the validation_curve?

Atm I try to study the difference between several optimizers:

validation_curve(grid_best 
                                          , X_train 
                                          , y_train
                                          , param_name = 'Adam', 'SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adamax', 'Nadam']
                                          , param_range = param_range 
                                          , cv=tscv
                                          , scoring="explained_variance" 
                                          , verbose = 1
                                          , n_jobs = n_jobs
                )

enter image description here

I use the explained_variance but I think the function has to be minimized cause the values are mostly below zero. That's why I think the explained variance does not make sense here?

edit:

When I use r2 I get the following curve:

nn

Is that normal?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'd use r2 for validating regression. It'd be much easier to compare different models. Here's wiki, here's sklearn.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Any explanation why R^2 over others? :) $\endgroup$
    – Ben
    Nov 21, 2019 at 11:22
  • 1
    $\begingroup$ It's not r square / r^2, it's just r2. It's much easier to compare this metric to other splits/experiments, as it converges to 1, 0 is constant answer, below 0 you're just bad. $\endgroup$
    – Piotr Rarus
    Nov 22, 2019 at 9:07
  • $\begingroup$ sure, but why not e.g. rmse? $\endgroup$
    – Ben
    Nov 22, 2019 at 9:25
  • $\begingroup$ It's just easier to compare values that limits to 1. With rmse it's harder to interpret results in a meaningful way. $\endgroup$
    – Piotr Rarus
    Nov 22, 2019 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.