1
$\begingroup$

I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these cases i'm not sure to interpret them, so thank you in advance. It's a binary classification problem, the classes in the dataset are quite balanced. The first model is a Random Forest with all the features of the dataset: enter image description here

The second one is a KNN Classifier with all the features: enter image description here

Then I selected only 4 features of the dataset and I applied the models (using gridsearchcv so changing hyperparameters), this is Random Forest again:

enter image description here

The last one is KNN Classifier with only 4 features:

enter image description here

Do the first 2 models have some problems looking at the learning curves? And looking at the last 2 models, they have improved comparing to the first 2? I see that the accuracy is worst in the second cases but maybe they are more solid models?

$\endgroup$

1 Answer 1

2
$\begingroup$

Difficult to tell without test data.

Validation data is important to fine-tune the hyperparameters and improve your model.

The final and most important result would be done with test data, i.e. unbiased data directly from the production environment.

In addition to that, a model is more robust if the validation result is closer to the training result, and if the score is neither too low(~0.7) nor too high(~0.95). Consequently, the third case "RF Classifier con best features" seems more reliable. But I could be wrong because I don't know the data and the results with test data or in the production environment.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.