0
$\begingroup$

What is the best way to optimize the parameters in a Sklearn classifier when I only have a data set with 684 rows and 177 columns, and the column I want to predict has 3 labels?

I know I should split my data into training, validation, and test set, and then find the parameters to train the training set that maximizes a metric in the validation set and use this optimized classifier in the test set. However, when I did this using a decision tree classifier, the parameters that worked best for the validation set showed a worse result in the test set than the default parameters. So what is the best way to find the best parameters in this data set? I do not know if it helps, but my data set is very sparse.

Improve this question
$\endgroup$
1
  • $\begingroup$ You could look into bagging and boosting. Also, decision tree classifiers have very high variance. You could use a random forest (which aggregates decision tree classifiers to reduce variance), but I would suggest thinking about using a different model. $\endgroup$
    – Andrew Maurer
    Commented Aug 8, 2019 at 4:14

1 Answer 1

Reset to default
1
$\begingroup$

You can't get around the test/train thing. I guess in your case, the problem is the choice of method. When I read your data description, I suspect that you might achieve better results with regularization, e.g. Lasso. See some examples here.

Basically you would apply a multiclass logit with lasso penalty in your case. Alternatively you could also check boosting with lasso (aka $l1$ penalty). Here is some playcode with the iris data.

Since your data has many columns and not so many rows, I might go for lasso (not ridge or elastic net), because lasso can shrink features to zero if they don't make a meaningful contributin to the classification. This is especially helpful with high dimensional data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.