1
$\begingroup$

I have the task of coming up with a model of 95% accuracy for a classification problem. I have training data and a hold-out data set. I have the opportunity to request data of a particular class with desired characteristics to achieve this objective.

What method shall I use to plan the data acquisition through another team? I am currently at 86% accuracy. I use LightGBM for the model development. Would consider parameter tuning and ensemble with XGBoost and TabNet. But I think I need better data to achieve higher accuracy. Feature engineering is also in play.

Also note that it is a multi-class classification problem.

$\endgroup$
0
1
$\begingroup$

Since it is a multi-class classification problem, look at the confusion matrix to find the specific categories that are being misclassified. Then acquire more data for categories where the most mistakes happen.

Another approach would to examine the decision boundary and acquire more data near the decision boundary.

These techniques can be combined - request data that has relevant feature values from commonly misclassified categories.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.