1
$\begingroup$

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance:

"Tresemme shampoo and conditioner - sulfate-free" = Personal Hygiene
"Lavender-scented handwash with moisturizer" = Personal Hygiene
"Doritos Ranch flavor 18 oz mega party pack" = Snacks
"Painting and Craft kit for adults above 18" = Art and Craft

However, my training dataset is highly imbalanced. A few classes have only 10 records while there is one that has 3,000 records. 50,000 records overall.

Can anyone suggest any good techniques to deal with the imbalance in text data?

Thanks, GD

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Do you use a particular classifier? Or are you also wondering which one to use? $\endgroup$
    – Valentin Calomme
    May 6, 2020 at 9:46
  • $\begingroup$ And are you assigning different values to the error types? $\endgroup$
    – S van Balen
    May 6, 2020 at 12:01
  • $\begingroup$ a typical way is by over-sampling your low count example or get more data. This is probably the time for an ensemble method. i.e. Logit followed by X or RF... $\endgroup$
    – mccurcio
    May 6, 2020 at 16:09

1 Answer 1

Reset to default
1
$\begingroup$

I too am working on same problem, found these below links very useful in getting started on oversampling and under sampling-

https://machinelearningmastery.com/random-oversampling-and-undersampling-for-imbalanced-classification/

https://en.wikipedia.org/wiki/Oversampling_and_undersampling_in_data_analysis

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.