1
$\begingroup$

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance:

"Tresemme shampoo and conditioner - sulfate-free" = Personal Hygiene
"Lavender-scented handwash with moisturizer" = Personal Hygiene
"Doritos Ranch flavor 18 oz mega party pack" = Snacks
"Painting and Craft kit for adults above 18" = Art and Craft

However, my training dataset is highly imbalanced. A few classes have only 10 records while there is one that has 3000 records. 50000 records overall.

Can anyone suggest any good techniques to deal with the imbalance in text data?

Thanks, GD

$\endgroup$
  • 1
    $\begingroup$ Do you use a particular classifier? Or are you also wondering which one to use? $\endgroup$ – Valentin Calomme May 6 at 9:46
  • $\begingroup$ And are you assigning different values to the error types? $\endgroup$ – S van Balen May 6 at 12:01
  • $\begingroup$ a typical way is by over-sampling your low count example or get more data. This is probably the time for an ensemble method. i.e. Logit followed by X or RF... $\endgroup$ – oaxacamatt May 6 at 16:09
0
$\begingroup$

I too am working on same problem, found these below links very useful in getting started on oversampling and under sampling-

https://machinelearningmastery.com/random-oversampling-and-undersampling-for-imbalanced-classification/

https://en.wikipedia.org/wiki/Oversampling_and_undersampling_in_data_analysis

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.