0
$\begingroup$

I am trying to train a sklearn K-NN classifier on a labeled text dataset (in Irish). There are 5 classes, 0-4, but there is a lot of variation between how many there are in each class.

What I have done is I've gotten a corpus of Irish text, iterated through every word and stripped a few letters from it based on a linguistic form it took (or not). The problem is, class 4 (which means no action was performed) accounts for 16.5M out of 20.1M entries and it goes all the way down to class 3 with only 36,000 entries.

Gathering more data probably won't help as this basically represents the proportion of times these forms of words appear in real life.

Is this bad for classification and will it bias the classifier in any way? If it does, is that bias actually of help?

Any help is appreciated.

Justin

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

I could think of 2 solutions:

  1. Since you mention stripping of the words why not make it a 2 step program where in the first classifier is a binary where in 1-3 is one class of Actions performed and the second class is 4 where there is No Action performed. If the word happens to be in the first category you can further run it for classification in between the 3 classes.

  2. Would be to cut down 4 to fit the distribution but this will result in a huge loss of data which I dont think is viable but worth a try!

Bias is never good for any program and that is clearly explained by Shiv!

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks a lot. May I ask about the first method, between the Actions performed, my distributions is around 2M, then 800K then ~200K down to about 30K. Do you think that distribution is not as extreme as the original and a bit more 'safer' to use when I train that second classifier? $\endgroup$
    – Justin Cunningham
    Aug 26, 2020 at 21:39
  • $\begingroup$ There will definitely be a bias present again! I wasn't aware of the distributions withing the first case! You could definitely try it but scaling the values down is the best option for now! Is there any way you can create synthetic data? That could really help! Also sorry for asking but could you please upvote it if it helped you ? :) $\endgroup$
    – Academic
    Aug 26, 2020 at 22:16
  • $\begingroup$ Yes, I think I'll try scaling the values down within both cases. I will look into creating synthetic data. Apologies for the repeated questions, but I'm still learning about this (I'm doing it as a project for a student science/tech exhibition), but what I am doing is getting one embedding from each sentence, so there's a large number associated w/ each one. Could I create synthetic data with that, I'm not too familiar? Also, I did upvote but since my rep is low, it does not display. $\endgroup$
    – Justin Cunningham
    Aug 26, 2020 at 22:44
  • $\begingroup$ That's great! No problem at all! Feel free to ask how many ever you like !! Im not really sure about what you mean, is there any way you can attach a sample of that data? Thanks :) All the best for the exhibition! $\endgroup$
    – Academic
    Aug 26, 2020 at 22:47
  • $\begingroup$ Hey if the answer is okay, could you accept it as verified? Would help a lot :) $\endgroup$
    – Academic
    Aug 27, 2020 at 3:18
1
$\begingroup$

Just imagine it practically, If class A data is 90% and class B data is 10% ,then if you just randomly classify the label you prediction as class A then your accuracy will be 90%.

So biased data will lead your model to be biased over the class which has more data as it will give better predictions in your model.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.