Many questions training unbalanced and duplicated data

Question

I'm a DS student. I have like 30.000 of bank statements, all labeled with a specific category(cat1, cat2, ...). With that data I'm trying to train a classification model but I found several problems:

• The category that has more statements has 10k, the one with less 100 (unbalanced dataset)
• Many rows are repeated with the same text ( for example "Buy for a total of X$ in the supermarket Y", but the repeated rows appears in each label)

• Should I train with repeated statements? if I do so the model will be trained for some specific cases and the predicted results will not be real because for example if I have repeated statements with the same label:

A, B, B, B, B, B (labels)

and I take only one row with label A and B, since the statement will be repeated the accuracy predicting the others B's will be 100%.

• If I do not train with repeated statements am I not deleting important information about my data and can affect the prediction later?
• Should I undersampling or oversampling? It's correct for this kind of data? I will have more repeated rows in the case of oversampling, and undersampling I'll lose a lot of information.

I do not know if what I'm thinking has some sense. I really need some tips in order to train my data.

Answer 1

Repeated entry is as good as you are adding weighted samples i.e. those samples will have extra weights in the learning processes.
So, you should look into the data and decide if that weight is valid for those samples. Otherwise, it might overfit the Model

On Oversample, Undersample
This is a very broad question. Please do read some relevant blogs/SE posts.
You should also try
- Both, Over-sampling and Under-sampling together i.e. Hybrid approach
- Adding appropriate class-weight to the learning i.e. Cost-sensitive learning

You may start here -
Python Library
Blog
Blog
Book