Consider a dataset that will be split into train and test. The model will be learned using the train set and evaluated using the unseen test set. Now the dataset is unbalanced -- it contains more examples belonging to a particular kind of class. In that case, one of the ways to balance it (apart form the ones mentioned here: Train, test split of unbalanced dataset classification) is by assigning weights based on the samples. What is the proper way to assign weights? Should I assign weights on the full dataset and then split into train and test?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
What is the proper way to assign weights? Should I assign weights on the full dataset and then split into train and test?
No, your test set doesn't have to be weighted and shouldn't be, because it should reflect the real distribution of the data. So you should split first and weight only the instances in the training set, since this is during the training stage that these weights are taken into account.
