I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the cost of False Positive (FP).
Considering so, I decided to insert the weights into my classification model. Which is the best way to compute these weights? Ideally, the weights have to take into account both the data unbalancing and the miss-classification cost.
I am not interested in putting a 0/1 label to each record, but just in ordering the test dataset according to the output score. The idea is to contact the records with the highest score to offer a product.
I would like to use these weights using, for example, the
sample_weights parameter available in most of scikit-learn classification algorithm (here the documentation).
Is it a good idea to oversample/downsample the data and then use the weight only to control the miss-classification cost? Or would it be better to use weights that are able to consider the whole situation? Are there any known ways to compute these weights?