Suppose that your supervised learning training set is made out of 3 different datasets, merged into a big one. Because of the way each of those was labeled before merging, you might be suspicious that one of them (maybe the smallest one) is more "important" than the other ones, meaning that their labels are more reliable. The others might contain more errors.

How could you weight the most reliable data points for the ML model to pay more attention to them and increase the loss when it makes a mistake on those samples? And is there a simple way to implement this using scikit-learn?


In scikit-learn, most algorithms (SVM, Decision Trees, SGD, etc.) have a sample_weight argument that you can pass when fitting. In your case, you could provide a different weight based on which of the 3 datasets the data point comes from.

If the algorithm you want to use doesn't provide the sample_weight argument, you can always sample with replacement. Simply put, you give each sample weight, and then you create your dataset by sampling them with replacement. This means that instances with higher weights may appear multiple times in your dataset.

  • $\begingroup$ Sampling with replacement can get complicated if your weights have 2 decimal points - that means that your dataset will grow 100x $\endgroup$ – Guillermo Mosse Jun 22 '20 at 10:21
  • $\begingroup$ Not necessarily. You can limit your sampled dataset to be the size of the original dataset. Simply sample your instances one-by-one until your dataset is of the desired size. $\endgroup$ – Valentin Calomme Jun 22 '20 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.