Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes?
I know that doesn't work that way, class_weight works as a misclassification cost. But I want to understand if it would give the same results as oversampling the minority classes.
From sklearn's documentation,
The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y))
It puts bigger misclassification weights on minority classes than majority classes. This method has nothing to do with resampling; it modifies the misclassification cost matrix instead.
Changing the misclassification cost of each class is a different approach from resampling approaches. In my opinion, it won't give exactly the same result as oversampling the minority class. Having said that, these two approaches are both helpful to dealing with imbalanced (or unbalanced) data classification
Yes, these should be the same, when upsampling results in the same number of duplicates per sample (provided you're correctly doing the upsampling only inside the training set). The idea is that the impurity/loss at a node is computed with a multiplier either from the class weights or from the number of duplicated samples, and that these are the same (when the weights are rational and the upsampling is made to match those ratios; otherwise, upsampling will happen on a random subsample, and that will be slightly different than weighting).
Here's a notebook (github/colab) that shows the resulting models are the same, at least in a smallish toy example. (Note that at the beginning I duplicated some of the majority class as well, so that the imbalance ratio was integral, and I could nicely upsample the minority class to equal size.)
