# Imbalanced classification data with a top decile conversion metric

I am working on a binary classification problem with imbalanced data. The dataset consists of 85% 'false' labels and 15% 'true' labels. I have about 1500 data points and am training with Sklearn ensemble tree methods and XGBoost. I know tree methods tend to work better with more data points but I'm not sure what sampling methods would help. I'm trying to optimize for more 'true' points to be in the top decile. I tried to oversample the 'true' class but since I'm using a top decile conversion, the top decile get artificially populated with a lot of 'true' points. This gave inconsistent conversion scores when I used stratified k-fold cross validation.

Should I split the datasets by class and apply sample techniques? What are some relevant techniques that I can try? I'm working with Python and any relevant libraries would be helpful. Thanks in advance!

That's because when you upsample the 'true' class, it affects leaf impurity metrics (such as Gini or entropy) only marginally. Imagine a leaf with $a$ 'true' and $b$ 'false' examples - the Gini would be $2\frac{ab}{a+b}$. When you upsample 'true' class by factor of $\lambda$, the Gini becomes $2\lambda\frac{ab}{a\lambda+b}$. If $a<<b$ (which is your case), then this change is approximately just multiplying all the Ginis by $\lambda$, which does not affect the choice of a next tree split.
Thus my advice would be to give up upsampling attempts, and concentrate on feature engineering, parameter tuning, and maybe getting more real observations. If you still want resampling, then you can just set unequal class_weights when fitting your trees - it will affect trainging, but keep cross-validation metrics intact.