Force selecting samples in majority class with random forest

Question

Context: I have some data to fit a random forest classifier (binary output) with 1 being a very rare event. In particular, in my training set, there are only 614 1's out of 29400 points. I am using sklearn RandomForestClassifier.

I am setting class_weight = balanced to prevent the model from simply predicting 0 to every case. And it is working great!

However, there is also a small group within the 0 class (maybe only 20 - 30 cases) (Edit: actually about 300) that I want my model to capture. I believe because of the sampling natures when building trees, these class are not chosen very often. Is there a known way to solve this problem?

My thought:

1. Add an additional filter after the RF. Trouble is, it's hard to find some easy categorization methods for these 20 -30 negative cases.

2. Force the RF to include these samples when building the trees. Hence this post...

Thanks.

Answer 1

I would suggest that you drop those 30 events and all 1's from your dataset (i presume you know which 30 they are). Then randomly select 584 samples from the remaining dataset and the stick it back together. This would give you a full dataset equally weighted that contains all 30 for definite.

See the following pseudo code (it won't run because i don't know how your identifying these 30 cases so i'm generalising)

data = import()

ones = data.select('1')

interesting = data.select('something to select the 30 data points')

data.drop('1' and 'something to select the 30 data points')

randomList = randomListGen()

selected = data.select(randomList)

joined = join(selected, ones, interesting)

As i said before this won't run on anything, if you give the language you're working in and a minimum working example then i might be able to produce something more solid.

Answer 2

If you can find a column that has a value to select on, you can use stratify in the train_test_split function. Stratify will try to select an equal number of cases of each value, similar to what you are using. You won't capture all of them, just an equal sample of value vs non-value, but this would be a better approach than forcing a non-random sampling on your model.

train_test_split(..., stratify=df['column'])

after this split, the train and the test splits will have the same ratio of values for the column you selected that the original dataframe has.