2
$\begingroup$

Context: I have some data to fit a random forest classifier (binary output) with 1 being a very rare event. In particular, in my training set, there are only 614 1's out of 29400 points. I am using sklearn RandomForestClassifier.

I am setting class_weight = balanced to prevent the model from simply predicting 0 to every case. And it is working great!

However, there is also a small group within the 0 class (maybe only 20 - 30 cases) (Edit: actually about 300) that I want my model to capture. I believe because of the sampling natures when building trees, these class are not chosen very often. Is there a known way to solve this problem?

My thought:

  1. Add an additional filter after the RF. Trouble is, it's hard to find some easy categorization methods for these 20 -30 negative cases.

  2. Force the RF to include these samples when building the trees. Hence this post...

Thanks.

$\endgroup$
0
$\begingroup$

I would suggest that you drop those 30 events and all 1's from your dataset (i presume you know which 30 they are). Then randomly select 584 samples from the remaining dataset and the stick it back together. This would give you a full dataset equally weighted that contains all 30 for definite.

See the following pseudo code (it won't run because i don't know how your identifying these 30 cases so i'm generalising)

data = import()

ones = data.select('1')

interesting = data.select('something to select the 30 data points')

data.drop('1' and 'something to select the 30 data points')

randomList = randomListGen()

selected = data.select(randomList)

joined = join(selected, ones, interesting)

As i said before this won't run on anything, if you give the language you're working in and a minimum working example then i might be able to produce something more solid.

$\endgroup$
  • $\begingroup$ I am running in python. Let me try this idea and get back to you. $\endgroup$ – Matt Keller Aug 14 at 16:24
  • $\begingroup$ Sorry my estimates were wrong. There are actually 200 - 300 such events instead of 20 to 30. I don't think including all 300 in a sample of 600 is wise... any suggestions? $\endgroup$ – Matt Keller Aug 15 at 9:27
  • $\begingroup$ In what way do the samples differ from the rest? With that many it might be worth having a pre-processer that decides if its a special case and if it is then it runs a different algorithim. In particularly how do you know that they are different? $\endgroup$ – Tasty213 Aug 15 at 9:49
  • $\begingroup$ Tbh I can't really tell. If I could, I would run the filter, as described in my thought 1. The 0/1 values in the training sets are entirely based on my own intuitive judgement, and I was hoping ML could replicate this process to some extent. $\endgroup$ – Matt Keller Aug 15 at 11:11
  • $\begingroup$ I think you may be better off using the Kmeans minibatch algorithim scikit-learn.org/stable/modules/… . I've never used it personally so might not be able to help much (would advise opening new topic for questions on it). In future this flowchart is pretty much my starting point to find which algorithim to use scikit-learn.org/stable/tutorial/machine_learning_map/… $\endgroup$ – Tasty213 Aug 15 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.