I have a very imbalanced sample in which I am trying to predict probability of a rare event (Out of around 25,000 observations, this event is observed around 30 times) and am reluctant to try over/under-sampling on that directly just because of the degree of imbalance.
Just to illustrate what I had in mind with an example: Let's say I'm trying to classify if a gem is an emerald.
But out of 25,000 samples, I only observe 30 emeralds. However, I have some other green stones like Jade and Peridot which bring up my observations to 300.
Would it be a good idea to determine P(Green Stone) and then P(Emerald | Green Stone) by running a two stage classification.
The latter stage will have imbalance ratio of 1 emerald : 9 non-emeralds, which might be more suitable for balancing.
Would appreciate any thoughts/insights on pursuing this idea.