1
$\begingroup$

I have very disbalanced dataset.

It is about the chance of having a car crash based on categorical variables... The idea is to offer insurance on the customers that drive like they are having a crash but haven't yet.

My idea was to check to find out how the entries that crashed look like (describe the category crash=1) and look for similar entires but with no crash(crash=0).

I am not sure how to develop this, any ideas?

$\endgroup$
1
$\begingroup$

A couple ideas:

  • Train a supervised binary classification model with undersampling crash=0 (negative) or oversampling crash=1 (positive). Due to resampling, this model is likely to be biased towards the positive class so it will predict the true negative instances which are similar to positive ones as positive. Normally this is a problem (False Positive) but in this particular case this is what you want. The proportion of resampling will determine the level of similarity to positive cases you want to consider. You could also use a probabilistic model and use the predicted probability.
  • Train a one-class classification model using only the positive instances (crash=1), then apply it to the negative instances. The ones predicted as positive are similar to the positive ones.
  • Clustering: cluster all the instances regardless of their class, then identify the clusters which have the highest proportion of positive instances. The negative instances which are also in these clusters share some similarities to the positive ones.
$\endgroup$
3
  • $\begingroup$ your answer is really good! Anyways, I will probably try one of the approaches you mentioned! Anyways, how is the process I mentioned? I want to see which features describe better the 1/0 $\endgroup$ Dec 10 '20 at 16:38
  • $\begingroup$ @AlejandroA your approach is good but it's not very precise: how exactly would you "describe the category" crash=1? You could calculate some kind of centroid of the category and then measure similarity of an instance with this centroid. The disadvantage is that maybe there are several distinct patterns, and a single centroid cannot represent that very well. Otherwise you could compute similarity of an instance against every instance in the category crash=1, but then there is the question of how to obtain the final similarity score: average, maximum, average of the top N... ? $\endgroup$
    – Erwan
    Dec 10 '20 at 17:04
  • 1
    $\begingroup$ Overall your approach is about designing a specific method, so you have more technical choices to make but probably also more control over the details. The approaches I mentioned are a bit more "global": you might not have as much control, it works or it doesn't. $\endgroup$
    – Erwan
    Dec 10 '20 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.