3
$\begingroup$

I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's)

I aim to get high precision (don't care about recall)

I can get high precision of 0.53 if I just assign y=1 if x=1 but when i train DecisionTree, xgboost, randomforest , they all produce model wihch just outputs 1's for any feature value, i.e. they cant find that simple rule (y=1 if x=1) (precision I get is only 0.38 using these algos) .

what algorithm should I use and how can i make some ML algo learn that simple rule to maximize precision, and do not degenerate to always output 1.

Note that the actual problem will involve many features, thus need robust ML algo.

# sample synthetic data,  DecisionTree fails to find the simple rule
df=pd.DataFrame({'x':np.random.choice([0, 1], size=10000, p=[.99, .01])})
df['y']=np.random.randint(0,2,10000)
df.loc[df.x==1,'y']=1

#precision by using rule  y=1 if x==1 else y=0
df.query('x==1')['y'].mean() # = 1.0 
$\endgroup$
1
$\begingroup$

I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's)

First, let me note that there's no point using ML if you have a single boolean feature: the only possible model can be written with a simple if .. else ...

I can get high precision of 0.53 if I just assign y=1 if x=1

This only means that 53% of the instances which have 1 as feature belong to class 1. In other words, this score doesn't take into account the majority of the instances, since most of the instances have 0 as feature.

but when i train DecisionTree, xgboost, randomforest , they all produce model wihch just outputs 1's for any feature value, i.e. they cant find that simple rule (y=1 if x=1) (precision I get is only 0.38 using these algos) .

This simply means that this rule is not a good rule if one takes into account all the instances. Since there's no better choice, the model simply predicts every instance as the majority class.

Your mistake is to look only at the precision score. You should also take recall into account, this way you would see that the majority baseline is a better choice than this rule. Or you could look at accuracy in this case, since your classes are balanced.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.