# Can I arbitrarily eliminate 20% of my training data if doing so significantly improves model accuracy?

My dataset contains 2000 records with 125 meaningful fields 5 of which are distributed along highly skewed lognormal behavior.

I've found that if I eliminate all records below some threshold of this lognormal behavior (by combining the fields together then filtering for Nth percentile), my model improves in accuracy from ~78% to ~86%, using a highly tuned random forests classifier. This filter is only done after splitting my data into train, test (which is done after SMOTE).

What makes this particularly odd is that that filter improves results across multiple sampling methods.

Is this filtering acceptable behavior? Why might it be resulting in better predictions?

• Do you also threshold test data ? – Elliot Sep 2 '19 at 18:14
• I'm not sure what you mean @Elliot – Yaakov Bressler Sep 2 '19 at 18:59
• do you also filter out the test data you have splitted before ? – Elliot Sep 2 '19 at 19:09
• No @Elliot, the data is split then the filter is applied to the train set only. A second iteration would start from the main data then resplit then refilter. – Yaakov Bressler Sep 2 '19 at 22:42
• Okay, I’ll make an answer. – Elliot Sep 2 '19 at 23:06