Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm trying to cap outliers in a column of my pandas DataFrame.
Here's the boxplot for a column of my original data.
So, using code from this stackoverflow answer, I tried capping outliers.
Here's how capped column looks like, with new outliers after upper bound.
My problem now is, I can't keep capping outliers, yes! What do I do now?
Once is all it takes; you're done. The point of dealing with outliers is that they are rare. Forcing your model to fit to those outliers can make its performance against inliers (which it will encounter far more often) worse, and that's A Bad Thing. Although you now have values that look like outliers when you plot them, this is an artifact of your earlier transformation - those values should be within the "normal" expected range of values for your data.
