1
$\begingroup$

I am using Haberman's cancer survival dataset https://www.kaggle.com/gilsousa/habermans-survival-data-set to draw a box plot. Here Surv_status is the target variable which has two classes and axil_nodes_det is the feature. I am getting the plot as follows with too many outliers from the first class

Box plot for habrman's survival dataset

I want to know whether more outliers will affect the prediction of output

$\endgroup$
  • $\begingroup$ Depends what models you are going to use and in what way. There is no way to tell beforehand. $\endgroup$ – user2974951 Oct 3 '18 at 14:07
  • $\begingroup$ Can a box plot have these many outliers? $\endgroup$ – syed amer Oct 3 '18 at 14:10
  • $\begingroup$ Are those really "outliers"? Class 1 has >50% of its samples with a value of 0, while Class 2 has <25% of its values equal to 0. It's tough to tell, but the distributions don't really look all that different aside from that. Boxplots are a useful visualization tool, but I wouldn't rely on it for outlier detection - the tradeoff between longer whiskers and fewer outliers is rather arbitrary. $\endgroup$ – Nuclear Hoagie Oct 3 '18 at 14:11
1
$\begingroup$

As the comments suggest, it's not always helpful to think of points outside the whiskers as "outliers". What you are seeing in these boxplots is a strong positive skew. And yes, a strongly skewed target is typically harder to predict than a less-skewed target.

In traditional survival analysis, it's common to model this skewed data using a parametric probability distribution that naturally produces positive and positive-skewed data, e.g. the Weibull distribution.

Otherwise, you can also try a Box-Cox or inverse hyperbolic sine (IHS) transformation on the survival time to reduce the skew.

For visualizing strongly-skewed data, you can either apply one of the above skew-reducing transformations, or use an "adjusted" boxplot, which have an R implementation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.