0
$\begingroup$

I have a normalised dataset with range 0,1. I have created boxplots for every feature in the dataset and need to identify which features have extreme outliers (by looking at their boxplots).

However, because of the look of my boxplots I am struggling to identify which would be extreme outliers or mild (or even have any at all). You can't see clearly the whiskers etc of the boxplots (due to the range of 0-1) but can see some outliers.

I'm hoping someone could help me identify if these boxplots (1, 2 and 3) in the attached image have extreme outliers or not?

(im particularly stuck with the 3rd one - because you can still see a circle)

Thanks in advance! enter image description here

$\endgroup$
1
$\begingroup$

These images aren't very useful to make decisions by eye, because the scale is far too compressed. In the second and third images the mean, entire box and both whiskers are all represented by a single vertical line, making it impossible to tell if those values are distributed over a range of 0.01, or 0.00001 or 0.000000001. In the third image, the circle looks overlaid directly onto the vertical line, so any attempt to guess how far outside the interquartile range that point is would be just that, a guess. That point in the circle could be just barely outside the whisker, or a million times greater than the mean value, we simply cannot tell from that plot. All we know from the circles is that those point are outside of the whiskers (typically the mean + 1.5 * IQR), but with no meaningful way to discern mean or IQR from the plots, that's not very valuable information.

When making a visual assessment, it's important to put your data on a scale that can actually be visualized. You wouldn't try to visually estimate the distance from your house to the pub on a map of the universe - you'd need a map that can discern distances on the order of miles, not millions of miles. There's not really any reason to plot the range from 0 to 1 when all your points fall in the range of 0 to 0.01, and it will compress potentially meaningful differences to the point where there is no difference whatsoever on a visual scale. You could also try log-transforming your data, which will compress the high end of the range and stretch the low end of the range, which could be useful for looking at data with a right tail.

Also consider that these might not be outliers at all, depending on the distribution of your data - a boxplot may not tell the whole story. For an exponentially distributed value, for example, it'll be completely expected that you'll have some values skewed to the right which are far from the mean. Those aren't weird outliers, they're just an expected part of the exponential distribution.

$\endgroup$
2
  • $\begingroup$ Thank you for your detailed answer! Some values for some of the features do go up to 1 that's why I had the scale as 0-1 so that none would be cut out. Is there any way I could still give them all the same range but ensure that the boxplot is shown properly? $\endgroup$ – bb25 Oct 30 '20 at 11:33
  • $\begingroup$ @bb25 It looks like you're making the plots in R, which should automatically select axis limits based on the data, if you don't explicitly give it the axis limits. It looks like the features exist on different scales, so there may not be one scale at which you can examine every feature effectively - if feature 1 is on the scale [0, 0.3] and feature 3 is on the scale [0. 0.000001], you'll have a hard time comparing on the same axes. A log-transform can be useful for certain types of data to make differences at the low end of the scale more apparent, as well. $\endgroup$ – Nuclear Hoagie Oct 30 '20 at 13:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.