Outlier detection and removal using scatter-plots and histograms

Question

Suppose, we use the following code to generate scatter plots,

function res = plot2features(tset, f1, f2)
% Plots tset samples on a 2-dimensional diagram
%   using features f1 and f2
% tset - training set; the first column contains class label
% f1 - index of the first feature (mapped to horizontal axis)
% f2 - index of the second feature (mapped to vertical axis)
% 
% res - matrix containing values of f1 and f2 features

    % plotting parameters for different classes
    %   restriction to 8 classes seems reasonable
    pattern(1,:) = 'ks';
    pattern(2,:) = 'rd';
    pattern(3,:) = 'mv';
    pattern(4,:) = 'b^';
    pattern(5,:) = 'gs';
    pattern(6,:) = 'md';
    pattern(7,:) = 'mv';
    pattern(8,:) = 'g^';

    res = tset(:, [f1, f2]);

    % extraction of all unique labels used in tset
    labels = unique(tset(:,1));

    % create diagram and switch to content preserving mode
    figure;
    hold on;
    for i=1:size(labels,1)
        idx = tset(:,1) == labels(i);
        plot(res(idx,1), res(idx,2), pattern(i,:));
    end
    hold off;
end

The following is its usage,

>> plot2features(train, 3,4)

This code generates the following image before removing outliers,

and following image after removing outliers,

---

I have the following questions,

(1) What do the 1st image tell us about the existence of outliers? I can guess that the plot at a distant position is an outlier. But, how can I find which row or column is generating outliers? according to the 1st picture, the outlier is situated at (27,375) coordinates. But, in the actual data it is situated on the train(184:188,:) rows. So, why is that difference?

(2) What do the color codes in the second picture represent?

(3) Why has the two images that much different? Why only removing 4 rows bring so radical differnce?

(4) How can we analyze the existence of outliers using histograms? Please, supply me any study material about outlier detection using histograms.

---

Suppose we have the following training and test data in our hands to be used in testing Bayes Classifier algorithm,

Training data

train.txt

Test data

test.txt

First column represents class. Rest of the columns represent features.

---

Answer 1

1. Yes. This code does nothing but to plot the points for two chosen columns and assign different color and markers to them. It does not do any outlier selection/removal.

2. Column 3 and 4 (which are features in your dataset) are reasonably predictive of different classes.

3. Plot a histogram. See if there is any clear disjoint in among the bars. Investigate a little time to find a suitable bin size.

In addition, the title of your question does not match the content. Can you please update the question?

Answer 2

To answer your last question, histograms will give you an idea about where your outlier is by (as mentioned previously from other users) displaying bars which are disjointed from others by a reasonable distance. You may have to play around with the bin-widths to discover it, but it should be noticeable.

Another tool that may be useful to you is the box-plot. This visualization will not only give you information about the distribution of the data (though, unlike the histogram, frequency won't be specified), but will also specifically identify outliers. I would recommend using a combination of the scatter-plot, histogram, and box-plot to better familiarize yourself with your data.

I'm uncertain if MATLAB has this functionality (though I image it would), but in R when you plot your data you can choose to display the index of each observation above the data point on scatter-plots. Digging into MATLAB's plot function a bit more couldn't hurt.