Outlier detection on categorical network log data

Question

I am working with a completely categorical network log data that consists of source ip address, destination ip address, source port, destination port, protocol.

Data Preprocessing performed : Converted IP addresses to integers. Removed rows with NA's. Dis not perform scaling as it doesn't make any sense with categorical data and also since I'm computing Hamming distance.

Modelling and Implementation : I have implemented 'Hamming Distance' to compute the distance matrix for the set. I have tried converting the categorical variables to numerical by converting them to factors and perform Kmeans on the data, however the accuracy I'm getting is very low as expected. I have used KModes as well and I have got cluster labels to the data, I'm having challenge proceeding further from here.

Question : Once the distance matrix is computed, how can I go about detecting the outliers with such data type? I really appreciate any recommendations. Thanks in advance

Data Snippet :  2887562076 2344654028        6   41940      80
                2344654028 2887562076        6      80   41940
                2344654028 2887562076        6      80   41940

Answer 1

Foremost, in my opinion categorical data should not be converted to continuous format at all. This is statistically incorrect.

Second, if a categorical data say Sex with two levels Male, Female exists then it can be written as 1,2, but, it should be specified that these numbers are to be treated as nominal and not continuous.

Continuing further, one method of detecting outliers in categorical data is to calculate the frequency of the data points and then average each such obtained frequency to the total number of data points. The data points with the minimum frequencies can be considered as outliers.

You might find this paper and this SO post implementation in R interesting.