Scalar entities for k means clustering

Question

I am trying to understand kmeans clustering and I read a article where kmeans is used for clustering the features generated in network logs. This clustering is followed by a supervised classification.

They mention in article :

"Due to extensive repetition of input and output network ip addresses, portnumbers, and protocol information in the collected data, these were deemed scalar entities for analysis and excluded while clustering. The remaining attributes were chosen as input to clustering algorithm"

I fail to understand what scalar entities mean here? Does this mean that these input and output ip address and port numbers are not considered as features during feature extraction stage? If the ip addresses arent considered, how are the features mapped to the input data of network logs?

Answer 1

While doing clustering analysis, you would prefer to have features which discriminate data points and help you discover more homogeneous segments. However, if the value of a particular feature or variable is same for large percentage of the data, then that feature would not add any information for clustering.

Let us take an example, say you are dealing with bank customers data and there's a feature called 'country' in it. If in the given data, most of the customers have country as USA, then you would want to simply drop that feature as it adds no useful information. This is the extreme case in which there is single unique value.

In your network data logs example, port numbers and protocols would be mostly the same. The web server might be listening on port 443 for HTTPS protocol connection. So in that case, the two columns can be excluded from clustering analysis.

Answer 2

k-means tries to minimize a sum-of-squares function, looking roughly like this: $$\sum_x \sum_i (x_i - \mu_{xi})^2$$ where $\mu_x$ is the centroid of the closest cluster.

This whole expression is only sensible to use on continuous variables. It makes no sense to use it on IP adresses, or port numbers, or binary numbers.

For example on port, port 80 (http) and 443 (https) are closely related, but their "distance" is $363^2 = 131769$. The port 88, (kerberos), which is used for a single-sign-on type of authentication system and not related to http has a distance of only $8^2=64$.

As you can see, while the attribute is "numbers", it nevertheless must not be used with the k-means objective.

Even with "real" numbers, this objective may be a bad idea. For example, the number of connections made, or the number of bytes transferred are not linear. The difference between 0 connection attempts (or 0 bytes transferred) and 1 connection attempt (or 1 megabyte transferred) is much more important than the difference between 1000 vs. 1001 connection attempts (or 1000 vs. 1001 megabytes transferred) - but with the k-means objective, these difference are equal.

Therefore, use a different clustering algorithm, not $k$-means.

k-means isn't suitable for this kind of data. K-means assumes continuous variables (if you observe a difference of x, you may also observe 2*x or 0.5*x), translation invariance (a difference of 1 is the same at 0 and at any other location x), and that larger errors are much more severe (a difference of 2 is 4 times as important as a difference of 1).