I have to detect anomalies from my data-set. The anomaly is about in which area and in which time of network usage(total_activity in my data) drastically improved. Help me to know how to apply k-means for this data-set.
Kmeans only works on numerical data, so
- throw you categorical data out,
- standardize your numerical data,
- calculate a distance matrix for a selected metric (for ex.
kmeans(distance_matrix,k=3)(try k=3 to say k=7)
- look at
model$clusterto get the data point classes
As @user2974951 said, K-means is for numerical data. There are other clustering algorithms that also work for categorical data, e.g. K-modes.
But let's consider your data and try to understand whether it is numerical:
- square_id - is the identifier, so it most probably does not represent a relationship between the data points. Unless it does, it is not a feature you should use for your algorithm.
- country_code - represents a country, so an intuitive way to model is to use it categorically. You might make considerations whether you can quantify it in any way (e.g. by deriving distances between countries), but this would be very dependent on the context of your problem, so a matter of further consideration.
- activity_date - dates are quantitative, but you need to covert the date format to a consistent numeric representation, e.g. via Unix time, which is the time passed since 1 January 1970.
- activity_time - maybe this is the hour of activity? If this is the case, I suggest that it be merged with activity_date in the representation of time.
- total_activity - seems to be your only contextual data, represented in scientific notation. Luckily it is numeric, so you can apply K-means on it only. But this would result in clustering on a single numerical line.
This was about how you could work with the data you have. However, the question that remains is what could you actually do?
It'd be worthwhile exploring the data to get some understanding of it. Why don't you for example try to plot your data 2-dimensionally to understand what it represents? For example, you could plot country vs total_activity, or time vs total_activity. You can enhance the representation by mapping the third feature (respectively time or country in each of my two suggestions). You could have country represented by color on the time vs activity plot, or time represented by color lightness (as in HSL color representation) or point size on the country vs activity plot.