2
$\begingroup$

this is my sample data-set

I have to detect anomalies from my data-set. The anomaly is about in which area and in which time of network usage(total_activity in my data) drastically improved. Help me to know how to apply k-means for this data-set.

$\endgroup$
1
$\begingroup$

Kmeans only works on numerical data, so

  1. throw you categorical data out,
  2. standardize your numerical data,
  3. calculate a distance matrix for a selected metric (for ex. dist() with euclidean),
  4. run kmeans(distance_matrix,k=3) (try k=3 to say k=7)
  5. look at model$cluster to get the data point classes
| improve this answer | |
$\endgroup$
1
$\begingroup$

In fact, I would not use k-means for your problem. Why not an svm-based approach like the one provided by sklearn?

See, this for the general method and this for a practical example.

| improve this answer | |
$\endgroup$
1
$\begingroup$

As @user2974951 said, K-means is for numerical data. There are other clustering algorithms that also work for categorical data, e.g. K-modes.

But let's consider your data and try to understand whether it is numerical:

  • square_id - is the identifier, so it most probably does not represent a relationship between the data points. Unless it does, it is not a feature you should use for your algorithm.
  • country_code - represents a country, so an intuitive way to model is to use it categorically. You might make considerations whether you can quantify it in any way (e.g. by deriving distances between countries), but this would be very dependent on the context of your problem, so a matter of further consideration.
  • activity_date - dates are quantitative, but you need to covert the date format to a consistent numeric representation, e.g. via Unix time, which is the time passed since 1 January 1970.
  • activity_time - maybe this is the hour of activity? If this is the case, I suggest that it be merged with activity_date in the representation of time.
  • total_activity - seems to be your only contextual data, represented in scientific notation. Luckily it is numeric, so you can apply K-means on it only. But this would result in clustering on a single numerical line.

This was about how you could work with the data you have. However, the question that remains is what could you actually do?

It'd be worthwhile exploring the data to get some understanding of it. Why don't you for example try to plot your data 2-dimensionally to understand what it represents? For example, you could plot country vs total_activity, or time vs total_activity. You can enhance the representation by mapping the third feature (respectively time or country in each of my two suggestions). You could have country represented by color on the time vs activity plot, or time represented by color lightness (as in HSL color representation) or point size on the country vs activity plot.

| improve this answer | |
$\endgroup$
0
$\begingroup$

KMeans is not appropriate for such data.

The math matters. Check the objective function of k-means, and whether optimizing this makes any sense for your application - probably it doesn't. Then it is the wrong screwdriver for this nail.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.