How to implement clustering on categorical feature vectors in R?

Question

I have a dataset like:

 id color  body  eyes
 1  A  blue  slim green
 2  B black   fat  blue
 3  A black  slim black
 4  C green  slim  blue
 5  D black medim black

whereas each id represents an individual with his individual physical characteristics.

Reproducible:

 structure(list(id = structure(c(1L, 2L, 1L, 3L, 4L), .Label = c("A", 
"B", "C", "D"), class = "factor"), color = structure(c(2L, 1L, 
1L, 3L, 1L), .Label = c("black", "blue", "green"), class = "factor"), 
body = structure(c(3L, 1L, 3L, 3L, 2L), .Label = c("fat", 
"medim", "slim"), class = "factor"), eyes = structure(c(3L, 
2L, 1L, 2L, 1L), .Label = c("black", "blue", "green"), class = "factor")),
 .Names = c("id", 
"color", "body", "eyes"), class = "data.frame", row.names = c(NA, 
-5L))

Then number of the characteristics is fixed (color: blue/black/green, body: slim/fat/medium, eyes: green/blue/black).

My aim is to cluster those individuals.

My conceptual question regards the approach:

1. A simple correlation could be a first step. A question could be: how the combination of these characteristic is likely to appear in groups of individuals?

   2. A more complicated approach. Maybe k-means clustering. How can address this given that these are categorical variables? should I convert them into dummies?

I'm new to this kind of analysis and any hint/reference to the implementation in R is highly appreciated! Thanks

Answer 1

You should use dummy variables and then you can toss it directly into K-means. If you have a lot of categories the efficient way to do this is through a one-hot-encoding (sparse-encoding).

Here's a little demo using clustering and then using the clusters in a regression model. In general, you should avoid doing that but it's illuminating in this case.

library(glmnet)
library(Matrix)
n <- 1e5
nclusters <- 5
set.seed(420)
ls <- data.frame(sample(letters, n, replace=TRUE))
xs <- sparse.model.matrix(~.,data=ls)
print(head(xs))  
# Now let's run k-means
out <- kmeans(xs, centers=nclusters)
bs <- rep(1, dim(xs)[2])
# Let's run k-means on the different categories
clusterpred <- data.frame(out[[1]])
ys <- xs %*% bs + rnorm(n)
print(table(clusterpred))
# Now let's use a clustered data set to predict some outcome
cxs <- sparse.model.matrix(~.,data=clusterpred)
model <- glmnet(y=ys, x=xs, alpha=0)
cmodel <- glmnet(y=ys, x=cxs, alpha=0)

# Predictions
yhat <- predict(model, xs)
yhatc <- predict(cmodel, cxs)
# Looking at the difference RMSEs 
print(sqrt( sum( (ys-yhat)**2 )))
print(sqrt( sum( (ys-yhatc)**2 )))

Answer 2

To start with, an initial brute force method would be to perform a mere count of similar occurrences to have a flavour of which combinations of features appear how many times. This can be easily one using the .N operator provided by the data.table package:

counts <- df[, .N, by = c("first_column", "second_column")]

Once so, you could calculate the correlation between categorical variables using the Pearson's chi-squared test of independence (available in R as chisq.test(x,y, ...)) that is an initial standard approach.

In order to perform the clustering you have to introduce a distance method that allows you to decide how close two points (whose coordinates are the categorical variables at hand) are. There are plenty of methods to assign distances to nominal variables, as in your case, each of which adapts to the use case one is dealing with (dummy variables assignments, dimensional reduction or simply similarity measures, i. e. the variables are either the same or they are not, the former case corresponding to distance 0, the latter to distance 1). There is a good working example in this other answer. After having introduced a suitable distance you can perform the $k$-means clustering or any other type of clustering that most suits you.

A very precise walkthrough is provided in this detailed answer (with examples and references).