# Clustering categorical variable values based on continuous target values [closed]

Let's say I have $$n$$ data points with just one categorical feature $$x$$ and a continuous target variable $$y$$. I want to divide the possible values of $$x$$ into subsets such that the value of $$y$$ doesn't vary much within a subset.

As an example, suppose there are only $$5$$ possible values of $$x$$: $$x_1,\ldots,x_5$$. We observe that the values of $$y$$ don't vary much across $$x_1$$ and $$x_2$$. $$y$$ doesn't vary much across $$x_3$$ and $$x_5$$, but is different from the typical $$y$$ value for $$x_1$$ and $$x_2$$. For $$x_4$$, $$y$$ values are quite different from the above groups. So we can say that $$(x_1,x_2), (x_3,x_5)$$ and $$(x_4)$$ can be considered as "clusters".

Now what's a concrete way of saying that $$y$$ doesn't "vary much across $$x_1$$ and $$x_2$$"? One natural way to define that is for $$y$$ to have the same distribution for $$x_1$$ and $$x_2$$, i.e. the same mean and standard deviation of $$y$$ values for both $$x_1$$ and $$x_2$$.

• Is there a better way to characterize the fact that $$y$$ doesn't "vary much across $$x_1$$ and $$x_2$$? Maybe the way I defined it above is too idealistic and I need a criterion more suited to a real-life dataset?

• Are there any popular existing methods or library functions to solve such kind of a problem (i.e. clustering categorical feature values on the basis of continuous target values)?

Since you are looking for a degree of similarity regarding $$y$$ and the values of $$x_1,...,x_5$$ do not matter you can view this as a clustering problem regarding $$y$$:

Let $$y_1,...y_5$$ be the target values with $$f(x_i)=y_i$$ for $$i\in\{1,...5\}$$ then you need to define a distance measure $$d(y_i,y_j)$$ which, since your variables $$y$$ are continuous, could be the euclidian distance: $$d(y_i,y_j) = (y_i-y_j)^2$$. But you could also choose the absolute difference. (note that I am assuming your $$y_i$$ to be one-dimensional here, i.e. $$y_i\in \mathbb R$$). That gives you a formula to measure "does not vary that much".

It also provides an answer to your second question: you can choose from a range of unsupervised ML algorithms to do clustering. The most popular one probably being $$K$$-means. The idea here is very straight forward:

1. select a number of clusters K
2. initialize the position of the K clusters randomly
3. Assign each y_i to the closest cluster
4. For each cluster k calculate the new cluster position as the mean of all y_i belonging to that cluster
5. repeat 3 and 4 until the assignments of y_i does not change anymore


Mathematically this gives you a mapping $$c(y_i)=k$$.

And eventually when you are done with your clustering you just pick a cluster $$k$$ and for each $$y_i$$ in that cluster you look up the corresponding $$x_i$$. Which will return what you asked for.