I have a question written at the end of the post which refers to the "Distances" paragraph. The other first two paragraphs give additional info.


I'm working on a project where I have to do clustering on a dataset containing numerical variables along with binary variables and ordinal variables. I'm working on Python 3.11. More specifically, I have some continuous variables (float), some binary variables and some ordinal variables (variables that can take values 0,1,2,3,4,5,etc. up to some max value). I need to do some clustering on this dataset, and I need to determine outlier points, so I decided to try using DBSCAN from scikit-learn library.


Now I need to decide which distance to use inside DBSCAN, i.e. which distance DBSCAN uses to cluster my points.

However, not all distance metrics are suitable. At first I thought the euclidean distance would be suitable, but then I switched to cosine distance and manhattan distance because I noticed that the silhouette coefficient was better.

After some research online I found that Gower distance is the one I may need to use: Gower distance is a distance geared towards mixed variables. However Gower distance is not implemented in scikit-learn nor in DBSCAN.

Another solution is to do clustering only on the continuous variables and deal with the binary/ordinal ones at a second time without clustering them. I don't like this approach though.


Additionally, I stopped to think about which distance was best. And I came up with this reasoning. I'd like to ask you if it's correct.

Manhattan Distance

The Manhattan Distance of two vectors is the the sum of the distance of each coordinate of the two vectors. Let's say that, for this example, I have a dataset containing only binary variables (0 or 1 values). If I plot them (in an n-dimension space where n is the cardinality of the features) I almost obtain a regular geometric structure, a lattice we could say. I say almost because some points in this lattice could be missing because I may not have any points that have those coordinates.

For example, if I have only two binary variables and I plot them, I (almost) obtain a square where my points can be (0,0), (0,1), (1,0), (1,1). If I have three binary variables I (almost) have a cube, and so on.

When I create the clusters, since DBSCAN uses a density-based approach to cluster the points, if I set the eps value as at least 1 (setting it less than 1 doesn't make sense because DBSCAN won't link any points), I will surely get only one clusters with all points inside. The only exception is if I have missing points, which would stop the "chain of aggregation" of DBSCAN (every points is clustered together with the adjacent ones in horizontal or vertical direction).

I think this reasoning could easily be extended to a dataset where I have only binary and ordinal variables, correct me if I'm wrong.

Cosine Distance

Let's use again the example of a dataset composed of only binary variables for the Cosine Distance. Again, in two dimensions I have a square, and in three dimensions I have a cube. Let's see the two dimensions example. If I calculate the Cosine Distance between point (0,1) and (1,1) (so upper left point and upper right point), the angle between them is 45°. Now if I calculate the Cosine Distance between point (1,0) and (1,1) (so lower right point and upper right point), the angle between them is 45° too. So if I use and eps value that allows me to cluster together two points, again I obtain only 1 cluster with all points, again with the only exception of a lattice with missing points.

And again, this reasoning could be extended to a dataset with only binary and ordinal variables.


My question is: are these reasonings valid also for my dataset (where I have also continuous variables), i.e. it's true that Manhattan Distance and Cosine Distance won't really be suitable for my dataset and I have to use Gower Distance?

Thank you.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy