0
$\begingroup$

I am working on clustering the customer base of a business-to-business company. I have data on customers that consists of both numerical (e.g. # of purchases made, avg. spend per purchase) and categorical (e.g. industry code) data.

Additionally, I have latitude and longitude information for each customer, which I would like to include in the clustering. Normal categorical and numerical data can be clustered using e.g. PAM / K-Prototypes / Hierarchical Clustering (anything where a distance matrix has to be computed, since there are distance functions that can differentiate between both types).

However, I do not know how to go about including latitude and longitude values. Latitude and longitude are in decimal degrees, therefore metrics like Euclidean distance cannot be used. Some possible approaches I have considered are:

  • calculating x, y, z points on a sphere from lat / lon coordinates using

    $x = \cos(lat) \times cos(lon)$

    $y = cos(lat) \times sin(lon)$

    $z = sin(lat)$

    which could then be treated as 3 numeric attributes using Euclidean distance.

  • somehow implement haversine distance in the calculation of the distance matrix. So create a distance function that calculates numeric differences using Euclidean, categorical (after one-hot encoding) using e.g. Jaccard, and lat-long dissimilarity using Haversine. How could I potentially go about implementing something like this? Is it possible, or am I overlooking something?

  • creating regions, such as "EMEA" (Europe, Middle East, Africa), "APAC (Asia Pacific), "NA" (North America) from the lat-lon values, thereby creating more categorical attributes.

Can someone comment on what a suitable approach might be?

$\endgroup$
3
  • $\begingroup$ Are you thinking here that because your data is geographically global, then if you project it onto a usual sort of 2-D rectangular map, some places that are close together in the world inevitably become very far apart on the map? $\endgroup$
    – DavidH
    Aug 29, 2022 at 8:03
  • $\begingroup$ Yes, that was what I was thinking. For some lat, lon pairs, the pairwise distance on a 2D map does not correspond to the actual shortest distance between locations, such as for eastern Russia and Alaska. They are very close, but if they were projected onto a 2D map, the distance between them would be large. Or have I made a mistake in my reasoning? $\endgroup$
    – Ultralite
    Aug 29, 2022 at 10:02
  • $\begingroup$ If that is what you are trying to solve you could embed lat-longs in 3-D space - that is, assign them a point in 3-D space that corresponds to their position on the globe. Euclidean distance between those points is then a reasonable proxy for actual (great circle) distance between points, even if it's not quite the same. There's guidance on how to do this at stackoverflow.com/questions/10473852/…. (I see now I re-read you've already considered this. seems a decent approach to me) $\endgroup$
    – DavidH
    Aug 29, 2022 at 10:36

1 Answer 1

0
$\begingroup$

I don't have your specific data, because you didn't share anything, but I think this generic example should suffice.

import statsmodels.api as sm
import numpy as np
import pandas as pd

df = your_df

from numpy import unique
from numpy import where
from sklearn.datasets import make_classification
from sklearn.cluster import KMeans
from matplotlib import pyplot

# define dataset
X = df[['lat','lon']]


# define the model
model = KMeans(n_clusters=8)
# fit the model
model.fit(X)

# assign a cluster to each example
yhat = model.predict(X)

X['kmeans']=yhat

pyplot.scatter(X['lat'], X['lon'], c=X['kmeans'], cmap='some_variable', s=50, alpha=0.8)

Your specific outcome will certainly be different, but you should see something essentially like this.

enter image description here

The key thing to focus on is this:

X = df[['lat','lon']]

Add more variables as you desire, and check the outcome each time you make a change. Ultimately, you need to figure out what makes sense, because clustering is non-supervised, so there is no R^2 metric, or accuracy metric, are any kind of supervised metric.

You can find more detail here.

https://github.com/ASH-WICUS/Notebooks/blob/master/Clustering%20-%20Historical%20Stock%20Prices.ipynb

https://github.com/ASH-WICUS/Notebooks/blob/master/Haversine%20Distance%20-%20Airport%20or%20Not.ipynb

$\endgroup$
2
  • $\begingroup$ I don't see how this solves the problem of latitude and longitude representing points in three-dimensional space, resulting in inaccuracies when calculating the distance between these points. I believe this method would only work if locations were confined to a geographically similar space (e.g. all points in Germany). Even then, Euclidean distance would only provide an approximation and not the true distance between locations. $\endgroup$
    – Ultralite
    Aug 29, 2022 at 10:04
  • $\begingroup$ Sorry. I just posted two updated links. $\endgroup$
    – ASH
    Aug 29, 2022 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.