0
$\begingroup$

I am trying to cluster customer behavior based on where they shop given by lat/long pairs. I also have other numeric attributes such as volume, average amount spent, etc. I am considering using HDBSCAN to create clusters. However, I'm not sure whether to feed the dataframe directly to the clustering algorithm or whether I would need to scale/normalize the data.

Is it wise to scale the geolocation pairs? Or would important location information be lost?

Any help would be much appreciated.

https://stats.stackexchange.com/questions/89809/is-it-important-to-scale-data-before-clustering

This page explains a lot. However, in the answer by @Anony-Mousse, he mentions not to scale lat/long pairs. That's good but what about other continuous variables?

$\endgroup$
1
$\begingroup$

Don't treat clustering algorithms as black boxes. If you don't understand the question, don't expect to understand the answer.

So before dumping the data and hoping that magically a desired results comes out, understand what you are doing...

  1. Standardizing latitude/longitude is a horrible idea. These values are angles on a sphere. Linearly scaling these values breaks everything that these values mean. There are many valid transformations - even rotations can be good to get a desirable Mercator protection, for example. But standardizing them, I cannot imagine what this would be good for.
  2. Mixing variables with different meaning rarely works well. It's not just the problem of scale. Scaling often helps as a heuristic to prevent one variable dominating another. It also has the nice property that it doesn't matter if your data were feet or yards. But the need to do so usually means that there is something wrong with your approach at a deeper level: that you apparently are trying really hard to compare apples and oranges... You'll get out some result. It's probably even interesting. But once you try to explain or act on it, you're back to square one: what does it mean if you scale your data this way, and why is that better than the infinitely many alternative ways, infinitely many of which lead to other results?
$\endgroup$
  • $\begingroup$ Thank you so much!! I need to understand the data first before expecting anything magical! $\endgroup$ – minion_1 Jul 23 '19 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.