2
$\begingroup$

I am trying to decide which particular algorithm would be most appropriate for my use-case.

I have dataset of about 1000 physical buildings in a city with feature space such as location, distance, year built and other characteristics etc. For each new data point, a building, I'd like to find 3-5 buildings that are most similar based on feature space comparison.

I define similarity as weighted comparison of features. I'd like to iterate over entire feature space (w/ filter like location) and choose 3-5 most similar buildings matching the new building data point.

Here's what my data looks like:

data

I'm wondering what similarity measure would make sense? I work in python, so prefer a pythonic/sci-kit learn way of doing this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

It appears to me that what you're looking for in your use-case is not clustering - it's a distance metric.

When you get a new data point, you want to find the 3-5 most similar data points; there's no need for clustering for it. Calculate the distance from the new data point to each of the 'old' data points, and select the top 3-5.

Now, which distance metric to pick? There are options. If you're using SKLearn, I'd look over this page for example of distance(/similarity) metrics.

If your features are continuous, you can normalize them and use cosine similarity; Start with this, and see if it fits.

Improve this answer
$\endgroup$
4
  • $\begingroup$ This makes sense. I am trying to figure out the most appropriate similarity metric for the data I have. (Updated the question with some sample data). Any thoughts on which distance metric make sense to rank properties by similarity? $\endgroup$
    – kms
    Commented Jul 11, 2020 at 14:27
  • $\begingroup$ ... Are these all of your features? It seems to me like almost all of them are categorical $\endgroup$
    – Itamar Mushkin
    Commented Jul 12, 2020 at 5:30
  • $\begingroup$ There are others. It's a combination of categorical and continuous features. $\endgroup$
    – kms
    Commented Jul 12, 2020 at 5:38
  • 1
    $\begingroup$ As a start - plug the categorical variables into a one-hot-encoder, normalize all non-binary features (or normalize all with min-max), and see what cosine similarity yields. That's not a magic trick that's sure to work, but it's a start, and it'll help you see where it makes and doesn't make sense. Also, when searching on this site I found your previous question, it has some leads: datascience.stackexchange.com/questions/8681/… $\endgroup$
    – Itamar Mushkin
    Commented Jul 12, 2020 at 5:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.