My goal is to partition a dataset (X) in distinct clusters. I'm using k-means to be able to pick the center of each cluster assuming all other datapoints behave the same.

Only for the feature selection phase, I have a evaluation dataset (y) which precicely describes the quality of the clusters obtained (by computing the actual distance between given datapoints from X, which does not necessarily correspond to distances in the input dataset X; given as a distance matrix for each pair of samples from X).

My input dataset has a lot of data available in terms of features from which I can choose (not in terms of actual samples); all continous but varying in scale. How can I determine important features and scale them properly to improve my clustering algorithm?

Simple feature elimination by removing features not improving the k-means was my first step, but the runtime is awful even for this simple example.


1 Answer 1


It seems to me that you are facing a metric learning problem. Here is a survey on the topic: https://people.bu.edu/bkulis/pubs/ftml_metric_learning.pdf

In particular, the scikit-learn library for Python provides algorithms to learn Mahanalobis distances over the labelled part of your data (supervised metric learning), which you can use to transform your complete dataset to a lower dimensional space before applying k-means. If the dimensionality reduction is significant, so will be the performance improvement of k-means which likely spends a lot of time calculating the distances between your large feature vectors.

  • 1
    $\begingroup$ Thanks, that looks exactly like what i was looking for ::) $\endgroup$
    – acocado
    Commented Feb 13 at 10:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.