My data contains over 200 features and over 500 observations. I want to place the observations into a number of clusters based on the features that make them different.
There are numerous ideas I have and I'm not sure which one is apt:
1) Conduct principal component analysis (PCA) to reduce the features to two dimensions. I've already done this so that I could visualize them on a 2D plot. It would now be quite easy to use k-means clustering with these two dimensions to create the clusters, but I wonder if this isn't a good idea because of all the components that are being lost. But then again, if they're being lost they're probably not that important? Not sure
2) Conduct principal component analysis (PCA) to determine which features are worth including and then conduct k-means clustering on those features. So I probably wouldn't be reducing the dimensions to two, but they would be reduced and then the k-means clustering would be done. This seems like the best idea intuitively to me, but I'm not sure.
3) Forget the PCA and just conduct k-means clustering on all the features I have at the beginning. This feels like it's probably the worst idea because some of the features could be useless but could still be factored into the distance calculations for the clustering, but I'm just including everything I've thought of.