As the question entails, I would like to know as a rule of thumb, an upper limit on the number of features I could have for a data set with 1,000,000 rows/observations, before hitting the curse of dimensionality. If not a million observations, an answer with a rough estimate for the number of features for a data set with a specified size would be great too.



The curse of dimensionality means that your intuition fails at a certain number of features. See Average Distance of Random Points in a Unit Hypercube for some examples. It does not matter how many points you have.

For example, at around 100 dimensions every two points randomly sampled points are really close.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.