Say I have a dataset with 1000 columns and 3M rows. I know that this is will definitely suffer from Curse of Dimensionality and that I need to reduce the number of dimensions. But to what extent am I supposed to reduce the dimensions by?
With each additional dimension the number of datapoints required such that the data is not too sparse increases exponentially according to my understanding.
So how do I know know for different number of columns what the golden number of data points is? Assuming that I have the capability to collect infinite amount of data but would still have a small cost associated for each datapoint, how much should I collect?
I am using these slides to understand the concept: http://www.dataminingbook.info/pmwiki.php/Main/BookPathUploads?action=download&upname=slides-chap6.pdf