I am relatively new to data science and big data munging
in general. I currently have various columns of data that range from $0-1$, but most of the values in each column are $0$. The data represents certain attributes about a customer, including the proportion of a certain customer's purchases in a particular category (so that the sum of each of these proportions is $1$), in addition to other data like the number of visits, the time between visits, etc. A set of sample rows could look like:
columns = [other_stuff,'C1','C2','C3','C4','C5','C6']
row = [1945,0.45, 0, 0, 0, 0.3, 0.25]
another_row = [438,0, 0.24, 0, 0.01, 0.5, 0.25]
A sample histogram of one of the "proportional" variables looks like:
As of now, I am struggling to find ways to scale the data to make it usable for clustering with other data with much different units/orders of magnitude. Should I:
Scale all the other variables to a 0-1 range using a Min-Max method and keep the "proportional" variables the same
Scale all variables using the variables' means and standard deviations, even though the "proportional variables" clearly do not follow a normal distribution (also cannot apply a log transformation since most of the values are indeed 0)
Keep everything as is, but perform dimensionality reduction on all variables and perform clustering based off the principal components (if Principal Component Analysis is used)
None of the above; use a completely different set of algorithms/methods
I am currently using option 3 using sckit-learn and Python 3.7. If there are packages in R
that could also help, please throw them my way. Thank you.