Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have this huge mixed data set consisting of both numerical and categorical attributes which upon OneHotEncoding results into a data set with very high dimensionality.
Is it wise to apply machine learning algorithms like K-means clustering, dimensionality reduction and regression on subsets of data set? For example applying K-means clustering to numerical columns first and join the result with categorical data set later.
Applying a machine learning algorithm on only a subset of the data and including other subsets later does not allow the algorithm to assess the importance of each attribute equally.
For example, say you have a data set called A, which has subsets B and C. Without loss of genearality, if you fit a model ('apply an algorithm') on subset B, and then include subset C later, then you're saying 'given subset B is already in the model, assess the impact of including subset C'. Instead, if you apply the entire algorithm on the entire data set (A), then you're allowing the algorithm to discover which features are most important for the desired outcome.
That being said, it may be wise to process the different elements of your data set differently. That is, categorical covariates may be modelled differently from continuous covariates. If you're using something like a feed-forward neural network, then it's not a big deal, but if you're using a more traditional statistical model you may need to take that into account. For example, in R, you need to specify that a categorical covariate is in fact a 'factor' variable.
