Background: As part of prediction analysis, I am given a train and test dataset. Both train and test data have numerical and categorical predictor variables and additionally, train data has a numerical target variable. The objective is to predict target in the test.
train = [c1,c2,x3,x4, y] = [Xc,X, y]
test = [c1,c2,x3,x4] = [Xc,X]
Xc, X denotes categorical and numerical predictor variables. I am trying to generate additional features from categorical variables Xc such as count features, count_mean, count_variance and similar features from a combination of categorical variable and a numerical variable (mean, variance etc).
Problem: Is it better to generate features on a combined dataset train+test or is it better to generate features separately on train and test datasets?
What are the implications when the distribution of a categorical variable are different in train and test and what happens when they are similar?