Say we have dataset D1 (columns A, B, C) and D2 (columns A, B, D) with target variable E. As both datasets are rather small, their respective predictive models do not perform really well. To improve predictive performance, may/should I apply the steps below?
- Remove the not-in-common columns (so column A, B and E remain in both datasets)
- Split each dataset in a train and a test set (cross-validation to evaluate, so no validation set)
- Train our predictive model on the combination of the training sets of D1 and D2
- Assess test set performance of D1 and D2 on their respective test sets (using 1 model rather than 2)
I assume this is allowed, but am not fully sure if there are any (bad) repercussions.