Given some dataset for prediction,
for eg say I have different housing price prediction dataset:
dataset 1 : 100 training and 100 testing sample, 50 feature
dataset 2 : 100 training and 100 testing sample, 120 feature
dataset 3 : 1000 training and 1000 testing sample, 50 feature
dataset 4 : 1000 training and 1000 testing sample, 5000 feature
how should I choose the best methods for estimating the unknown parameters ( predict price) in a linear regression model from the following for each of these dataset?
Ordinary least squares
Stepwise regression
Principal component regression
Partial least squares regression
Should I experiment with each of these one by one and compare the results or is there any rule of thump on when to use each of them based on the dataset ?
Please help