How to choose variables for regression

I have a dataset of long/short equity hedge funds returns and their associated benchmarks (market indices).

I need to form multiple regression on the fund returns using the benchmarks returns as independent variables (i am allowed to form linear combination or manipulation of the indices or even non-linear combinations).

Of course, I do not know which independent variables to choose. Are techniques such as subset selection, Lasso, and Ridge supposed to be used in situation like this?

There are a few steps you can take to choose features for linear regression:

1 - Exclude variables that are highly correlated with each other. If variables are highly correlated you are essentially inputting the same information multiple times which can cause over-fitting and does not satisfy the properties no multi-collinearity for linear regression. You can create a Pearson correlation matrix and decided which variables are too highly correlated using some chosen threshold i.e only keep variables with a correlation coefficient of < 0.3

2 - If you have many variables you could perform principal component analysis (PCA) to reduce the dimensions of the data and use those as your linear regression features. The idea of PCA is reduce dimensions while holding all of the information. Each component from PCA are uncorrelated, satisfying the no multi-collinearity property.

3 - There is also a method known as stepwise linear regression. You allow all variables to enter the model and it will iteratively remove and add variables until the model with the highest R-squared (or whatever your chosen model metric is) is produced. You do have to be cautious using the stepwise method as it can lead to overfitting, but it can give an indication on what features to use. Here's some info on stepwise: https://en.wikipedia.org/wiki/Stepwise_regression

4 - If you are using R, there is a brilliant package called "caret" that can help with feature selection. Here is a fantastic link to use as a guide: https://machinelearningmastery.com/feature-selection-with-the-caret-r-package/

Hope this helps out as a starting point

• Thank you very much for your comment. Would it be okay to use lasso/ridge too? Commented Feb 14, 2018 at 3:45
• yes as far as I can tell you could use the lasso/ridge method. My advice would be to test out a number of methods and see which gives you a model of best fit :) Commented Feb 14, 2018 at 3:55
• May I ask you one more thing if you are familiar with finance? Commented Feb 15, 2018 at 1:54
• You can ask for sure and I'll let you know whether I can help or not Commented Feb 15, 2018 at 3:31
• Thank you very much! So I have about 20 long short equity hedge fund returns and their benchmarks (each fund is associated with one benchmark). I need to pick 2 or 3 funds and perform multi regression on the fund returns (dependent variable) with the benchmarks (independent variables). I have about 66 rows of data. I thought that 20 independent variables is too many and some are highly correlated with each other, so techniques like PCA would be good to use, as you suggest. Commented Feb 15, 2018 at 3:34