I have a dataset of long/short equity hedge funds returns and their associated benchmarks (market indices).
I need to form multiple regression on the fund returns using the benchmarks returns as independent variables (i am allowed to form linear combination or manipulation of the indices or even non-linear combinations).
Of course, I do not know which independent variables to choose. Are techniques such as subset selection, Lasso, and Ridge supposed to be used in situation like this?
There are a few steps you can take to choose features for linear regression:
1 - Exclude variables that are highly correlated with each other. If variables are highly correlated you are essentially inputting the same information multiple times which can cause over-fitting and does not satisfy the properties no multi-collinearity for linear regression. You can create a Pearson correlation matrix and decided which variables are too highly correlated using some chosen threshold i.e only keep variables with a correlation coefficient of < 0.3
2 - If you have many variables you could perform principal component analysis (PCA) to reduce the dimensions of the data and use those as your linear regression features. The idea of PCA is reduce dimensions while holding all of the information. Each component from PCA are uncorrelated, satisfying the no multi-collinearity property.
3 - There is also a method known as stepwise linear regression. You allow all variables to enter the model and it will iteratively remove and add variables until the model with the highest R-squared (or whatever your chosen model metric is) is produced. You do have to be cautious using the stepwise method as it can lead to overfitting, but it can give an indication on what features to use. Here's some info on stepwise: https://en.wikipedia.org/wiki/Stepwise_regression
4 - If you are using R, there is a brilliant package called "caret" that can help with feature selection. Here is a fantastic link to use as a guide: https://machinelearningmastery.com/feature-selection-with-the-caret-r-package/
Hope this helps out as a starting point
