I'm running a linear regression model as baseline for a specific estimation problem. Based on the resulting R-squared, regressor coefficients and their respective p-values, I can conclude that specific independent variables can be removed from the model.
- What is the induced risk of removing these variables from the feature set?
- Can other models—that are better at modelling nonlinear relationships—suffer from this decision?
- How can I be sure that I am not loosing valuable nonlinear information without running "nonlinear" regressors?