Can I consider this pattern of data as a linear and use parametric multiple linear regression?

Question

In the data, there are 355 observations including one continuous dependent variable (Y: ranges from 15-55) and 12 independent variables (continuous, categorical, and ordinal). The X1 (2 levels) and X6 (3 levels) are considered as categorical variables. Here are some questions that I have:

1. Can I assume that all the coefficients (except X1 and X6 which are categorical) are linear with respect to Y?

2. Can I consider X5 as continuous variable; however, it is ordinal and ranges from (1-7)?

3. Can I get the X7 (year) as continuous variable; however, it’s ordinal and rages from 2002-2006 (In fact, year of data per se does not improve the response; it is the other factors occurring in the same time period which result in improvements and we don’t know those factors), does this approach seem logical?.

4. In general if I use different transformations on independent variables such as log, squared, square root, and inverse, do I need to standardize the data also?

Here is the scatter plot:

Any feedback and insights would be highly appreciated. Thank you

Answer 1

I don't think "can" is the right question to ask; it's not going to give you a syntax error. The right question is "what could go wrong?". Any modeling technique will have assumptions that may be broken, and knowing how those assumptions impact the results will help you know what to look for (and how much to care when those assumptions are broken).

1. The best test of whether or not linearity is appropriate is whether the residuals are white or structured. For example, it looks like X9 might have a nonlinear relationship with Y. But that might be an artifact of the interaction between X9 and other variables, especially categorical variables. Fit your full model, then plot the residuals against X9 and see what it looks like.

2. Treating it as continuous won't cause serious problems, but you might want to think about what this implies. Is the relationship between 1 and 2 in the same direction and half the strength as the relationship between 2 and 4? If not, you might want to transform this to a scale where you do think the differences are linear.

3. Same as 2, except it's even more reasonable to see time as linear.

4. Standardization is not necessary for most linear regression techniques, as they contain their own standardization. The primary exception is techniques that use regularization, where the scale of the parameters is relevant.

It's also worth pointing out that multivariate linear relationships, while they can capture general trends well, are very poor at capturing logical trends. For example, looking at X3 and X4, it could very well be that there are rules like Y>X3 and Y>X4 in place, which is hinted at but not captured by linear regression.

Answer 2

Thanks Matthew. There is a confusion to me, you mean we’ll build the model through multiple linear regression method and then the residuals will be checked VS the significant factors to see if they are structureless. We shouldn’t first check whether or not the variables (coefficients) are linear with respect to y.

1. Since the X9 and X5 were not statistically significant, shall I check the scatter plot of residuals vs those two?

2. What are your insights about the scatter plot of residuals vs time, do you think it still needs any transformation? If yes, can you explain the reason?

Here are the scatterplot of residuals VS significant variables and different Residuals plots vs Y to check if the normality assumptions were met.

Please let me know your feedback. On the other hand, can you explain a bit more what your last paragraph meant (note that both X3 and X4 are statistically significant)? Thank you again.

Answer 3

In fact, I used additive method to calculate the X3 value (X3 is the summation of 21 binary variables) and multiplicative method to calculate X4 (X4 is the multiplication of 3 continues variables). I was wondering if this information will change the way of my calculation.

Here are the residuals plot versus X7, X5, and X9 plus probability plot of residuals for the X7 (time or year). It can be discovered that the residuals of X7 in each time horizon is normal.

However, I'm still confused about this point, you mean we’ll build the model through multiple linear regression and then the residuals will be checked VS the significant or none-significant factors to see if they are structureless and follow the normal distribution. We shouldn’t first check whether or not the variables (coefficients) are linear with respect to y.

Thank you for your feedback.