1
$\begingroup$

In the data, there are 355 observations including one continuous dependent variable (Y: ranges from 15-55) and 12 independent variables (continuous, categorical, and ordinal). The X1 (2 levels) and X6 (3 levels) are considered as categorical variables. Here are some questions that I have:

  1. Can I assume that all the coefficients (except X1 and X6 which are categorical) are linear with respect to Y?

  2. Can I consider X5 as continuous variable; however, it is ordinal and ranges from (1-7)?

  3. Can I get the X7 (year) as continuous variable; however, it’s ordinal and rages from 2002-2006 (In fact, year of data per se does not improve the response; it is the other factors occurring in the same time period which result in improvements and we don’t know those factors), does this approach seem logical?.

  4. In general if I use different transformations on independent variables such as log, squared, square root, and inverse, do I need to standardize the data also?

Here is the scatter plot:

enter image description here

Any feedback and insights would be highly appreciated. Thank you

$\endgroup$
2
$\begingroup$

I don't think "can" is the right question to ask; it's not going to give you a syntax error. The right question is "what could go wrong?". Any modeling technique will have assumptions that may be broken, and knowing how those assumptions impact the results will help you know what to look for (and how much to care when those assumptions are broken).

  1. The best test of whether or not linearity is appropriate is whether the residuals are white or structured. For example, it looks like X9 might have a nonlinear relationship with Y. But that might be an artifact of the interaction between X9 and other variables, especially categorical variables. Fit your full model, then plot the residuals against X9 and see what it looks like.

  2. Treating it as continuous won't cause serious problems, but you might want to think about what this implies. Is the relationship between 1 and 2 in the same direction and half the strength as the relationship between 2 and 4? If not, you might want to transform this to a scale where you do think the differences are linear.

  3. Same as 2, except it's even more reasonable to see time as linear.

  4. Standardization is not necessary for most linear regression techniques, as they contain their own standardization. The primary exception is techniques that use regularization, where the scale of the parameters is relevant.

It's also worth pointing out that multivariate linear relationships, while they can capture general trends well, are very poor at capturing logical trends. For example, looking at X3 and X4, it could very well be that there are rules like Y>X3 and Y>X4 in place, which is hinted at but not captured by linear regression.

$\endgroup$
0
$\begingroup$

Thanks Matthew. There is a confusion to me, you mean we’ll build the model through multiple linear regression method and then the residuals will be checked VS the significant factors to see if they are structureless. We shouldn’t first check whether or not the variables (coefficients) are linear with respect to y.

  1. Since the X9 and X5 were not statistically significant, shall I check the scatter plot of residuals vs those two?

  2. What are your insights about the scatter plot of residuals vs time, do you think it still needs any transformation? If yes, can you explain the reason?

Here are the scatterplot of residuals VS significant variables and different Residuals plots vs Y to check if the normality assumptions were met.

enter image description here

enter image description here

Please let me know your feedback. On the other hand, can you explain a bit more what your last paragraph meant (note that both X3 and X4 are statistically significant)? Thank you again.

$\endgroup$
  • $\begingroup$ If you know there's a more reasonable relationship than linear--maybe you expect a factor to have a multiplicative effect instead of an additive effect--then start off by transforming that factor. But if you're uncertain, I'd start with linear and work up to transformations if the multivariate fit isn't capturing the underlying structure. $\endgroup$ – Matthew Graves Oct 10 '15 at 14:50
  • $\begingroup$ 1. There isn't a statistically significant linear relationship, but there very well could be signfiicant nonlinear relationships. I'd check the residuals anyway. $\endgroup$ – Matthew Graves Oct 10 '15 at 14:54
  • $\begingroup$ 2. The residual plot of X7 (I assume that's what you meant by time) looks fine to me, but a scatter plot is not the most informative plot when you have non-continuous values like that. Can you easily generate a prob plot of resi, grouped by X7 value? $\endgroup$ – Matthew Graves Oct 10 '15 at 14:54
0
$\begingroup$

In fact, I used additive method to calculate the X3 value (X3 is the summation of 21 binary variables) and multiplicative method to calculate X4 (X4 is the multiplication of 3 continues variables). I was wondering if this information will change the way of my calculation.

Here are the residuals plot versus X7, X5, and X9 plus probability plot of residuals for the X7 (time or year). It can be discovered that the residuals of X7 in each time horizon is normal.

enter image description here

enter image description here

However, I'm still confused about this point, you mean we’ll build the model through multiple linear regression and then the residuals will be checked VS the significant or none-significant factors to see if they are structureless and follow the normal distribution. We shouldn’t first check whether or not the variables (coefficients) are linear with respect to y.

Thank you for your feedback.

$\endgroup$
  • $\begingroup$ @ Matthew. Please let me know your feedback. Thanks. $\endgroup$ – Amir Oct 12 '15 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.