2
$\begingroup$

I have 10000 samples. There are 4 independent variables and 1 dependent variable.

The independent variables are all centered with 0 mean.

enter image description here

I found the correlation coefficients between each of these variables which are as below:

enter image description here

I used linear regression model and below is the summary of that model:

enter image description here

Now, based on the coefficients of the predictor variables in the linear regression model, I have been asked to find the significant predictor (s).

Based on just the correlation values, I was thinking X 4 will be the significant predictor but its regression coefficient says a different story altogether. (x4 has the least coefficient value in lm summary output). Can you help me understand what exactly is the correct way to identify the significant predictor (s)?

Also, Additionally even if I remove the x4 variable from the lm model, the Residual standard error remains the same which kinda re-iterates the fact the it is not a significant predictor? Is

my understanding correct here?

Also, I ran the VarImp function available in R which again returned a smaller value for x4.

> varImp(lm_df, scale = TRUE)
     Overall
x1 33.673993
x2 34.858260
x3 33.820908
x4  1.969445
$\endgroup$
2
  • $\begingroup$ what is the difference between linear model and regression model? $\endgroup$
    – Selvam
    Mar 18, 2020 at 7:03
  • $\begingroup$ Linear Regression...and in particular to determine whether some explanatory variables may have no linear relationship with the response at all,(see wikipaedia). $\endgroup$ Apr 2 at 9:31

3 Answers 3

2
$\begingroup$

The table of correlation coefficients shows the pairwise correlation between the variables in your data set: on a range from 0 (no correlation) to 1 (full correlation), to what extent does variation in one variable explain variation in the other variable?

The coefficients from the regression table, on the other hand, describe the relation between y and the different x's all else being equal. For example, the coefficient estimate of x1 tells you that, provided x2 to x4 are held constant, y is expected to change by .959 units when x1 changes by 1 unit. To understand the relation between your y and the different x's, these coefficients are usually more informative than pairwise correlations.

In addition to the strength of the effect (the site is the coefficients) which can be used to assess the substantive significance of a predictor variable, the regression table gives p-values that are used to judge the statistical significance of a predictor variable. A low p-value (conventionally below .05) suggests statistical significance (meaning a low probability of observing your data under the null hypothesis).

So looking at your regression output, you can see that all four predictor variables have a statistically significant relation with y. X1 to x3 are more significant than x4, both in terms of statistical and substantive significance. The contrast between the strong pairwise correlation between x4 and y and the small regression coefficient is due to multicollinearity between the x's: x4 covaries with the other predictors, and it is this covariance which accounts for the correlation with y.

$\endgroup$
9
  • 1
    $\begingroup$ Exactly, because of multicollinearity. X4 is correlated with the other predictors, and the correlation between x4 and y is mainly because of this. $\endgroup$
    – Fabian
    Feb 25, 2020 at 8:11
  • 1
    $\begingroup$ I added this to my answer $\endgroup$
    – Fabian
    Feb 25, 2020 at 8:15
  • 1
    $\begingroup$ Hi Selvam, since x4 is statistically significant, most people would probably advise you to keep it in the model, even if it does not add a lot of explanatory power. On the other hand, if for some reason you are interested in having a simple model (in the real world, this could happen if acquiring the data is costly, for example), you might choose to omit x4. $\endgroup$
    – Fabian
    Feb 25, 2020 at 20:00
  • 1
    $\begingroup$ Regarding how is it possible that x4 is the least significant predictor, even if it is strongly correlated with y: since the x's are not perfectly correlated with each other, the model can estimate how y changes when x4 changes, but x1 to x3 stay constant. Imagine comparing two observation that only differ with regard to x4, not x1 to x3. $\endgroup$
    – Fabian
    Feb 25, 2020 at 20:10
  • 1
    $\begingroup$ The point is that if you'd compare a number of observations that only differed regarding x4 (and not regarding the other x's), you would find that y hardly changes with x4. The fact that you see a strong correlation between x4 and y is because most of the time, variation in x4 goes along with variation in x1 to x3. $\endgroup$
    – Fabian
    Feb 25, 2020 at 20:22
0
$\begingroup$

The correlation coefficient(.58 ) between the two variables (X4 and y is significant statistically given a very large sample. This positive relationship is corroborated by the multiple-regression model

results. Your model produced a t-value = 1.969 which is statistically significant at alpha = .05. It is actually significant at .0489 (see pr more than t )

"Based on just the correlation values, I was thinking x4 will be the significant predictor but the predictor coefficient in multiple linear regression says a different story altogether. (x4 has the least coefficient value in lm summary output."
Ans: The way you interpret the regression coefficient is completely naive and incorrect. An absolute value of regression coefficient indicates the effect-size and not t-statistic. It is the t- statistics that reflects an inference value.

$\endgroup$
0
0
$\begingroup$

Background: The case study revolves around a financial analyst, Sarah, working for a renowned investment bank. Sarah is tasked with developing an econometric model to predict stock market returns and identify potential investment opportunities. She decides to use a multiple linear regression model, considering various macroeconomic and company-specific variables. The dataset includes quarterly data from 2001 to 2020. Variables: Dependent Variable: Stock Market Returns (SMR) Independent Variables: Gross Domestic Product (GDP) Growth Rate Inflation Rate Interest Rate Corporate Earnings (CE) Market Volatility (VIX) Model: SMR = β0 + β1GDP Growth Rate + β2Inflation Rate + β3Interest Rate + β4Corporate Earnings + β5*Market Volatility + ε Sarah conducts a series of diagnostic tests, including checking for multicollinearity, autocorrelation, and heteroskedasticity, and ensures that the model's assumptions are met. After refining her model, Sarah finds that all variables, except for Inflation Rate, are significant. Questions:

  1. Interpretation of Coefficients: How should Sarah interpret the coefficients of the significant variables in her model? What does the sign and magnitude of each coefficient imply about its relationship with stock market returns?
  2. Model Assumptions and Diagnostics: Discuss the importance of checking for multicollinearity, autocorrelation, and heteroskedasticity in Sarah's econometric model. What potential issues could arise if these assumptions are violated, and how might they impact her predictions?
  3. Inflation Rate’s Insignificance: Given that the Inflation Rate turned out to be insignificant in predicting stock market returns in Sarah’s model, what could be the potential reasons behind this? How might she improve her model or investigation to better understand this relationship?
  4. Out-of-Sample Forecasting: Sarah intends to use her model for out-of-sample forecasting to predict future stock market returns. What challenges might she face in this endeavor, and how can she assess the accuracy and reliability of her forecasts?
  5. Model Complexity and Simplicity: Sarah’s model includes both macroeconomic and company-specific variables. Discuss the trade-offs between having a more complex model with numerous variables versus a simpler model. How might the inclusion of more variables impact the model’s performance, interpretability, and the risk of overfitting?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.