In backward elimination, I heard the steps of fitting the model by keep removing the highest p-value(a.k.a. insignificant independent variable) each time like below
- Select a significance level to stay in the model(e.g. SL = 0.05)
- Fit the full model with all possible predictors
- Consider the predictor with the highest P-Value(P > SL)
- Remove the predictor
- Fit model without this variable (Repeat step 3-5 until P <= SL)
But the part which I don't get is why is having higher p-value makes the corresponding independent variable insignificant. Doesn't having high p-value mean it's more close to the null hypothesis so that that variable is more significant?