3
$\begingroup$

Background to the problem: I am estimating individual treatment effects using double machine learning model. I do not know true treatment effects for my problem.

Double ML: Given Y (outcome), T (treatment) and X ( features)

Y = aT + bX + error

coefficient a is of interest (measures treatment effect) .

Double ML procedure:

  1. Fit Y ~ X => Compute residuals (Y* = Y – Y’) – Residuals are treatment effects to be estimated
  2. Fit T ~ X => Compute residuals (T* = T- T’) – This model captures variation in T explained by X
  3. Fitting a model (Y* ~ T* ) on residuals will give the average treatment effects

I am fitting a linear regression model (Y* ~ T* ) and none of the coefficients are statistically significant. Instead of relying on point estimates, I am computing prediction confidence intervals and p-value to check if the predicted value is statistically significant or not.

Is this approach good?

$\endgroup$
4
  • $\begingroup$ Is the model overall significant? And I don’t follow how you’re proceeding with prediction confidence intervals (sounds oxymoronic) and p-values of the predicted values. Please elaborate, $\endgroup$
    – Dave
    Jun 7, 2020 at 2:05
  • $\begingroup$ I updated my question with more details $\endgroup$
    – Chandra
    Jun 7, 2020 at 3:26
  • $\begingroup$ Have you considered variable interactions? Are T and X correlated? Have you tried any transformations? $\endgroup$
    – bstrain
    Jun 24, 2020 at 16:38
  • $\begingroup$ Show the full regression outcome, including all statistics $\endgroup$
    – Peter
    Aug 23, 2020 at 21:48

1 Answer 1

-1
$\begingroup$

It depends on the goal of the project.

Statistical significance is more important for publishing in academic journal. For applied projects, statistical significance is less important.

Even if a model is not statistical significant, it can have value. The model may summarize the data and identify relationships. The model maybe practically important, the effect maybe of value to practitioners.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.