Let me define the problem space.

I am working a binary classification problem. I am trying to build a causal model as well as predictive model.

My aim is to find list of significant features (based on causal model) and use that to build a predictive model. I did refer the suggestions provided in this post and it was very much useful but I have few more questions due to my limitations with ML field.

I understood from literature that there are two ways to adjust/control for confounders. one is through study design phase and other is during modelling/analysis phase.

As I am working on retrospective data analysis, I can only adjust confounders during analysis phase.

We know that certain features like Age in a typical example like "gender causes heart disease" is a confounder.

1) So during analysis phase, we include age as a variable in our model. Similarly all the potential confounders that we could think of are put in the model as features. ex: X_train will have all the columns/features that I think of as potential confounders and then it is fed to the model (logistic regression). Am I right till here?

2) Does this mean our LR model is adjusted for confounders? How would you do confounding adjustment during logistic regression modelling phase? If we include all potential confounders in our model and if the coeff of already existing variables (gender) change by 10% or so, I understand that age is a confounder but does this also mean that LR is adjusted for confounders?

3) Then, why it is said that logistic regression doesn't consider feature interaction? Is feature interaction different from confounding? I understand that feature interaction is usually denoted as gender*age but does this mean both variables work together to influence the outcome? doesn't confounder mean the same?

4) What's the usefulness of having interaction variables? I mean if gender*age impacts the outcome, can I understand that gender (individually) and age(individually) impact the outcome?

5) I see that people usually create 2x2 tables called as strata for stratified analysis and compute risk ratio and compare it with crude risk ratio. But how can we do this for all variables which I think as confounders in my dataset? I know we can use tools like SPSS, STATA etc but is it the only way to do? But then can't we do using multivariate regression?

6) Is it mandatory that all our continuous variables be converted into some categorical variable for analysis/confounder adjustment?

7) Any simple examples/explanation would be helpful as I couldn't find any tutorial for adjusting confounding during logistic regression and finding significant variables. I have been referring this though it's useful, some links are broken. lot of questions arise because I am neither a stats or biostats person. I usually build models using classic ML algorithms and now trying to learn all this.


1 Answer 1


Some remarks (as far as I understand your questions):

  1. In a causal model you need to reflect the "data generating process" (DGP). The DGP is a theoretical construct. You need to come up with an idea what is relevant for your research question, so what $X$ explain $y$ in a causal way. You can also include variables which are not so important. In tendence underspecification (exluding important variables) is a real problem. Overspecification can also be a problem but the consequences are less salient.
  2. I think the wording is unclear here. In some fields "adjusting for confounders" means to "control" for all relevant variables (aka. confounders). First thing to do: get the wording right and understand what you mean when you say "confounders".
  3. Where does this claim come from ("feature interaction"). In general, Logit is a model that is linear in the way you include variables, so if you think about interaction of two or more $x$ in your model, then no: Logit does not consider interaction unless you specify your model so to include interaction terms.
  4. An easy way of thingking about interactions is the interaction of a continuous variable $x_1$ with a dummy variable (1 or 0) $x_2$. In the model $$y=\beta_0+\beta_1 x_1$$ you only have one intercept ($\beta_0$) and one slope ($\beta_1$). In a model with interaction between $x_1$ and $x_2$ you have $$ y=\beta_0+\beta_1 x_1 + \beta_2 x_2$$ where there is the same slope for $x_1$ but a different intercept in case $x_2=1$. Another way to think about interactions is to add "squared" terms because a squared term is just the interaction $x_1 * x_1$.
  5. I don't understand this: needs a reference! I guess you speak about the inclusion of indicator variables (aka dummies) as interactions??!
  6. No you can use any numbers as $x$.
  7. Look here and read this book.
  • $\begingroup$ Hi, thanks for the response. Upvoted. regarding point 2, yes, I am aware and understand what does confounder mean. I have given an example of Age. I read in literature that few ways to control confounders. One is through startified analysis and other is through multivariate regression. In my case, am using multivariate regression, so is it only about putting all variables (which will include confounders as well) in a regression model which is known as confounding adjustment through regression? $\endgroup$
    – The Great
    Commented Dec 31, 2019 at 0:23
  • $\begingroup$ Regarding point 5, I meant this approach to adjust confounders. You can refer this link sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/… My question here was that, this is not just the only way to adjust confounders and we can do it using multiple regression as well. Am I right? $\endgroup$
    – The Great
    Commented Dec 31, 2019 at 7:34
  • $\begingroup$ w.r.t point 4, by interaction what I was interested to know was that, none of the algorithms has this as their property. Am I right? I mean it has to be done by me to select 2/3/4 features and decides to multiply/add/subtract etc. This is called creating interaction variables. Am I right? There is no algorithm that does this by itself. Or is there any ML model that considers interactions between variables by default? $\endgroup$
    – The Great
    Commented Dec 31, 2019 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.