From a behavioral study data was extracted. The study was about how people change their eating behavior, following visual cues. There were to groups of people: One was shown visual cues and then it was recorded what they chose to eat and the other group was just shown random stuff or nothing and then it was recorded was they chose to eat. This experiment was repeated with the same people a large number of occasions, at different times of day. I have a dataset that precisely recordes personal information about each individual as well as their eating behaviour during each time the experiment was carried out.

The goal is to predict whether the visual cue will make someone eat the stuff presented in the cue or not. The problem is that it is unclear to me how to separate the influence of the treatment from the behavior that they might have shown anyway. I.e., suppose a person sees an image of a cake and then, when he is presented with a variety of different foods, eats cake. How can we know that it was actually the image that influenced him and that he did not wanted to eat cake anyway, so the image change actually nothing?

Thus I can't directly treat it as a binary classification and define a categorical feature, where I assign "1" if someone ate what was in the picture and "0" if he didn't, since by that way I may also identify those who wanted to eat what was in the cue before it was shown to them. How can I solve this problem?


I think you can approach it as a logistic regression problem, where you will have a on/off (1/0) feature called "exposure_to_image". And your goal will be to detect if that coefficient is statistically significantly different than 0. If yes, then the probability of eating the item was influenced by the exposure.

As for the assumption that they would have eaten it anyway: That's the whole point of having the control group. So your dataset will have people where exposure_to_image=1, and people for who exposure_to_image=0. All of them will have some baseline probability of wanting to eat the cake. But that coefficient tells you by how much that baseline is affected.

  • $\begingroup$ +1. But before I accept, the following is still unclear to me: I understand, as a human, how control groups work and if in the group of the people who were shown the visual significantly more ate cake, then then the visual cue must be the cause. But I'm not sure that if I'll use a binary classifier with a target feature "exposure_to_image" as described (1 if the subject eats cake, 0 else), that the binary classifier will also be able to figure this out. $\endgroup$ – user47580 Oct 14 '18 at 20:01
  • $\begingroup$ Wouldn't I rather have needed to define the target feature "exposure_to_image" a bit differently, namely as 1 if a subject ate cake and we know he would not have eaten it otherwise, 0 else, to make sure the binary classifier can make this distinction and exclude the people who anyway would have eaten cake? (Though I have no idea how to define formally this more elaborate classifier.) $\endgroup$ – user47580 Oct 14 '18 at 20:08
  • $\begingroup$ Also, are you sure a linear classifier like logistic regression is enough - and I would not need something like SVMs (with probability outputs), or some other more nonlinear model, in order to capture potential, complex scenarios (rules) like "young males, who are orphans are likely to respond to the image, and old males who have looked at the image more than 20 seconds as well"? $\endgroup$ – user47580 Oct 14 '18 at 20:08
  • $\begingroup$ You cannot know, through the experiment, what the intent of each individual person was, or whether they would have eaten the cake otherwise. You are trying to find the overall influence of the picture - a statistical description of the effect. f.g.'s answer has it right. You can use logistic regression with a set of categorical variables. I wouldn't worry about using SVM until you've inspected the accuracy of the regression and done a bit of graphing. If there's obvious residual patterns, then start looking at a more complicated model like SVM. $\endgroup$ – Ingolifs Oct 15 '18 at 5:00
  • $\begingroup$ @Ingolifs But doesn't bellmaneqn's answer seem to suggest otherwise - i.e. that I should first estimate the background probability that a participant will eat cake on a random day and then use that number as an additional input feature? $\endgroup$ – user47580 Oct 15 '18 at 7:13

What you are trying to estimate is the average treatment effect (ATE), i.e. the average effect of showing a random sample of people a picture of a cake on the likelihood that they choose to eat a cake later. There will be people who will eat cake anyway and people who will never eat cake in both treatment and control groups if your group assignment is sufficiently randomized. But that is OK for ATE, which is the effect (or no effect) you should expect to see when you take the treatment to a larger general population.

The confound you are concerned about fundamentally change your research question. If you want to know the treatment effect of showing a picture of cake to people who will not eat cake, on the likelihood that they choose to eat cake later, you will need more control. One way I can think of is to study a well separated survey on participants' diet routine to see how likely each participant will eat a cake in any random day. Include that as a control variable on the RHS of your regression model could potentially be helpful. However, there is no perfect control for anything. You need to make assumption and build argument around it.

I think you will see very similar estimation results from linear regression and logistic regression (in terms of the relative size and direction of the coefficients). What I have seen in my field (social science) is that linear regression (or linear probability model, LPM) is often preferred due to its ease of interpretation. It's always a pain to explain in odds ratio for a logistic regression model.

Linear model rules most research fields where interpretation is more important than anything else. You can fit your data with sophisticated, nonlinear model to any arbitrarily degree. But when you require interpretation, or more importantly causal interpretation, linear model is your best chance.

  • $\begingroup$ This is a a really good answer, thanks! But I was a bit lost on some terminology you used, because I'm not familiar with social science vocabulary: When you say "The confound you are concerned with" you mean the target feature I'm concerned with? Similarly, when you say "include that as a control variable on the RHS", you mean to use the feature, containing estimated probability for each participant he will eat cake on a random day, as another input feature of my linear model? $\endgroup$ – user47580 Oct 15 '18 at 7:11
  • $\begingroup$ By "the confound" I am referring to your concern that people who will not eat cake anyway will not eat cake no matter what you show them. You need to be upfront with your research question first. In your original post you have in fact raised two different research questions. $\endgroup$ – BellmanEqn Oct 15 '18 at 12:33
  • $\begingroup$ A control variable is just another X in your model, or as you prefer, another "feature". There are some assumptions that must be justified for every control so your estimate of the coefficient for the variable of interest (i.e. "showing a picture of a cake or not") is unbiased. If you are really into this, you should pick up a book on Econometrics. $\endgroup$ – BellmanEqn Oct 15 '18 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.