4
$\begingroup$

Let's say I want to measure a medicine’s impact on height. So I randomly broke down my user base into two groups (control and experiment, obviously). I calculated the average height for both groups before experiment and found out there’s a 2 cm difference. Then, after the medicine is applied to the experiment group, I found a 4 cm difference.

If I recall correctly, the t-test doesn’t handle selection bias, is there a test I can do to put the initial 2 cm difference between group into consideration?

Improve this question
$\endgroup$
3
  • $\begingroup$ Welcome to the site, Helene :) $\endgroup$
    – Dawny33
    Commented May 30, 2016 at 6:02
  • $\begingroup$ have you looked at chi square? or Fisher's exact test? $\endgroup$
    – Brandon Loudermilk
    Commented May 30, 2016 at 13:38
  • $\begingroup$ @BrandonLoudermilk the chi square test only applies for frequency / counted data, not to measured means. The t-test would be the correct choice to compare the 4cm difference. I'm not sure how to incorporate the initial 2cm difference though.. $\endgroup$
    – stmax
    Commented Jun 1, 2016 at 8:03

1 Answer 1

Reset to default
2
$\begingroup$

As far as I understand you simply don't have to worry about that.

So, you have a sample split randomly in control and treatment. You measure something before treatment, and after, for the same individuals. Because you measure on the same individuals than you have paired measures: delta = before - after. You are interested to measure if the mean of delta for control sample is significantly different than the mean of delta for control sample. This is done with paired sample test known also as dependent test.

If you assume a normal distribution you can use paired t test. If you can't reasonably assume the normal distribution than you can use Wilcoxon signed rank test.

The idea of pairings in test is to eliminate the effect of confounders. What you computed can be the effect of a confounder, but if the sample was split randomly in control and treatment you should not worry about that, unless you have strong reasons to doubt the randomization procedure. In the later case you perhaps should include confounders in the equation and take the route of random effects modeling.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.