1
$\begingroup$

I have a general question of unbalanced data. I'm performing a T test on Group A and Group B. Group A has 20 data while Group B has 500. I set unequal variance (Welch) for the adjustment and the P-value is 0.01. Can I conclude that there is significance?

I have another idea that I'm not sure about. I randomly select about 20 samples from Group B multiple times and test against the Group A. Should I do some adjustment for the result? I am not sure if this is works or not. If it works, what kind of adjustment I should use for multi-testing?

Thank you!

I appreciate any replies!

$\endgroup$

2 Answers 2

2
$\begingroup$

it does not have anything to do with whether t test or welch t test. As long as your group A samples estimate the mean and standard deviation of population A very well, and group B samples estimate the mean and standard deviation of population B very well, then there shouldn't be any problem. If the SD of population A is assumed different from that of population B, then you better go welch t test. Your question may be better asked at cross-validated, who cares more statistics, I think.

$\endgroup$
0
2
$\begingroup$

It sounds like you are using Welch's t-test to compare groups of different sizes. That should work in your situation. It's more reliable that the standard Student's t-test when you have unequal variances and unequal sample sizes.

Note that sample sizes do not have to be identical for a Student's t-test. The important thing is that the variances are the same. If the variances are not equal, this is where the extra robustness of Welch's becomes helpful.

Whether or not you can conclude significance depends on what P-value you are you are looking for. It sounds like you are targeting p<.01, so if your result is less than .01, you can report that you found a significant difference.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.