def get_pvalue(con_conv, test_conv,con_size, test_size,):
lift = - abs(test_conv - con_conv)
scale_one = con_conv * (1 - con_conv) * (1 / con_size)
scale_two = test_conv * (1 - test_conv) * (1 / test_size)
scale_val = (scale_one + scale_two)**0.5
p_value = 2 * stats.norm.cdf(lift, loc = 0, scale = scale_val )
return p_value
I have this function and I would like to know what it is actually doing and how it is actually calculating the p-value.
This is to find the difference between the conversion rate of control and test and group from an A/B test.
con_conv --> Conversion rate for control group
test_conv --> Conversion rate for test group
con_size --> population size for control group
test_size --> population size for test group
I understand that scale_one
and scale_two
are calculating the variance for each group, but I don't understand why they are adding both of them to calculate the standard deviation and why they are multiplying the cdf with 2 to get the p_value
.