Statistical comparison of 2 small data sets for 2X increase in the population mean

Question

I am trying to determine whether or not we are 90% confident that the mean of a proposed population is at least 2 times that of the mean of the incumbant population based on samples from each population which is all the data I have right now. Here are the data.

incumbantvalues = (7.3, 8.4, 8.4, 8.5, 8.7, 9.1, 9.8, 11.0, 11.1, 11.9)

proposedvalues = (17.3, 17.9, 19.2, 20.3, 20.5, 20.6, 21.1, 21.2, 21.3, 21.7)

I have no idea if either population is or will be normal.

The ratio of the sample means does exceed 2.0 but how does that translate to confidence that the proposed population mean will be at least twice that of the mean of the incumbant population with 90% confidence ?

Can re-sampling (bootstrapping with replacement) help answer this question ?

Answer 1

Yes, in principle, resampling can help answer this question.

incumbent <- c(7.3, 8.4, 8.4, 8.5, 8.7, 9.1, 9.8, 11.0, 11.1, 11.9)
proposed  <- c(17.3, 17.9, 19.2, 20.3, 20.5, 20.6, 21.1, 21.2, 21.3, 21.7)

set.seed(42)

M  <- 2000
rs <- double(M)

for (i in 1:M) {
    rs[i] <- mean(sample(proposed, replace=T)) - 2 * mean(sample(incumbent, replace=T))
}

To make the assessment, you should choose one (not both) of the following:

A. The (two-tailed) 90% confidence interval for the difference in the (weighted) means using Hall's method is:

ci.hall <- 2 * (mean(proposed)-2*mean(incumbent)) - rev(quantile(rs,prob=c(0.05, 0.95)))
names(ci.hall) <- rev(names(ci.hall))
ci.hall

   5%   95% 
-0.29  2.95 

This is appropriate if you have any concern about missing the possibility that mean(proposed) might actually be less than 2 * mean(incumbent).

B. The proportion of resample means >= 0 provides the (one-tailed) estimate that mean(proposed) is at least twice mean(incumbent):

sum(rs>=0)/M

[1] 0.8915

The problem is that the samples are really rather small and resampling estimates can be unstable for small n. The same issue applies if you want to assess normality and go with parametric comparisons.

If you can get to, say, n >= 30, the approach described here should be fine.

Answer 2

Here is what I programmed within a loop.

1. randomly take 10 values (with replacement) from the incumbant sample, determine its mean
2. randomly take 10 values (with replacement) from the proposed sample, determine its mean
3. form the ratio of the above two means and append it to a master list
4. repeat steps 1 thru 3 many times (I chose 1 million)
5. % Confidence=(number of ratios that equal or exceed 2.0/1000000)*100

Results: Exactly 897450 ratios were found to be greater than or equal to 2.0, producing a confidence of 89.745%.

Conclusion: We are less than 90% confident that the proposed population will have a mean at least twice that of the incumbant population.