I asked a data science question regarding how to decide on the best variation of a split test on the Statistics section of StackExchange. I hope I will have better luck here. The question is basically, "Why is mean revenue per user the best metric to make your decision on in a split test?"
The original question is here: https://stats.stackexchange.com/questions/107599/better-estimator-of-expected-sum-than-mean
Since it was not well received/understood I simplified the problem to a discrete set of purchases and phrased it as a classical probability problem. That question is here: https://stats.stackexchange.com/questions/107848/drawing-numbered-balls-from-an-urn
The mean may be the best metric for such a decision but I am not convinced. We often have a lot of prior information so a Bayesian method would likely improve our estimates. I realize that this is a difficult question but Data Scientists are doing such split tests everyday.