Exploration vs exploitation tradeoff to find a price that maximizes revenue

Question

Is there a practical strategy that can learn to price a product optimally? Right now I have the following arbitrary hill-climbing algorithm:

• Run an experiment at starting price P and gather 500 data points (e.x. 20 buy and 480 not buy).
• Run a t-test on what confidence level P yields higher revenue per visitor than P * 1.1 and P * 0.9. Then do a 3-way weighted coin-flip and the winner gets to run the next experiment.

There's many problems with this approach. For example, if price is at optimal, it can't price a product at a more optimal pricing e.x. P * 1.03. Another is that if at some price point P = K we happen to get really unlucky and get 1 buy of 500 data points, the algorithm won't converge fast.

The problem gets easy if we take lots of data points but that would reduce long term revenue. Is there a fast algorithm that can converge to the optimal price and then not do anymore exploration?

Answer 1

Without making any underlying assumptions you will not get anywhere. That said, there are multi-arm bandit strategies that try to optimize the rewards, there is a ton of research on this field. It comes down to sampling from a distribution of your options (in your case two) and adapting this distribution based on the rewards.

https://en.wikipedia.org/wiki/Multi-armed_bandit

Once you know that the reward distribution from each bandit comes from a specific distribution, you can deduce optimal sampling strategies. Once you have at least some prior information you can do fairly well although not always optimal. Regardless, most strategies will do better than normal A/B testing if the strategy is not super greedy.