I've just started reading about AB testing, as it pertains to optimizing website design. I find it interesting that most of the methods assume that changes to the layout and appearance are independent of each other. I understand that the most common method of optimization is the 'multi-armed bandit' procedure. While I grasp the concept of it, it seems to ignore the fact that changes (changes to the website in this case) are not independent to each other.

For example, if company is testing the placement and color of the logo on the website, they find the optimal color first then the optimal placement. Not that I'm some expert on human psychology, but shouldn't these be related? Can the multi-armed bandit method be efficiently used in this case or more complicated cases?

My first instinct is to say no. On that note, why haven't people used heuristic algorithms to optimize over complicated AB testing sample spaces? For an example, I thought someone might have used a genetic algorithm to optimize a website layout, but I can find no examples of something like this out there. This leads me to believe that I'm missing something important in my understanding of AB testing as it applies to website optimization.

Why isn't heuristic optimization used on more complicated websites?


1 Answer 1


If I understand you question correctly, there are two reasons why genetic algorithm might not a good idea for optimizing website features:

1) Feedback data is coming in too slow, say once a day, genetic algorithm might take a while to converge.

2) In the process of testing genetic algorithm will probably come up with combinations that are 'strange' and that might not be the risk the company wants to take.

  • $\begingroup$ Thanks! I thought of #1, but maybe that there was a highly visited site that might be doing it. And you're right about #2- heuristic optimization is not guaranteed to come up with a interpretable solution. $\endgroup$
    – nfmcclure
    Commented Aug 25, 2014 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.