1
$\begingroup$

I'm obscuring the data I'm working on, but I think this should get the point across. I'm trying to design a test for the following:

A client sells various popsicles. They think that on days when it is very hot, one particular kind of popsicle sells more than on days when it's not hot. How can I prove if he's correct or not?

My thought is that I could do an A/B test, splitting up between not hot & hot days. For each of those groups, I could find the proportion of sales for that popsicle from all popsicles sold that day. If that proportion rises on hot days, then I can do a test for statistical significance and come to a conclusion.

But I think there have to be other tests/models I can apply based on sales counts alone right? What tests can I run that only involve things like the raw counts of sales on days when it was hot and not? Note that I am planning to use the temperature as a categorical variable but I can also make it continuous if that makes more sense.

$\endgroup$
1
$\begingroup$

I think you need to start by thinking very carefully about what exactly your client wants to know, because there are two different things you could be testing:

  • Is each individual customer is more likely to buy a popsicle of the designated type on a hot day than on a cool day?
  • Is the client likely to sell more popsicles of the designated type on hot days than on cool days?

For the first one, you would analyze proportions; for the second, you would analyze absolute sales counts for the designated type of popsicle, and in fact you could probably ignore sales of all other types of popsicles.

If that latter question is the one you want to analyze, I do think running a regression analysis on sales of the particular type of popsicle vs temperature (both as continuous variables) is a reasonable place to start. In fact you could start by just plotting them and looking at the graph, to see if any trend (linear, logarithmic, logistic, etc.) jumps out at you. I wouldn't go for A/B testing right off the bat because you're not dealing with a categorical variable, and there's not a single natural way to convert it.

Of course it could make sense to do that conversion, but in doing so, I'd suggest looking at overall sales figures and other indicators of whether people perceive a day to be hot or cool to try to pin down the boundaries. Something like logistic regression (possibly multinomial) might come in handy, though I could easily see it giving worthless results if the data aren't clean enough. Perhaps you could exploit a clustering algorithm to identify groups of temperatures that have similar sales figures and use that to determine your categories. You could also try to correlate external data sets like community pool attendance, ice cream sales, electricity usage (for air conditioners), etc., if those are available, to further inform your analysis of what temperature class each day would fall in. (Maybe that's overkill, but it's an idea at least.)


Incidentally, A/B testing typically designates a process where you control the assignment of the classes A and B. Like in web design, you choose (or program a computer to choose, perhaps at random) whether each visitor gets version A or version B. You don't get to choose whether each day is hot or not, so it's not quite the same thing. That being said, you can probably use most of the same methods of analysis you would on a proper A/B test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.