6
$\begingroup$

Suppose, for example, that the first search result on a page of Google search results is swapped with the second result. How much would this change the click-through probabilities of the two results? How much would its click-through probability drop if the fifth search result was swapped with the sixth?

Can we say something, with some level of assurance, about how expected click-through probabilities change if we do these types of pairwise swaps within pages of search results?

What we seek is a measure of the contribution to click-through rates made specifically by position bias.

Likely, how position ranking would affect the sales in Amazon or other online shopping website? If we cast the sales into two parts, the product quality and its ranking effect.

sales = alpha*quality + beta*position + epsilon

How can we quantify the beta?

Improve this question
$\endgroup$
1
  • $\begingroup$ Is it possible to run A/B tests in your situation? That could give you some training data, or even just a general idea. $\endgroup$
    – sheldonkreger
    Dec 12, 2014 at 21:54

3 Answers 3

Reset to default
2
$\begingroup$

you might want to look at this paper Predicting Clicks: Estimating the Click-Through Rate for New Ads

Whenever an ad is displayed on the search results page, it has some chance of being viewed by the user. The farther down the page an ad is displayed, the less likely it is to be viewed. As a simplification, we consider the probability that an ad is clicked on to be dependent on two factors: a) the probability that it is viewed, and b) the probability that it is clicked on, given that it is viewed:

Improve this answer
$\endgroup$
1
$\begingroup$

It's just logistic regression. Get a bunch of data about presentations of search results, along with whether an item was clicked on. An instance is a search result item, with possible features being rank, "quality" (not sure what you mean by this) etc. The what you're asking about is a question of inference on the parameter related to rank.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, Ben. The quality I mentioned, is the quality, or true worth, of a product in an shopping website. Actually, I'm going to measure/inference on its ranking itself, but try to identity two main factors towards the sales of a product: 1.quality(it that product good enough?) 2.ranking in page(a bias contributes to sales, but doesn't reflect product's quality) $\endgroup$
    – zihaolucky
    Oct 14, 2014 at 9:22
  • $\begingroup$ Sure---I think you can include whatever predictors are useful (and indeed doing so may improve your model's fit). The point is you should view it as a logistic regression problem and understand which parameters you're doing inference about $\endgroup$
    – Ben Allison
    Oct 14, 2014 at 15:42
1
$\begingroup$

In the way that you've defined or set up the problem, i.e.

sales = alpha*quality + beta*position + epsilon

We can easily quantify beta given that your model is correct. You just need to run it through linear regression and it will give you the coefficient for beta*.

If you would like to model click through rates, you would have to train a classifier. So you would have to fit a logistic model that models: 

clicks ~ alpha*quality + beta*position + epsilon

*I believe you would have to restrict the training set to contain results where all impressions were obtained on the first page otherwise your model will not hold (I would guess that beta is going to be strongly dependent on the page).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.