In Bag of Tricks for Efficient Text Classification paper which is popular right now, he calculates prec@1 for the datasets in the experimentation segment. What does that mean?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ en.wikipedia.org/wiki/Information_retrieval#Precision_at_K $\endgroup$– EmreCommented Aug 9, 2016 at 17:37
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Its "Precision at 1", or how often the highest ranked document is relevant:
http://ir-ratio.blogspot.co.uk/2012/03/precision-at-1-and-reciprocal-rank.html
Suppose you are looking for items about monkeys. Your query engine queries documents for "monkeys" and ranks by relevance. If the highest ranked document is indeed about monkeys, then that's a win for your query algorithm. But if the highest ranked document is ranked 1 because it has the text "Enough of your monkey business" then its a loss, because that's not really about monkeys.
Repeat over a bunch of search terms. The Precision-at-one is then the number of wins over the total number of search terms tried.
-
$\begingroup$ So essentially, in this paper (for sentiment analysis) it just means simple precision? $\endgroup$ Commented Aug 10, 2016 at 4:29
-
-
$\begingroup$ Sorry for that. I mean prec@1 is essentially the precision i'll obtain if I use scikit's metrics library and calculate for multiclass classification? $\endgroup$ Commented Aug 11, 2016 at 9:03