|
|
| Line 1: |
Line 1: |
| {{Refimprove|date=June 2007}}
| | Golda is what's written on my birth certificate even though it is not the name on my birth certification. Invoicing is what I do. For years he's been residing in Mississippi and he doesn't strategy on altering it. To climb is some thing I truly enjoy performing.<br><br>Feel free to surf to my site; [http://cpacs.org/index.php?document_srl=90091&mid=board_zTGg26 love psychics] |
| '''Mean reciprocal rank''' is a [[statistic]] measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by probability of correctness. The reciprocal rank of a query response is the [[multiplicative inverse]] of the rank of the first correct answer. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q:<ref>{{cite conference | title=Proceedings of the 8th Text Retrieval Conference | booktitle=TREC-8 Question Answering Track Report | author=E.M. Voorhees |year=1999 | pages=77–82}}</ref>
| |
| | |
| :<math> \text{MRR} = \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{\text{rank}_i}. \!</math>
| |
| | |
| The reciprocal value of the mean reciprocal rank corresponds to the [[harmonic mean]] of the ranks.
| |
| | |
| == Example ==
| |
| For example, suppose we have the following three sample queries for a system that tries to translate English words to their plurals. In each case, the system makes three guesses, with the first one being the one it thinks is most likely correct:
| |
| | |
| {| class="wikitable"
| |
| |-
| |
| ! Query
| |
| ! Results
| |
| ! Correct response
| |
| ! Rank
| |
| ! Reciprocal rank
| |
| |-
| |
| | cat
| |
| | catten, cati, '''cats'''
| |
| | cats
| |
| | 3
| |
| | 1/3
| |
| |-
| |
| | torus
| |
| | torii, '''tori''', toruses
| |
| | tori
| |
| | 2
| |
| | 1/2
| |
| |-
| |
| | virus
| |
| | '''viruses''', virii, viri
| |
| | viruses
| |
| | 1
| |
| | 1
| |
| |}
| |
| | |
| Given those three samples, we could calculate the mean reciprocal rank as (1/3 + 1/2 + 1)/3 = 11/18 or about 0.61.
| |
| | |
| This basic definition does not specify what to do if...
| |
| # none of the proposed results are correct (use mean reciprocal rank 0), or if
| |
| # there are multiple correct answers in the list. Consider using [[Information_retrieval#Mean_average_precision|mean average precision (MAP)]].
| |
| | |
| See also [[Information retrieval]] and [[Question answering]].<ref>{{cite conference | title=Evaluating web-based question answering systems | booktitle=Proceedings of LREC | author=D. R. Radev, H. Qi, H. Wu, W. Fan |year=2002 }}</ref>
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| [[Category:Summary statistics]]
| |
| [[Category:Information retrieval]]
| |
Golda is what's written on my birth certificate even though it is not the name on my birth certification. Invoicing is what I do. For years he's been residing in Mississippi and he doesn't strategy on altering it. To climb is some thing I truly enjoy performing.
Feel free to surf to my site; love psychics