|
|
| Line 1: |
Line 1: |
| '''Discounted cumulative gain''' ('''DCG''') is a measure of effectiveness of a [[World Wide Web|Web]] [[search engine]] [[algorithm]] or related applications, often used in [[information retrieval]]. Using a [[Relevance (information retrieval)|graded relevance]] scale of documents in a search engine result set, DCG measures the usefulness, or ''gain'', of a document based on its position in the result list. The gain is accumulated from the top of the result list to the bottom with the gain of each result discounted at lower ranks.<ref>Kalervo Jarvelin, Jaana Kekalainen: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20(4), 422–446 (2002)</ref>
| | If an existing Word - Press code is found vulnerable, Word - Press will immediately issue an update for that. Offshore expert Word - Press developers high level of interactivity, accessibility, functionality and usability of our website can add custom online to using. Your parishioners and certainly interested audience can come in to you for further information from the group and sometimes even approaching happenings and systems with the church. They found out all the possible information about bringing up your baby and save money at the same time. Over a million people are using Wordpress to blog and the number of Wordpress users is increasing every day. <br><br>As you know today Word - Press has turn out to be a tremendously popular open source publishing and blogging display place. If you wish to sell your services or products via internet using your website, you have to put together on the website the facility for trouble-free payment transfer between customers and the company. There are number of web services that offer Word press development across the world. This is identical to doing a research as in depth above, nevertheless you can see various statistical details like the number of downloads and when the template was not long ago updated. W3C compliant HTML and a good open source powered by Word - Press CMS site is regarded as the prime minister. <br><br>In case you have just about any issues concerning exactly where in addition to the best way to make use of [http://teenyweenyurl.com/WordpressDropboxBackup35423 wordpress backup], you are able to e-mail us at the page. The least difficult and very best way to do this is by acquiring a Word - Press site. By using Word - Press MLM websites or blogs, you and your companies presence on the internet can be created swiftly and simply. If Gandhi was empowered with a blogging system, every event in his life would have been minutely documented so that it could be recounted to the future generations. Provide the best and updated information to the web searchers and make use of these wonderful free themes and create beautiful websites. If you have any questions on starting a Word - Press food blog or any blog for that matter, please post them below and I will try to answer them. <br><br>Digg Digg Social Sharing - This plugin that is accountable for the floating social icon located at the left aspect corner of just about every submit. Quttera - Quttera describes itself as a 'Saa - S [Software as a Service] web-malware monitoring and alerting solution for websites of any size and complexity. re creating a Word - Press design yourself, the good news is there are tons of Word - Press themes to choose from. IVF ,fertility,infertility expert,surrogacy specialist in India at Rotundaivf. It does take time to come up having a website that gives you the much needed results hence the web developer must be ready to help you along the route. <br><br>Website security has become a major concern among individuals all over the world. An ease of use which pertains to both internet site back-end and front-end users alike. However, you must also manually approve or reject comments so that your website does not promote parasitic behavior. Page speed is an important factor in ranking, especially with Google. Get started today so that people searching for your type of business will be directed to you. |
| | |
| == Overview ==
| |
| | |
| Two assumptions are made in using DCG and its related measures.
| |
| | |
| # Highly relevant documents are more useful when appearing earlier in a search engine result list (have higher ranks)
| |
| # Highly relevant documents are more useful than marginally relevant documents, which are in turn more useful than irrelevant documents.
| |
| | |
| DCG originates from an earlier, more primitive, measure called Cumulative Gain.
| |
| | |
| === Cumulative Gain ===
| |
| | |
| Cumulative Gain (CG) is the predecessor of DCG and does not include the position of a result in the consideration of the usefulness of a result set. In this way, it is the sum of the graded relevance values of all results in a search result list. The CG at a particular rank position <math>p</math> is defined as:
| |
| | |
| :<math> \mathrm{CG_{p}} = \sum_{i=1}^{p} rel_{i} </math>
| |
| | |
| Where <math>rel_{i}</math> is the graded relevance of the result at position <math>i</math>.
| |
| | |
| The value computed with the CG function is unaffected by changes in the ordering of search results. That is, moving a highly relevant document <math>d_{i}</math> above a higher ranked, less relevant, document <math>d_{j}</math> does not change the computed value for CG. Based on the two assumptions made above about the usefulness of search results, DCG is used in place of CG for a more accurate measure.
| |
| | |
| === Discounted Cumulative Gain ===
| |
| | |
| The premise of DCG is that highly relevant documents appearing lower in a search result list should be penalized as the graded relevance value is reduced logarithmically proportional to the position of the result. The discounted CG accumulated at a particular rank position <math>p</math> is defined as:
| |
| | |
| :<math> \mathrm{DCG_{p}} = rel_{1} + \sum_{i=2}^{p} \frac{rel_{i}}{\log_{2}(i)} </math>
| |
| | |
| Previously there has not been shown any theoretically sound justification for using a [[logarithm|logarithmic]] reduction factor<ref>{{cite book | title=Search Engines: Information Retrieval in Practice | author=B. Croft, D. Metzler, and T. Strohman |date=2009 | publisher=''Addison Wesley"}}</ref> other than the fact that it produces a smooth reduction. An alternative formulation of DCG<ref>Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning (ICML '05). ACM, New York, NY, USA, 89-96. DOI=10.1145/1102351.1102363 http://doi.acm.org/10.1145/1102351.1102363</ref> places stronger emphasis on retrieving relevant documents:
| |
| | |
| :<math> \mathrm{DCG_{p}} = \sum_{i=1}^{p} \frac{ 2^{rel_{i}} - 1 }{ \log_{2}(i+1)} </math>
| |
| | |
| In Croft, Metzler and Strohman (page 320, 2010), the authors mistakenly claim that these two formulations of DCG are the same when the relevance values of documents are [[binary function|binary]]; <math>rel_{i} \in \{0,1\}</math>. To see that they are not the same, let there be one relevant document and that relevant document is at rank 2. The first version of DCG equals 1 / log2(2) = 1. The second version of DCG equals 1 / log2(2+1) = 0.631. The way that the two formulations of DCG are the same for binary judgments is in the way gain in the numerator is calculated. For both formulations of DCG, binary relevance produces gain at rank i of 0 or 1. No matter the number of relevance grades, the two formulations differ in their discount of gain. Note that Croft et al. (2010) and Burges et al. (2005) present the second DCG with a log of base e, while both versions of DCG above use a log of base 2. When computing NDCG with the second formulation of DCG, the base of the log does not matter, but the base of the log does affect the value of NDCG for the first formulation. Clearly, the base of the log affects the value of DCG in both formulations.
| |
| | |
| Recently, Wang et al.(2013)<ref>Yining Wang, Liwei Wang, Yuanzhi Li, Di He, Wei Chen, Tie-Yan Liu. 2013. A Theoretical Analysis of NDCG Ranking Measures. In Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013).</ref> give theoretical guarantee for using the logarithmic reduction factor in NDCG. Specifically, the authors prove for every pair of substantially different ranking functions, the ranking measure can decide which one is better in a consistent manner on almost all datasets.
| |
| | |
| === Normalized DCG ===
| |
| | |
| Search result lists vary in length depending on the [[Web search query|query]]. Comparing a search engine's performance from one query to the next cannot be consistently achieved using DCG alone, so the cumulative gain at each position for a chosen value of <math>p</math> should be normalized across queries. This is done by sorting documents of a result list by relevance, producing the maximum possible DCG till position <math>p</math>, also called Ideal DCG (IDCG) till that position. For a query, the ''normalized discounted cumulative gain'', or nDCG, is computed as:
| |
| | |
| :<math> \mathrm{nDCG_{p}} = \frac{DCG_{p}}{IDCG_{p}} </math>
| |
| | |
| The nDCG values for all queries can be averaged to obtain a measure of the average performance of a search engine's ranking algorithm. Note that in a perfect ranking algorithm, the <math>DCG_p</math> will be the same as the <math>IDCG_p</math> producing an nDCG of 1.0. All nDCG calculations are then relative values on the interval 0.0 to 1.0 and so are cross-query comparable.
| |
| | |
| The main difficulty encountered in using nDCG is the unavailability of an ideal ordering of results when only partial [[relevance feedback]] is available.
| |
| | |
| == Example ==
| |
| | |
| Presented with a list of documents in response to a search query, an experiment participant is asked to judge the relevance of each document to the query. Each document is to be judged on a scale of 0-3 with 0 meaning irrelevant, 3 meaning completely relevant, and 1 and 2 meaning "somewhere in between". For the documents ordered by the ranking algorithm as
| |
| | |
| :<math> D_{1}, D_{2}, D_{3}, D_{4}, D_{5}, D_{6} </math>
| |
| | |
| the user provides the following relevance scores:
| |
| | |
| :<math> 3, 2, 3, 0, 1, 2 </math>
| |
| | |
| That is: document 1 has a relevance of 3, document 2 has a relevance of 2, etc. The Cumulative Gain of this search result listing is:
| |
| | |
| :<math> \mathrm{CG_{p}} = \sum_{i=1}^{p} rel_{i} = 3 + 2 + 3 + 0 + 1 + 2 = 11</math>
| |
| | |
| Changing the order of any two documents does not affect the CG measure. If <math>D_3</math> and <math>D_4</math> are switched, the CG remains the same, 11. DCG is used to emphasize highly relevant documents appearing early in the result list. Using the logarithmic scale for reduction, the DCG for each result in order is:
| |
| | |
| {| class="wikitable" border="1"
| |
| |-
| |
| ! <math>i</math>
| |
| ! <math>rel_{i}</math>
| |
| ! <math>\log_{2}i</math>
| |
| ! <math> \frac{rel_{i}}{\log_{2}i} </math>
| |
| |-
| |
| | 1
| |
| | 3
| |
| | 0
| |
| | N/A
| |
| |-
| |
| | 2
| |
| | 2
| |
| | 1
| |
| | 2
| |
| |-
| |
| | 3
| |
| | 3
| |
| | 1.585
| |
| | 1.892
| |
| |-
| |
| | 4
| |
| | 0
| |
| | 2.0
| |
| | 0
| |
| |-
| |
| | 5
| |
| | 1
| |
| | 2.322
| |
| | 0.431
| |
| |-
| |
| | 6
| |
| | 2
| |
| | 2.584
| |
| | 0.774
| |
| |}
| |
| | |
| So the <math>DCG_{6}</math> of this ranking is:
| |
| | |
| :<math> \mathrm{DCG_{6}} = rel_{1} + \sum_{i=2}^{6} \frac{rel_{i}}{\log_{2}i} = 3 + (2 + 1.892 + 0 + 0.431 + 0.774) = 8.10</math>
| |
| | |
| Now a switch of <math>D_3</math> and <math>D_4</math> results in a reduced DCG because a less relevant document is placed higher in the ranking; that is, a more relevant document is discounted more by being placed in a lower rank.
| |
| | |
| The performance of this query to another is incomparable in this form since the other query may have more results, resulting in a larger overall DCG which may not necessarily be better. In order to compare, the DCG values must be normalized.
| |
| | |
| To normalize DCG values, an ideal ordering for the given query is needed. For this example, that ordering would be the [[Monotonic|monotonically decreasing]] sort of the relevance judgments provided by the experiment participant, which is:
| |
| | |
| :<math> 3, 3, 2, 2, 1, 0 </math>
| |
| | |
| The DCG of this ideal ordering, or ''IDCG'', is then:
| |
| | |
| :<math> \mathrm{IDCG_{6}} = 8.69 </math>
| |
| | |
| And so the nDCG for this query is given as:
| |
| | |
| :<math> \mathrm{nDCG_{6}} = \frac{DCG_{6}}{IDCG_{6}} = \frac{8.10}{8.69} = 0.932 </math>
| |
| | |
| == Limitations ==
| |
| # Normalized DCG metric does not penalize for bad documents in the result. For example, if a query returns two results with scores <math> 1,1,1 </math> and <math> 1,1,1,0 </math> respectively, both would be considered equally good even if later contains a bad result. One way to take into account this limitation is use <math>1 - 2^{rel_{i}}</math> in numerator for scores for which we want to penalize and <math>2^{rel_{i}} - 1</math> for all others. For example, for the ranking judgments <math>Excellent, Fair, Bad</math> one might use numerical scores <math>1,0,-1</math> instead of <math>2,1,0</math>.
| |
| # Normalized DCG does not penalize for missing documents in the result. For example, if a query returns two results with scores <math> 1,1,1 </math> and <math> 1,1,1,1,1 </math> respectively, both would be considered equally good. One way to take into account this limitation is to enforce fixed set size for the result set and use minimum scores for the missing documents. In previous example, we would use the scores scores <math> 1,1,1,0,0 </math> and <math> 1,1,1,1,1 </math> and quote nDCG as nDCG@5.
| |
| # Normalized DCG may not be suitable to measure performance of queries that may typically often have several equally good results. This is especially true when this metric is limited to only first few results as it is done in practice. For example, for queries such as "restaurants" nDCG@1 would account for only first result and hence if one result set contains only 1 restaurant from the nearby area while the other contains 5, both would end up having same score even though later is more comprehensive.
| |
| | |
| == References ==
| |
| {{Reflist|1}}
| |
| | |
| [[Category:Information retrieval|*]]
| |
If an existing Word - Press code is found vulnerable, Word - Press will immediately issue an update for that. Offshore expert Word - Press developers high level of interactivity, accessibility, functionality and usability of our website can add custom online to using. Your parishioners and certainly interested audience can come in to you for further information from the group and sometimes even approaching happenings and systems with the church. They found out all the possible information about bringing up your baby and save money at the same time. Over a million people are using Wordpress to blog and the number of Wordpress users is increasing every day.
As you know today Word - Press has turn out to be a tremendously popular open source publishing and blogging display place. If you wish to sell your services or products via internet using your website, you have to put together on the website the facility for trouble-free payment transfer between customers and the company. There are number of web services that offer Word press development across the world. This is identical to doing a research as in depth above, nevertheless you can see various statistical details like the number of downloads and when the template was not long ago updated. W3C compliant HTML and a good open source powered by Word - Press CMS site is regarded as the prime minister.
In case you have just about any issues concerning exactly where in addition to the best way to make use of wordpress backup, you are able to e-mail us at the page. The least difficult and very best way to do this is by acquiring a Word - Press site. By using Word - Press MLM websites or blogs, you and your companies presence on the internet can be created swiftly and simply. If Gandhi was empowered with a blogging system, every event in his life would have been minutely documented so that it could be recounted to the future generations. Provide the best and updated information to the web searchers and make use of these wonderful free themes and create beautiful websites. If you have any questions on starting a Word - Press food blog or any blog for that matter, please post them below and I will try to answer them.
Digg Digg Social Sharing - This plugin that is accountable for the floating social icon located at the left aspect corner of just about every submit. Quttera - Quttera describes itself as a 'Saa - S [Software as a Service] web-malware monitoring and alerting solution for websites of any size and complexity. re creating a Word - Press design yourself, the good news is there are tons of Word - Press themes to choose from. IVF ,fertility,infertility expert,surrogacy specialist in India at Rotundaivf. It does take time to come up having a website that gives you the much needed results hence the web developer must be ready to help you along the route.
Website security has become a major concern among individuals all over the world. An ease of use which pertains to both internet site back-end and front-end users alike. However, you must also manually approve or reject comments so that your website does not promote parasitic behavior. Page speed is an important factor in ranking, especially with Google. Get started today so that people searching for your type of business will be directed to you.