|
|
| Line 1: |
Line 1: |
| {{context|date=February 2012}}
| | Hello! <br>My name is Holley and I'm a 30 years old boy from Harparary.<br><br>Here is my page [http://www.termlifepolicy.com/insurance-agents/utah/ term life insurance ratings] |
| In [[information theory]], '''Sanov's theorem''' gives a bound on the probability of observing an [[Typical set|atypical]] sequence of samples from a given [[probability distribution]].
| |
| | |
| Let ''A'' be a set of probability distributions over an alphabet ''X'', and let ''q'' be an arbitrary distribution over ''X'' (where ''q'' may or may not be in ''A''). Suppose we draw ''n'' [[Independent and identically distributed random variables|i.i.d.]] samples from ''q'', represented by the vector <math>x^n = x_1, x_2, \ldots, x_n</math>. Further, let us ask that the [[Empirical distribution function|empirical distribution]], <math>\hat{p}_{x^n}</math>, of the samples falls within the set ''A'' -- formally, we write <math>\{x^n : \hat{p}_{x^n} \in A\}</math>. Then,
| |
| | |
| :<math>q^n(x^n) \le (n+1)^{|X|} 2^{-nD_{\mathrm{KL}}(p^*||q)}</math>,
| |
| | |
| where
| |
| * <math>q^n(x^n)</math> is shorthand for <math>q(x_1)q(x_2) \cdots q(x_n)</math>, and
| |
| * <math>p^*</math> is the [[information projection]] of ''q'' onto ''A''.
| |
| | |
| In words, the probability of drawing an atypical distribution is proportional to the [[Kullback–Leibler divergence|KL distance]] from the true distribution to the atypical one; in the case that we consider a set of possible atypical distributions, there is a dominant atypical distribution, given by the information projection.
| |
| | |
| Furthermore, if ''A'' is a [[closed set]],
| |
| | |
| :<math>\lim_{n\to\infty}\frac{1}{n}\log q^n(x^n) = -D_{\mathrm{KL}}(p^*||q).</math>
| |
| | |
| {{inline|date=February 2012}}
| |
| ==References==
| |
| | |
| *{{Cite book
| |
| | last1 = Cover | first1 = Thomas M.
| |
| | last2 = Thomas | first2 = Joy A.
| |
| | title = Elements of Information Theory
| |
| | publisher = Wiley Interscience
| |
| | edition = 2
| |
| | date = 2006
| |
| | location = Hoboken, New Jersey
| |
| | pages = 362}}
| |
| | |
| *Sanov, I. N. (1957) "On the probability of large deviations of random variables". ''Mat. Sbornik'' 42, 11–44.
| |
| | |
| [[Category:Information theory]]
| |
| [[Category:Probabilistic inequalities]]
| |
| | |
| | |
| {{probability-stub}}
| |
Hello!
My name is Holley and I'm a 30 years old boy from Harparary.
Here is my page term life insurance ratings