en>Daniele Pugliesi: -Category:Fluid mechanics; ±Category:Fluid dynamics→Category:Flow regimes using HotCat

2014-05-01T12:26:43Z

-Category:Fluid mechanics; ±Category:Fluid dynamics→Category:Flow regimes using HotCat

← Older revision		Revision as of 14:26, 1 May 2014
Line 1:		Line 1:
	~~{{context\|date=February 2012}}~~		Hello! <br>My name is Holley and I'm a 30 years old boy from Harparary.<br><br>Here is my page [http://www.termlifepolicy.com/insurance-agents/utah/ term life insurance ratings]
	~~In [[information theory]], '''Sanov's theorem''~~' ~~gives~~ a ~~bound on the probability of observing an [[Typical set\|atypical]] sequence of samples~~ from ~~a given [[probability distribution]].~~

	Let ''A'' be a set of probability distributions over an alphabet ''X'', and let ''q'' be an arbitrary distribution over ''X'' (where ''q'' may or may not be in ''A''). Suppose we draw ''n'' [[Independent and identically distributed random variables\|i.~~i.d.]] samples from ''q'', represented by the vector~~ <~~math~~>~~x^n = x_1, x_2, \ldots, x_n~~<~~/math~~>~~. Further, let us ask that the~~ [~~[Empirical distribution function\|empirical distribution]], <math>\hat{p}_{x^n}</math>, of the samples falls within the set ''A'' -- formally, we write <math>\{x^n~~ : ~~\hat{p}_{x^n} \in A\}<~~/~~math>. Then,~~

	~~:<math>q^n(x^n) \le (n+1)^{\|X\|} 2^{-nD_{\mathrm{KL}}(p^*\|\|q)}</math>,~~

	~~where~~
	* <math>q^n(x^n)</math> is shorthand for <math>q(x_1)q(x_2) \cdots q(x_n)</math>, and
	* <math>p^*</~~math> is the [[information projection]] of ''q'' onto ''A''~~.

	In words, the probability of drawing an atypical distribution is proportional to the [[Kullback–Leibler divergence\|KL distance]] from the true distribution to the atypical one; in the case that we consider a set of possible atypical distributions, there is a dominant atypical distribution, given by the information projection.

	~~Furthermore, if ''A'' is a [[closed set]],~~

	~~:<math>\lim_{n\to\infty}\frac{1}{n}\log q^n(x^n) =~~ -~~D_{\mathrm{KL}}(p^*\|\|q).<~~/~~math>~~

	~~{{inline\|date=February 2012}}~~
	~~==References==~~

	*{{Cite book
	~~\| last1 = Cover \| first1 = Thomas M.~~
	~~\| last2 = Thomas \| first2 = Joy A.~~
	~~\| title = Elements of Information Theory~~
	~~\| publisher = Wiley Interscience~~
	~~\| edition = 2~~
	~~\| date = 2006~~
	~~\| location = Hoboken, New Jersey~~
	~~\| pages = 362}}~~

	*Sanov, I. N. (1957) "On the probability of large deviations of random variables". ''Mat. Sbornik'' 42, 11–44.

	~~[[Category:Information theory]]~~
	~~[[Category:Probabilistic inequalities~~]]


	~~{{probability-stub}}~~

en>Mr Sheep Measham: Added categories

2011-04-27T04:22:36Z

Added categories

New page

{{context|date=February 2012}}
In [[information theory]], '''Sanov's theorem''' gives a bound on the probability of observing an [[Typical set|atypical]] sequence of samples from a given [[probability distribution]].

Let ''A'' be a set of probability distributions over an alphabet ''X'', and let ''q'' be an arbitrary distribution over ''X'' (where ''q'' may or may not be in ''A''). Suppose we draw ''n'' [[Independent and identically distributed random variables|i.i.d.]] samples from ''q'', represented by the vector <math>x^n = x_1, x_2, \ldots, x_n</math>. Further, let us ask that the [[Empirical distribution function|empirical distribution]], <math>\hat{p}_{x^n}</math>, of the samples falls within the set ''A'' -- formally, we write <math>\{x^n : \hat{p}_{x^n} \in A\}</math>. Then,

:<math>q^n(x^n) \le (n+1)^{|X|} 2^{-nD_{\mathrm{KL}}(p^*||q)}</math>,

where
* <math>q^n(x^n)</math> is shorthand for <math>q(x_1)q(x_2) \cdots q(x_n)</math>, and
* <math>p^*</math> is the [[information projection]] of ''q'' onto ''A''.

In words, the probability of drawing an atypical distribution is proportional to the [[Kullback–Leibler divergence|KL distance]] from the true distribution to the atypical one; in the case that we consider a set of possible atypical distributions, there is a dominant atypical distribution, given by the information projection.

Furthermore, if ''A'' is a [[closed set]],

:<math>\lim_{n\to\infty}\frac{1}{n}\log q^n(x^n) = -D_{\mathrm{KL}}(p^*||q).</math>

{{inline|date=February 2012}}
==References==

*{{Cite book
| last1 = Cover | first1 = Thomas M.
| last2 = Thomas | first2 = Joy A.
| title = Elements of Information Theory
| publisher = Wiley Interscience
| edition = 2
| date = 2006
| location = Hoboken, New Jersey
| pages = 362}}

*Sanov, I. N. (1957) "On the probability of large deviations of random variables". ''Mat. Sbornik'' 42, 11–44.

[[Category:Information theory]]
[[Category:Probabilistic inequalities]]

{{probability-stub}}

Collapsible flow - Revision history

en>Daniele Pugliesi: -Category:Fluid mechanics; ±Category:Fluid dynamics→Category:Flow regimes using HotCat

en>Mr Sheep Measham: Added categories