Bennett acceptance ratio: Difference between revisions
en>Mandarax →Preliminaries: Typo patrol, typos fixed: temperatue → temperature using AWB (7910) |
Described MBAR; Added AlchemistryWiki link; Added pymbar download site. |
||
Line 1: | Line 1: | ||
In [[probability theory]], '''Kolmogorov's Three-Series Theorem''', named after [[Andrey Kolmogorov]], gives a criterion for the [[almost sure]] [[convergence of random variables|convergence]] of an [[infinite series]] of [[random variable]]s in terms of the convergence of three different series involving properties of their [[probability distribution]]s. Kolmogorov's three-series theorem, combined with [[Kronecker's lemma]], can be used to give a relatively easy proof of the [[Strong_Law_of_Large_Numbers#Strong_law|Strong Law of Large Numbers]].<ref>Durrett, Rick. "Probability: Theory and Examples." Duxbury advanced series, Third Edition, Thomson Brooks/Cole, 2005, Section 1.8, pp. 60-69.</ref> | |||
<!-- EDIT BELOW THIS LINE --> | |||
== Statement of the Theorem == | |||
Let (X<sub>n</sub>)<sub>n∈N</sub> be [[independent random variables]]. The random series ∑<sup>∞</sup><sub>n=1</sub>X<sub>n</sub> [[convergence of random variables|converges]] [[almost surely]] in ℝ if and only if the following conditions hold for some A > 0: | |||
'''i.''' ∑<sup>∞</sup><sub>n=1</sub>[[Image:U+2119.svg|9px]](|X|<sub>n</sub> ≥ A) converges | |||
'''ii.''' Let Y<sub>n</sub>:= X<sub>n</sub>1<sub>{|X|<sub>n</sub> ≤ A}</sub>, then ∑<sup>∞</sup><sub>n=1</sub>𝔼(Y<sub>n</sub>), the series of [[expected value]]s of Y<sub>n</sub> , converges | |||
'''iii.''' ∑<sup>∞</sup><sub>n=1</sub>𝕍ar(Y<sub>n</sub>) converges | |||
== Proof == | |||
=== Sufficiency of Conditions ("if") === | |||
Condition (i) and [[Borel-Cantelli]] give that [[almost surely]] X<sub>n</sub> = Y<sub>n</sub> for n large, hence ∑<sup>∞</sup><sub>n=1</sub>X<sub>n</sub> converges if and only if ∑<sup>∞</sup><sub>n=1</sub>Y<sub>n</sub> converges. Conditions (ii)-(iii) and [[Kolmogorov's Two-Series Theorem]] given the almost sure convergence of ∑<sup>∞</sup><sub>n=1</sub>𝔼(Y<sub>n</sub>). | |||
=== Necessity of Conditions ("only if") === | |||
Suppose that ∑<sup>∞</sup><sub>n=1</sub>X<sub>n</sub> converges almost surely. | |||
Without condition (i), by Borel-Cantelli there would exist some A > 0 such that almost surely {|X<sub>n</sub>| ≤ A} for infinitely many values n, but then the series would diverge. Therefore we must have condition (i). | |||
We see that condition (iii) implies condition (ii): [[Kolmogorov's Two-Series Theorem]] along with condition (i) applied to the case A =1 gives the convergence of ∑<sub>n</sub>(Y<sub>n</sub> - 𝔼(Y<sub>n</sub>)). So given the convergence of ∑<sup>∞</sup><sub>n=1</sub>Y<sub>n</sub>, we have ∑<sup>∞</sup><sub>n=1</sub>𝔼(Y<sub>n</sub>) converges, so condition (ii) is implied. | |||
Thus, it only remains to demonstrate the necessity of condition (iii), and we will have obtained the full result. It is equivalent to check condition (iii) for the series ∑<sup>∞</sup><sub>n=1</sub>Z<sub>n</sub> = ∑<sup>∞</sup><sub>n=1</sub>(Y<sub>n</sub> - Y'<sub>n</sub>) where for each n, Y<sub>n</sub> and Y'<sub>n</sub> are [[IID]]-- that is, to employ the assumption that 𝔼(Y<sub>n</sub>) = 0, since Z<sub>n</sub> is a sequence of random variables bounded by 2, converging almost surely, and with 𝕍ar(Z<sub>n</sub>) = 2𝕍ar(Y<sub>n</sub>). So we wish to check that if ∑<sup>∞</sup><sub>n=1</sub>Z<sub>n</sub> converges, ∑<sup>∞</sup><sub>n=1</sub>𝕍ar(Z<sub>n</sub>) converges as well. This is a special case of a more general result from [[martingale theory]] with summands equal to the increments of a [[martingale]] sequence and the same conditions (𝔼(Z<sub>n</sub>) = 0, series of the [[variance]]s converging, summands [[bounded]]).<ref>Sun, Rongfeng. Lecture notes. http://www.math.nus.edu.sg/~matsr/ProbI/Lecture4.pdf</ref><ref>M. Loève, "Probability theory", Princeton Univ. Press (1963) pp. Sect. 16.3</ref><ref>W. Feller, "An introduction to probability theory and its applications", 2, Wiley (1971) pp. Sect. IX.9</ref> | |||
== Example == | |||
As an illustration of the theorem, consider the example of the ''[[Harmonic_series_(mathematics)#Random_harmonic_series|harmonic series with random signs]]'': | |||
: <math> \sum_{n=1}^\infty \pm \frac{1}{n}.</math> | |||
Here, "<math>\pm</math>" means that each term <math>1/n</math> is taken with a random sign that is either <math>1</math> or <math>-1</math> with respective probabilities <math>1/2,\ 1/2</math>, and all random signs are chosen independently. Letting <math>X_n</math> in the theorem denote a random variable that takes the values <math>1/n</math> and <math>-1/n</math> with equal probabilities, one can check easily that the conditions of the theorem are satisfied, so it follows that the harmonic series with random signs converges almost surely. On the other hand, the analogous series of (for example) square root reciprocals with random signs, namely | |||
: <math> \sum_{n=1}^\infty \pm \frac{1}{\sqrt{n}},</math> | |||
''diverges'' almost surely, since condition (3) in the theorem is not satisfied. Note that this is different from the behavior of the analogous series with ''alternating'' signs, <math>\sum_{n=1}^\infty (-1)^n/\sqrt{n}</math>, which does converge. In fact one can simply check that | |||
: <math> \sum_{n=1}^\infty \frac{(-1)^n}{\sqrt{n}}=(\sqrt{2}-1)\zeta(\frac{1}{2})\approx -0.604899.</math> | |||
==Notes== | |||
<references/> | |||
== External links == | |||
*{{cite web|title=Romik, Dan. Probability Theory Lecture Notes, Fall 2009, UC Davis.|url=http://www.math.ucdavis.edu/~romik/teaching/lectures2.pdf}} | |||
[[Category:Mathematical series]] | |||
[[Category:Probability theorems]] |
Revision as of 11:44, 18 December 2013
In probability theory, Kolmogorov's Three-Series Theorem, named after Andrey Kolmogorov, gives a criterion for the almost sure convergence of an infinite series of random variables in terms of the convergence of three different series involving properties of their probability distributions. Kolmogorov's three-series theorem, combined with Kronecker's lemma, can be used to give a relatively easy proof of the Strong Law of Large Numbers.[1]
Statement of the Theorem
Let (Xn)n∈N be independent random variables. The random series ∑∞n=1Xn converges almost surely in ℝ if and only if the following conditions hold for some A > 0:
ii. Let Yn:= Xn1{|X|n ≤ A}, then ∑∞n=1𝔼(Yn), the series of expected values of Yn , converges
iii. ∑∞n=1𝕍ar(Yn) converges
Proof
Sufficiency of Conditions ("if")
Condition (i) and Borel-Cantelli give that almost surely Xn = Yn for n large, hence ∑∞n=1Xn converges if and only if ∑∞n=1Yn converges. Conditions (ii)-(iii) and Kolmogorov's Two-Series Theorem given the almost sure convergence of ∑∞n=1𝔼(Yn).
Necessity of Conditions ("only if")
Suppose that ∑∞n=1Xn converges almost surely.
Without condition (i), by Borel-Cantelli there would exist some A > 0 such that almost surely {|Xn| ≤ A} for infinitely many values n, but then the series would diverge. Therefore we must have condition (i).
We see that condition (iii) implies condition (ii): Kolmogorov's Two-Series Theorem along with condition (i) applied to the case A =1 gives the convergence of ∑n(Yn - 𝔼(Yn)). So given the convergence of ∑∞n=1Yn, we have ∑∞n=1𝔼(Yn) converges, so condition (ii) is implied.
Thus, it only remains to demonstrate the necessity of condition (iii), and we will have obtained the full result. It is equivalent to check condition (iii) for the series ∑∞n=1Zn = ∑∞n=1(Yn - Y'n) where for each n, Yn and Y'n are IID-- that is, to employ the assumption that 𝔼(Yn) = 0, since Zn is a sequence of random variables bounded by 2, converging almost surely, and with 𝕍ar(Zn) = 2𝕍ar(Yn). So we wish to check that if ∑∞n=1Zn converges, ∑∞n=1𝕍ar(Zn) converges as well. This is a special case of a more general result from martingale theory with summands equal to the increments of a martingale sequence and the same conditions (𝔼(Zn) = 0, series of the variances converging, summands bounded).[2][3][4]
Example
As an illustration of the theorem, consider the example of the harmonic series with random signs:
Here, "" means that each term is taken with a random sign that is either or with respective probabilities , and all random signs are chosen independently. Letting in the theorem denote a random variable that takes the values and with equal probabilities, one can check easily that the conditions of the theorem are satisfied, so it follows that the harmonic series with random signs converges almost surely. On the other hand, the analogous series of (for example) square root reciprocals with random signs, namely
diverges almost surely, since condition (3) in the theorem is not satisfied. Note that this is different from the behavior of the analogous series with alternating signs, , which does converge. In fact one can simply check that
Notes
- ↑ Durrett, Rick. "Probability: Theory and Examples." Duxbury advanced series, Third Edition, Thomson Brooks/Cole, 2005, Section 1.8, pp. 60-69.
- ↑ Sun, Rongfeng. Lecture notes. http://www.math.nus.edu.sg/~matsr/ProbI/Lecture4.pdf
- ↑ M. Loève, "Probability theory", Princeton Univ. Press (1963) pp. Sect. 16.3
- ↑ W. Feller, "An introduction to probability theory and its applications", 2, Wiley (1971) pp. Sect. IX.9