Monetary policy reaction function

In the theory of probability, the Glivenko–Cantelli theorem, named after Valery Ivanovich Glivenko and Francesco Paolo Cantelli, determines the asymptotic behaviour of the empirical distribution function as the number of independent and identically distributed observations grows. The uniform convergence of more general empirical measures becomes an important property of the Glivenko–Cantelli classes of functions or sets. The Glivenko–Cantelli classes arise in Vapnik–Chervonenkis theory, with applications to machine learning. Applications can be found in econometrics making use of M-estimators.

Assume that $X_1,X_2,\dots$ are independent and identically-distributed random variables in $\mathbb{ℝ}$ with common cumulative distribution function $F(x)$. The empirical distribution function for $X_1,\dots ,X_n$ is defined by

$F_n(x)=\frac{1}{n}\sum _{i=1}^n{I}_{(-\mathrm{\infty },x]}(X_i),$

where $I_C$ is the indicator function of the set $C$. For every (fixed) $x$, $F_n(x)$ is a sequence of random variables which converge to $F(x)$ almost surely by the strong law of large numbers, that is, $F_n$ converges to $F$ pointwise. Glivenko and Cantelli strengthened this result by proving uniform convergence of $F_n$ to $F$.

Theorem

$\Vert F_n-F{\Vert }_{\mathrm{\infty }}={\sup }_{x\in \mathbb{ℝ}}|F_n(x)-F(x)|⟶0$ almost surely.

This theorem originates with Valery Glivenko, and Francesco Cantelli, in 1933.

Remarks

• If $X_n$ is a stationary ergodic process, then $F_n(x)$ converges almost surely to $F(x)=E(1_{X_1\le x})$. The Glivenko–Cantelli theorem gives a stronger mode of convergence than this in the iid case.
• An even stronger uniform convergence result for the empirical distribution function is available in the form of an extended type of law of the iterated logarithm. See asymptotic properties of the Empirical distribution function for this and related results.

Empirical measures

One can generalize the empirical distribution function by replacing the set $(-\mathrm{\infty },x]$ by an arbitrary set C from a class of sets $\mathcal{𝒞}$ to obtain an empirical measure indexed by sets $C\in \mathcal{𝒞}.$

$P_n(C)=\frac{1}{n}\sum _{i=1}^n{I}_C(X_i),C\in \mathcal{𝒞}.$

Further generalization is the map induced by $P_n$ on measurable real-valued functions f, which is given by

$f\mapsto P_{n}f=\underset{S}{\int }fd{P}_n=\frac{1}{n}\sum _{i=1}^{n}f(X_i),f\in \mathcal{ℱ}.$

Then it becomes an important property of these classes that the strong law of large numbers holds uniformly on $\mathcal{ℱ}$ or $\mathcal{𝒞}$.

Glivenko–Cantelli class

Consider a set $\mathcal{𝒮}$ with a sigma algebra of Borel subsets A and a probability measure P. For a class of subsets,

$\mathcal{𝒞}\subset \{C:C\text{ is measurable subset of }\mathcal{𝒮}\}$

and a class of functions

$\mathcal{ℱ}\subset \{f:\mathcal{𝒮}\to \mathbb{ℝ},f\text{ is measurable}\}$

define random variables

$\Vert P_n-P{\Vert }_{\mathcal{𝒞}}={\sup }_{c\in \mathcal{𝒞}}|P_n(C)-P(C)|$

$\Vert P_n-P{\Vert }_{\mathcal{ℱ}}={\sup }_{f\in \mathcal{ℱ}}|P_{n}f-P(f)|$

where $P_n(C)$ is the empirical measure, $P_{n}f$ is the corresponding map, and

$\mathbb{𝔼}f=\underset{\mathcal{𝒮}}{\int }fdP=P(f)$, assuming that it exists.

Definitions

• A class $\mathcal{𝒞}$ is called a Glivenko–Cantelli class (or GC class) with respect to a probability measure P if any of the following equivalent statements is true.

1. $\Vert P_n-P{\Vert }_{\mathcal{𝒞}}\to 0$ almost surely as $n\to \mathrm{\infty }$.

2. $\Vert P_n-P{\Vert }_{\mathcal{𝒞}}\to 0$ in probability as $n\to \mathrm{\infty }$.

3. $\mathbb{𝔼}\Vert P_n-P{\Vert }_{\mathcal{𝒞}}\to 0$, as $n\to \mathrm{\infty }$ (convergence in mean).

The Glivenko–Cantelli classes of functions are defined similarly.

• A class is called a universal Glivenko–Cantelli class if it is a GC class with respect to any probability measure P on (S,A).
• A class is called uniformly Glivenko–Cantelli if the convergence occurs uniformly over all probability measures P on (S,A):

${\sup }_{P\in \mathcal{𝒫}(S,A)}\mathbb{𝔼}\Vert P_n-P{\Vert }_{\mathcal{𝒞}}\to 0;$

${\sup }_{P\in \mathcal{𝒫}(S,A)}\mathbb{𝔼}\Vert P_n-P{\Vert }_{\mathcal{ℱ}}\to 0.$

Theorem (Vapnik and Chervonenkis, 1968)

A class of sets $\mathcal{𝒞}$ is uniformly GC if and only if it is a Vapnik–Chervonenkis class.

Examples

• Let $S=\mathbb{ℝ}$ and $\mathcal{𝒞}=\{(-\mathrm{\infty },t]:t\in \mathbb{ℝ}\}$. The classical Glivenko–Cantelli theorem implies that this class is a universal GC class. Furthermore, by Kolmogorov's theorem,

${\sup }_{P\in \mathcal{𝒫}(S,A)}\Vert P_n-P{\Vert }_{\mathcal{𝒞}}\sim n^{-1/2}$, that is $\mathcal{𝒞}$ is uniformly Glivenko–Cantelli class.

• Let P be a nonatomic probability measure on S and $\mathcal{𝒞}$ be a class of all finite subsets in S. Because $A_n=\{X_1,\dots ,X_n\}\in \mathcal{𝒞}$, $P(A_n)=0$, $P_n(A_n)=1$, we have that $\Vert P_n-P{\Vert }_{\mathcal{𝒞}}=1$ and so $\mathcal{𝒞}$ is not a GC class with respect to P.

See also

• Donsker's theorem
• Dvoretzky–Kiefer–Wolfowitz inequality – strengthens Glivenko–Cantelli theorem by quantifying the rate of convergence.

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

Further reading

• Dudley, R. M. (1999). Uniform Central Limit Theorems, Cambridge University Press. ISBN 0-521-46102-2.
• Shorack, G.R., Wellner J.A. (1986) Empirical Processes with Applications to Statistics, Wiley. ISBN 0-471-86725-X.
• van der Vaart, A.W. and Wellner, J.A. (1996) Weak Convergence and Empirical Processes, Springer. ISBN 0-387-94640-3.
• Aad W. van der Vaart, Jon A. Wellner (1996) Glivenko-Cantelli Theorems, Springer.
• Aad W. van der Vaart, Jon A. Wellner (2000) Preservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes, Springer