Symmetric set

The order in probability notation is used in probability theory and statistical theory in direct parallel to the big-O notation that is standard in mathematics. Where the big-O notation deals with the convergence of sequences or sets of ordinary numbers, the order in probability notation deals with convergence of sets of random variables, where convergence is in the sense of convergence in probability.

For a set of random variables X_n and a corresponding set of constants a_n (both indexed by n, which need not be discrete), the notation

$X_n=o_p(a_n)$

means that the set of values X_n/a_n converges to zero in probability as n approaches an appropriate limit. Equivalently, X_n = o_p(a_n) can be written as a_n o_p(1) where X_n = o_p(1) is defined as,

$\underset{n\to \mathrm{\infty }}{\lim }P(|X_n|\ge \epsilon )=0,$

for every positive ε.

The notation,

$X_n=O_p(a_n),$

means that the set of values X_n/a_n is stochastically bounded. That is, for any ε > 0, there exists a finite M > 0 such that,

$P(|X_n/a_n|>M)<\epsilon ,\mathrm{\forall }n.$

Example

If $(X_n)$ is a stochastic sequence such that each element has finite variance, then

$X_n-E(X_n)=O_p(\sqrt{\mathrm{var}(X_n)})$

(see Theorem 14.4-1 in Bishop et al.)

If, moreover, $a_n^{-2}\mathrm{var}(X_n)=\mathrm{var}(a_n^{-1}{X}_n)$ is a null sequence for a sequence $(a_n)$ of real numbers, then $a_n^{-1}(X_n-E(X_n))$ converges to zero in probability by Chebyshev's inequality, so

$X_n-E(X_n)=o_p(a_n)$.