# Normal distribution

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

{{#invoke:Hatnote|hatnote}} Template:Use mdy dates

In probability theory, the normal (or Gaussian) distribution is a very commonly occurring continuous probability distribution—a function that tells the probability that any real observation will fall between any two real limits or real numbers, as the curve approaches zero on either side. Normal distributions are extremely important in statistics and are often used in the natural and social sciences for real-valued random variables whose distributions are not known.[1][2]

The normal distribution is immensely useful because of the central limit theorem, which states that, under mild conditions, the mean of many random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution: physical quantities that are expected to be the sum of many independent processes (such as measurement errors) often have a distribution very close to the normal.[3] Moreover, many results and methods (such as propagation of uncertainty and least squares parameter fitting) can be derived analytically in explicit form when the relevant variables are normally distributed.

The Gaussian distribution is sometimes informally called the bell curve. However, many other distributions are bell-shaped (such as Cauchy's, Student's, and logistic). The terms Gaussian function and Gaussian bell curve are also ambiguous because they sometimes refer to multiples of the normal distribution that cannot be directly interpreted in terms of probabilities.

A normal distribution is:

${\displaystyle f(x,\mu ,\sigma )={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}}}$

The parameter ${\displaystyle \mu }$ in this definition is the mean or expectation of the distribution (and also its median and mode). The parameter ${\displaystyle \sigma }$ is its standard deviation; its variance is therefore ${\displaystyle \sigma ^{2}}$. A random variable with a Gaussian distribution is said to be normally distributed and is called a normal deviate.

If ${\displaystyle \mu =0}$ and ${\displaystyle \sigma =1}$, the distribution is called the standard normal distribution or the unit normal distribution denoted by ${\displaystyle N(0,1)}$ and a random variable with that distribution is a standard normal deviate.

The normal distribution is the only absolutely continuous distribution all of whose cumulants beyond the first two (i.e., other than the mean and variance) are zero. It is also the continuous distribution with the maximum entropy for a given mean and variance.[4][5]

The normal distribution is a subclass of the elliptical distributions. The normal distribution is symmetric about its mean, and is non-zero over the entire real line. As such it may not be a suitable model for variables that are inherently positive or strongly skewed, such as the weight of a person or the price of a share. Such variables may be better described by other distributions, such as the log-normal distribution or the Pareto distribution.

The value of the normal distribution is practically zero when the value x lies more than a few standard deviations away from the mean. Therefore, it may not be an appropriate model when one expects a significant fraction of outliers—values that lie many standard deviations away from the mean — and least squares and other statistical inference methods that are optimal for normally distributed variables often become highly unreliable when applied to such data. In those cases, a more heavy-tailed distribution should be assumed and the appropriate robust statistical inference methods applied.

The Gaussian distribution belongs to the family of stable distributions which are the attractors of sums of independent, identically distributed (i.i.d.) distributions whether or not the mean or variance is finite. Except for the Gaussian which is a limiting case, all stable distributions have heavy tails and infinite variance.

## Definition

### Standard normal distribution

The simplest case of a normal distribution is known as the standard normal distribution. This is a special case where μ=0 and σ=1, and it is described by this probability density function:

${\displaystyle \phi (x)={\frac {e^{-{\frac {\scriptscriptstyle 1}{\scriptscriptstyle 2}}x^{2}}}{\sqrt {2\pi }}}\,}$

The factor ${\displaystyle \scriptstyle \ 1/{\sqrt {2\pi }}}$ in this expression ensures that the total area under the curve ϕ(x) is equal to one.[6] The Template:Frac2 in the exponent ensures that the distribution has unit variance (and therefore also unit standard deviation). This function is symmetric around x=0, where it attains its maximum value ${\displaystyle 1/{\sqrt {2\pi }}}$; and has inflection points at +1 and −1.

Authors may differ also on which normal distribution should be called the "standard" one. Gauss himself defined the standard normal as having variance , that is

${\displaystyle \phi (x)={\frac {e^{-x^{2}}}{\sqrt {\pi }}}\,}$

Stigler[7] goes even further, defining the standard normal with variance  :

${\displaystyle \phi (x)=e^{-\pi x^{2}}}$

### General normal distribution

Any normal distribution is a version of the standard normal distribution whose domain has been stretched by a factor σ (the standard deviation) and then translated by μ (the mean value):

${\displaystyle f(x,\mu ,\sigma )={\frac {1}{\sigma }}\phi \left({\frac {x-\mu }{\sigma }}\right).}$

The probability density must be scaled by ${\displaystyle 1/\sigma }$ so that the integral is still 1.

If Z is a standard normal deviate, then X = + μ will have a normal distribution with expected value μ and standard deviation σ. Conversely, if X is a general normal deviate, then Z = (Xμ)/σ will have a standard normal distribution.

Every normal distribution is the exponential of a quadratic function:

${\displaystyle f(x)=e^{ax^{2}+bx+c}}$

where a is negative and c is ${\displaystyle b^{2}/(4a)+\ln(-a/\pi )/2}$. In this form, the mean value μ is −b/(2a), and the variance σ2 is −1/(2a). For the standard normal distribution, a is −1/2, b is zero, and c is ${\displaystyle -\ln(2\pi )/2}$.

### Notation

The standard Gaussian distribution (with zero mean and unit variance) is often denoted with the Greek letter ϕ (phi).[8] The alternative form of the Greek phi letter, φ, is also used quite often.

The normal distribution is also often denoted by N(μ, σ2).[9] Thus when a random variable X is distributed normally with mean μ and variance σ2, we write

${\displaystyle X\ \sim \ {\mathcal {N}}(\mu ,\,\sigma ^{2}).}$

### Alternative parameterizations

Some authors advocate using the precision τ as the parameter defining the width of the distribution, instead of the deviation σ or the variance σ2. The precision is normally defined as the reciprocal of the variance, 1/σ2.[10] The formula for the distribution then becomes

${\displaystyle f(x)={\sqrt {\frac {\tau }{2\pi }}}\,e^{\frac {-\tau (x-\mu )^{2}}{2}}.}$

This choice is claimed to have advantages in numerical computations when σ is very close to zero and simplify formulas in some contexts, such as in the Bayesian inference of variables with multivariate normal distribution.

Also the reciprocal of the standard deviation ${\displaystyle \tau ^{\prime }=1/\sigma }$ might be defined as the precision and the expression of the normal distribution becomes

${\displaystyle f(x)={\frac {\tau ^{\prime }}{\sqrt {2\pi }}}\,e^{\frac {-(\tau ^{\prime })^{2}(x-\mu )^{2}}{2}}.}$

According to Stigler, this formulation is advantageous because of a much simpler and easier-to-remember formula, the fact that the pdf has unit height at zero, and simple approximate formulas for the quantiles of the distribution.

## Properties

### Symmetries and derivatives

The normal distribution f(x), with any mean μ and any positive deviation σ, has the following properties:

• It is symmetric around the point x = μ, which is at the same time the mode, the median and the mean of the distribution.[11]
• It is unimodal: its first derivative is positive for x < μ, negative for x > μ, and zero only at x = μ.
• Its density has two inflection points (where the second derivative of f is zero and changes sign), located one standard deviation away from the mean, namely at x = μ − σ and x = μ + σ.[11]
• Its density is log-concave.[11]
• Its density is infinitely differentiable, indeed supersmooth of order 2.[12]
• Its second derivative f′′(x) is equal to its derivative with respect to its variance σ2

Furthermore, the density ϕ of the standard normal distribution (with μ = 0 and σ = 1) also has the following properties:

• Its first derivative ϕ′(x) is −(x).
• Its second derivative ϕ′′(x) is (x2 − 1)ϕ(x)
• More generally, its n-th derivative ϕ(n)(x) is (−1)nHn(x)ϕ(x), where Hn is the Hermite polynomial of order n.[13]
• It satisfies the differential equation
${\displaystyle \sigma ^{2}f'(x)+f(x)(x-\mu )=0,\qquad f(0)={\frac {e^{-\mu ^{2}/(2\sigma ^{2})}}{{\sqrt {2\pi }}\sigma }}}$
or
${\displaystyle f'(x)+\tau f(x)(x-\mu )=0,\qquad f(0)={\frac {{\sqrt {\tau }}e^{-\mu ^{2}\tau /2}}{\sqrt {2\pi }}}.}$

### Moments

{{#invoke:see also|seealso}} The plain and absolute moments of a variable X are the expected values of Xp and |X|p,respectively. If the expected value μ of X is zero, these parameters are called central moments. Usually we are interested only in moments with integer order p.

If X has a normal distribution, these moments exist and are finite for any p whose real part is greater than −1. For any non-negative integer p, the plain central moments are

${\displaystyle {\mathrm {E} }\left[X^{p}\right]={\begin{cases}0&{\text{if }}p{\text{ is odd,}}\\\sigma ^{p}\,(p-1)!!&{\text{if }}p{\text{ is even.}}\end{cases}}}$

Here n!! denotes the double factorial, that is, the product of every number from n to 1 that has the same parity as n.

The central absolute moments coincide with plain moments for all even orders, but are nonzero for odd orders. For any non-negative integer p,

${\displaystyle \operatorname {E} \left[|X|^{p}\right]=\sigma ^{p}\,(p-1)!!\cdot \left.{\begin{cases}{\sqrt {\frac {2}{\pi }}}&{\text{if }}p{\text{ is odd}}\\1&{\text{if }}p{\text{ is even}}\end{cases}}\right\}=\sigma ^{p}\cdot {\frac {2^{\frac {p}{2}}\Gamma \left({\frac {p+1}{2}}\right)}{\sqrt {\pi }}}}$

## Citations

1. Normal Distribution, Gale Encyclopedia of Psychology
2. Template:Harvtxt
3. Lyon, A. (2014). Why are Normal Distributions Normal?, The British Journal for the Philosophy of Science.
4. {{#invoke:citation/CS1|citation |CitationClass=book }}
5. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
6. For the proof see Gaussian integral
7. Template:Harvtxt
8. Template:Harvtxt
9. Template:Harvtxt
10. Template:Harvtxt
11. Template:Harvtxt
12. Template:Harvtxt
13. Template:Harvtxt
14. Template:Harvtxt
15. Template:Harvtxt
16. Template:Cite web
17. Template:Cite web
18. WolframAlpha.com
19. Normal Approximation to Poisson(λ) Distribution, http://www.stat.ucla.edu/
20. Template:Harvtxt
21. Template:Harvtxt
22. Template:Harvtxt
23. Template:Harvtxt
24. Template:Harvtxt
25. Quine, M.P. (1993) "On three characterisations of the normal distribution", Probability and Mathematical Statistics, 14 (2), 257-263
26. Normal Product Distribution, Mathworld
27. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
28. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
29. {{#invoke:citation/CS1|citation |CitationClass=book }}
30. Template:Harvtxt
31. Template:Harvtxt
32. Template:Harvtxt
33. Template:Harvtxt
34. {{#invoke:citation/CS1|citation |CitationClass=book }}
35. {{#invoke:citation/CS1|citation |CitationClass=book }}
36. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
37. Template:Harvtxt
38. Template:Harvtxt
39. Template:Harvtxt
40. Template:Harvtxt
41. Template:Harvtxt
42. Template:Harvtxt
43. De Moivre, Abraham (1733), Corollary I – see Template:Harvtxt
44. Template:Harvtxt
45. Template:Harvtxt
46. Template:Harvtxt
47. Template:Harvtxt
48. Template:Harvtxt
49. Template:Harvtxt
50. Template:Harvtxt
51. Template:Harvtxt
52. Template:Harvtxt
53. Jaynes, Edwin J.; Probability Theory: The Logic of Science, Ch 7
54. Peirce, Charles S. (c. 1909 MS), Collected Papers v. 6, paragraph 327
55. Template:Harvtxt
56. Template:Cite web

## References

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=citation }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }} Translated by Stephen M. Stigler in Statistical Science 1 (3), 1986: Template:Jstor.

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}