|
|
| Line 1: |
Line 1: |
| In [[probability]] and [[statistics]], a '''natural exponential family''' (NEF) is a class of [[probability distribution]]s that is a special case of an [[exponential family]] (EF). Every distribution possessing a [[moment-generating function]] is a member of a natural exponential family, and the use of such distributions simplifies the theory and computation of [[generalized linear models]].
| | Hello and welcome. My title is Irwin and [http://Healthland.Time.com/2014/01/07/std-tests-theres-an-app-for-that/ I totally] dig that title. For a while I've been in South Dakota and my mothers and std testing [http://www.articlestunner.com/cures-for-the-yeast-infection-suggestions-to-use-now/ at home std testing] home fathers live nearby. She is a [http://Www.mydr.com.au/sexual-health/herpes-your-questions-answered librarian] but she's always wanted her own business. The preferred hobby for my children and me is to perform baseball but I haven't made a dime with it.<br><br>Feel std [http://www.hotporn123.com/blog/154316 home std test] test free to visit my homepage :: [http://Inspirationpedi.com/groups/useful-guidance-for-battling-your-yeast-infection/ at home std test] |
| | |
| == Definition ==
| |
| | |
| === Probability distribution function (PDF) of the univariate case (scalar domain, scalar parameter) ===
| |
| | |
| The natural exponential family (NEF) is a subset of the [[exponential family]]. NEF is an exponential family in which the natural parameter η and the natural statistic ''T(x)'' are both the identity. A distribution in the [[exponential family]] with parameter θ can be written with [[probability density function]] (PDF)
| |
| :<math> f_X(x| \theta) = h(x)\ \exp\Big(\ \eta(\theta) T(x) - A(\theta)\ \Big) \,\! ,</math> | |
| where <math>h(x)</math> and <math>A(\theta)</math> are known functions.
| |
| A distribution in the natural exponential family with parameter θ can thus be written with PDF
| |
| :<math> f_X(x| \theta) = h(x)\ \exp\Big(\ \theta x - A(\theta)\ \Big) \,\! .</math>
| |
|
| |
| [Note that slightly different notation is used by the originator of the NEF, Carl Morris.<ref name=Morris2006>Morris C. (2006) "Natural exponential families", ''Encyclopedia of Statistical Sciences''.</ref> Morris uses ω instead of η and ψ instead of ''A''.]
| |
| | |
| === Probability distribution function (PDF) of the general case (multivariate domain and/or parameter) ===
| |
| Suppose that <math>\mathbf{x} \in \mathcal{X} \subseteq \mathbb{R}^p</math>, then a natural exponential family of order ''p'' has density or mass function of the form:
| |
| :<math> f_X(\mathbf{x}|\boldsymbol\theta) = h(\mathbf{x})\ \exp\Big(\boldsymbol\theta^{\rm T} \mathbf{x} - A(\boldsymbol\theta)\ \Big) \,\! ,</math>
| |
| where in this case the parameter <math>\boldsymbol\theta \in \mathbb{R}^p .</math>
| |
| | |
| === Moment and cumulant generating function ===
| |
| A member of a natural exponential family has [[moment generating function]] (MGF) of the form
| |
| | |
| :<math>M_X(\mathbf{t}) = \exp\Big(\ A(\boldsymbol\theta + \mathbf{t}) - A(\boldsymbol\theta)\ \Big) \, .</math>
| |
| | |
| The [[cumulant generating function]] is by definition the logarithm of the MGF, so it is
| |
| | |
| :<math>K_X(\mathbf{t}) = A(\boldsymbol\theta + \mathbf{t}) - A(\boldsymbol\theta) \, .</math>
| |
| | |
| == Examples ==
| |
| The five most important univariate cases are:
| |
| * [[normal distribution]] with known variance
| |
| * [[Poisson distribution]]
| |
| * [[gamma distribution]] with known shape parameter α (or k depending on notation set used)
| |
| * [[binomial distribution]] with known number of trials, ''n''
| |
| * [[negative binomial distribution]] with known <math>r</math>
| |
| | |
| These five examples – Poisson, binomial, negative binomial, normal, and gamma – are a special subset of NEF, called NEF with quadratic variance function (NEF-QVF) because the variance can be written as a quadratic function of the mean. NEF-QVF are discussed below.
| |
| | |
| Distributions such as the [[exponential distribution|exponential]], [[chi-squared distribution|chi-squared]], [[Rayleigh distribution|Rayleigh]], [[Weibull distribution|Weibull]], [[Bernoulli distribution|Bernoulli]], and [[geometric distribution]]s are special cases of the above five distributions. Many common distributions are either NEF or can be related to the NEF. For example: the [[chi-squared distribution]] is a special case of the [[gamma distribution]]. The [[Bernoulli distribution]] is a [[binomial distribution]] with ''n'' = 1 trial. The [[exponential distribution]] is a gamma distribution with shape parameter α = 1 (or ''k'' = 1 ). The [[Rayleigh distribution|Rayleigh]] and [[Weibull distribution]]s can each be written in terms of an exponential distribution.
| |
| | |
| Some exponential family distributions are not NEF. The [[lognormal]] and [[Beta distribution]] are in the exponential family, but not the natural exponential family.
| |
| | |
| The parameterization of most of the above distributions has been written differently than the parameterization commonly used in textbooks and the above linked pages. For example, the above parameterization differs from the parameterization in the linked article in the Poisson case. The two parameterizations are related by <math> \theta = \log(\lambda) </math>, where λ is the mean parameter, and so that the density may be written as
| |
| :<math>f(k;\theta) = \frac{1}{k!} \exp\Big(\ \theta\ k - \exp(\theta)\ \Big) \ ,</math>
| |
| for <math> \theta \in \mathbb{R}</math>, so
| |
| :<math>h(k) = \frac{1}{k!} \ </math>, and <math> A(\theta) = \exp(\theta)\ .</math>
| |
| | |
| This alternative parameterization can greatly simplify calculations in [[mathematical statistics]]. For example, in [[Bayesian inference]], a [[posterior probability distribution]] is calculated as the product of two distributions. Normally this calculation requires writing out the probability distribution functions (PDF) and integrating; with the above parameterization, however, that calculation can be avoided. Instead, relationships between distributions can be abstracted due to the properties of the NEF described below.
| |
|
| |
| An example of the multivariate case is the [[multinomial distribution]] with known number of trials.
| |
| | |
| == Properties ==
| |
| | |
| The properties of the natural exponential family can be used to simplify calculations involving these distributions.
| |
| | |
| === Univariant case ===
| |
| 1. The cumulants of an NEF can be calculated as derivatives of the NEF's cumulant generating function. The nth cumulant is the nth derivative with respect to θ of the cumulant generating function.{{cn|date=July 2012}}
| |
| | |
| The [[cumulant generating function]] is
| |
| :<math>K_X(t) = A(\theta + t) - A(\theta) \, .</math>
| |
| The first cumulant is
| |
| :<math> K_1 = \frac{d}{d\theta} A(t) \, .</math>
| |
| The mean is the first moment and always equal to the first cumulant, so
| |
| :<math> \mu_1' = \kappa_1 = \mathrm{E}[X] = K'_X(0) = A'(\theta)\, .</math>
| |
| | |
| The variance is always the second moment, and it is always related to the first and second cumulants by
| |
| :<math> \mathrm{Var}[X] = \mu_2' = \kappa_2 + \kappa_1^2 \, ,</math>
| |
| so that
| |
| :<math> \mathrm{Var}[X] = K''_X(0) = A''(\theta) \, .</math> | |
| | |
| The nth cumulant is
| |
| :<math> K_n = \frac{d^{(n)}}{d\theta^{(n)}} A(t) \, .</math>
| |
| | |
| 2. Natural exponential families (NEF) are closed under convolution.{{cn|date=July 2012}}
| |
| | |
| Given [[independent identically distributed]] (iid) <math>X_1,\ldots,X_n</math> with distribution from an NEF, then <math>\sum_{i=1}^n X_i\,</math> is an NEF, although not necessarily the original NEF. This follows from the properties of the cumulant generating function.
| |
| | |
| 3. The variance function for random variables with an NEF distribution can be written in terms of the mean.{{cn|date=July 2012}}
| |
| | |
| :<math>Var(X) = V(\mu).</math>
| |
| | |
| 4. The first two moments of a NEF distribution uniquely specify the distribution within that family of distributions.{{cn|date=July 2012}}
| |
| | |
| :<math> X \sim NEF [\mu, V(\mu)] .</math>
| |
| | |
| === Multivariate case ===
| |
| | |
| In the multivariate case, the mean vector and covariance matrix are{{cn|date=July 2012}}
| |
| :<math> \mathrm{E}[X] = \nabla A(\boldsymbol\theta)\,</math> and <math>\mathrm{Cov}[X] = \nabla \nabla^{\rm T} A(\boldsymbol\theta)\, ,</math>
| |
| where<math>\nabla</math> is the [[gradient]] and <math>\nabla \nabla^{\rm T} </math> is the [[Hessian matrix]].
| |
| | |
| ==Natural exponential families with quadratic variance functions (NEF-QVF)==
| |
| | |
| A special case of the natural exponential families are those with quadratic variance functions.
| |
| Six NEFs have quadratic variance functions (QVF) in which the variance of the distribution can be written as a quadratic function of the mean. These are called NEF-QVF. The properties of these distributions were first described by [[Carl Morris (statistician)|Carl Morris]].<ref>Morris C. (1982) "Natural exponential families with quadratic variance functions". ''Ann. Statist.'', 10(1), 65–80.</ref>
| |
| | |
| :<math>Var(X) = V(\mu) = \nu_0 + \nu_1 \mu + \nu_2 \mu^2.</math>
| |
| | |
| === The six NEF-QVFs ===
| |
| | |
| The six NEF-QVF are written here in increasing complexity of the relationship between variance and mean.
| |
| | |
| 1. The normal distribution with fixed variance <math>X \,\sim N(\mu, \sigma^2) </math> is NEF-QVF because the variance is constant. The variance can be written <math> Var(X) = V(\mu) = \sigma^2</math>, so variance is a degree 0 function of the mean.
| |
| | |
| 2. The Poisson distribution <math>X \,\sim Pois (\mu) </math> is NEF-QVF because all Poisson distributions have variance equal to the mean <math>Var(X) = V(\mu) = \mu</math>, so variance is a linear function of the mean.
| |
| | |
| 3. The Gamma distribution <math>X \,\sim Gam(r, \lambda) </math> is NEF-QVF because the mean of the Gamma distribution is <math>\mu = r\lambda</math> and the variance of the Gamma distribution is <math>Var(X) = V(\mu) = \mu^2/r</math>, so the variance is a quadratic function of the mean.
| |
| | |
| 4. The binomial distribution <math> X \,\sim Bin(n, p) </math> is NEF-QVF because the mean is <math>\mu = np</math> and the variance is <math> Var(X) = np(1-p) </math> which can be written in terms of the mean as
| |
| <math>V(X) = - np^2 + np = -\mu^2/n + \mu.</math>
| |
| | |
| 5. The negative binomial distribution <math> X \sim NegBin(n, p) </math> is NEF-QVF because the mean is <math>\mu = np/(1-p)</math> and the variance is
| |
| <math>V(\mu) = \mu^2/n + \mu.</math>
| |
| | |
| 6. The (not very famous) distribution generated by the generalized{{clarify|date=July 2012}} [[hyperbolic secant distribution]] (NEF-GHS) has{{cn|date=July 2012}}
| |
| <math>V(\mu) = \mu^2/n +n</math> and <math>\mu > 0.</math>
| |
| | |
| === Properties of NEF-QVF ===
| |
| | |
| The properties of NEF-QVF can simplify calculations that use these distributions.
| |
| | |
| 1. Natural exponential families with quadratic variance functions (NEF-QVF) are closed under convolutions of a linear transformation.{{cn|date=July 2012}} That is, a convolution of a linear transformation of an NEF-QVF is also an NEF-QVF, although not necessarily the original one.
| |
| | |
| Given [[independent identically distributed]] (iid) <math>X_1,\ldots,X_n</math> with distribution from a NEF-QVF. A convolution of a linear transformation of an NEF-QVF is also an NEF-QVF.
| |
| | |
| Let <math>Y = \sum_{i=1}^n (X_i - b)/c \,</math> be the convolution of a linear transformation of X.
| |
| The mean of Y is <math> \mu^* = n(\mu - b)/c \,</math>. The variance of Y can be written in terms of the variance function of the original NEF-QVF. If the original NEF-QVF had variance function
| |
| :<math> Var(X) = V(\mu) = \nu_0 + \nu_1 \mu + \nu_2 \mu^2,</math>
| |
| then the new NEF-QVF has variance function
| |
| :<math> Var(Y) = V^*(\mu^*) = \nu^*_0 + \nu^*_1 \mu + \nu^*_2 \mu^2 ,</math>
| |
| where
| |
| :<math> \nu^*_0 = nV(b)/c^2 \, ,</math> | |
| :<math> \nu^*_1 = V'(b)/c \, ,</math> | |
| :<math> \nu^*_2/n = \nu_2/n \, .</math> | |
| | |
| 2. Let <math> X_1</math> and <math>X_2</math> be independent NEF with the same parameter θ and let <math> Y = X_1 + X_2 </math>. Then the conditional distribution of <math>X_1</math> given Y <math>f(X_1 | Y)</math> has quadratic variance in Y if and only if <math> X_1</math> and <math>X_2</math> are NEF-QVF. Examples of conditional distributions <math>f(X_1 | Y)</math> are the [[normal distribution|normal]], [[binomial distribution|binomial]], [[beta distribution|beta]], [[hypergeometric distribution|hypergeometric]] and [[geometric distribution]]s, which are not all NEF-QVF.<ref name=Morris2006/>
| |
| | |
| 3. NEF-QVF have [[conjugate prior distribution]]s on μ in the Pearson system of distributions (also called the [[Pearson distribution]] although the Pearson system of distributions is actually a family of distributions rather than a single distribution.) Examples of conjugate prior distributions of NEF-QVF distributions are the [[normal distribution|normal]], [[gamma distribution|gamma]], reciprocal gamma, [[Beta distribution|beta]], [[F-distribution|F-]], and [[Student's t-distribution|t-]] distributions. Again, these conjugate priors are not all NEF-QVF.<ref name=Morris2006/>
| |
| | |
| 4. If <math> X | \mu </math> has an NEF-QVF distribution and μ has a conjugate prior distribution then the marginal distributions are well-known distributions.<ref name=Morris2006/>
| |
| | |
| These properties together with the above notation can simplify calculations in [[mathematical statistics]] that would normally be done using complicated calculations and calculus.
| |
| | |
| {{single source|date=June 2012}}
| |
| {{refimprove|date=June 2012}}
| |
| | |
| == References ==
| |
| {{Reflist}}
| |
| | |
| * Morris C. (1982) ''Natural exponential families with quadratic variance functions: statistical theory''. Dept of mathematics, Institute of Statistics, University of Texas, Austin.
| |
| | |
| {{ProbDistributions|families}}
| |
| | |
| [[Category:Exponentials]]
| |
| [[Category:Theory of probability distributions]]
| |
| [[Category:Types of probability distributions]]
| |