Scheme (mathematics)

In statistics, the Wishart distribution is a generalization to multiple dimensions of the chi-squared distribution, or, in the case of non-integer degrees of freedom, of the gamma distribution. It is named in honor of John Wishart, who first formulated the distribution in 1928.

It is any of a family of probability distributions defined over symmetric, nonnegative-definite matrix-valued random variables (“random matrices”). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics. In Bayesian statistics, the Wishart distribution is the conjugate prior of the inverse covariance-matrix of a multivariate-normal random-vector.

Definition

Suppose X is an n × p matrix, each row of which is independently drawn from a p-variate normal distribution with zero mean:

$X_{(i)}=(x_i^1,\dots ,x_i^p{)}^T\sim N_p(0,V).$

Then the Wishart distribution is the probability distribution of the p×p random matrix

$S=X^{T}X$

known as the scatter matrix. One indicates that S has that probability distribution by writing

$S\sim W_p(V,n).$

The positive integer n is the number of degrees of freedom. Sometimes this is written W(V, p, n). For n ≥ p the matrix S is invertible with probability 1 if V is invertible.

If p = 1 and V = 1 then this distribution is a chi-squared distribution with n degrees of freedom.

Occurrence

The Wishart distribution arises as the distribution of the sample covariance matrix for a sample from a multivariate normal distribution.Potter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park. It occurs frequently in likelihood-ratio tests in multivariate statistical analysis. It also arises in the spectral theory of random matricesPotter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park. and in multidimensional Bayesian analysis.Potter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park.

Probability density function

The Wishart distribution can be characterized by its probability density function as follows:

Let $\mathbf{X}$ be a p × p symmetric matrix of random variables that is positive definite. Let $\mathbf{V}$ be a (fixed) positive definite matrix of size p × p.

Then, if n ≥ p, $\mathbf{X}$ has a Wishart distribution with n degrees of freedom if it has a probability density function given by

$\frac{1}{2^{\frac{np}{2}}{\left|\mathbf{V}\right|}^{\frac{n}{2}}{\mathrm{\Gamma }}_p(\frac{n}{2})}{\left|\mathbf{X}\right|}^{\frac{n-p-1}{2}}{e}^{-\frac{1}{2}\mathrm{t}\mathrm{r}({\mathbf{V}}^{-1}\mathbf{X})}$

where Γ_p(·) is the multivariate gamma function defined as

${\mathrm{\Gamma }}_p(n/2)={\pi }^{p(p-1)/4}{\mathrm{\Pi }}_{j=1}^p\mathrm{\Gamma }[n/2+(1-j)/2].$

In fact the above definition can be extended to any real n > p − 1. If n ≤ p − 2, then the Wishart no longer has a density—instead it represents a singular distribution.

Use in Bayesian statistics

In Bayesian statistics, in the context of the multivariate normal distribution, the Wishart distribution is the conjugate prior to the precision matrix $\mathrm{\Omega }={\mathrm{\Sigma }}^{-1}$, where $\mathrm{\Sigma }$ is the covariance matrix.

Choice of W

The least informative, proper Wishart prior is obtained by setting $n=p$.

The prior mean of $W_p(\mathbf{V},n)$ is $n{\mathbf{V}}^{-1}$. This implies that a good choice for $\mathbf{V}$ is $n{\mathrm{\Sigma }}_0$, where ${\mathrm{\Sigma }}_0$ is some prior guess for the covariance matrix.

Properties

Log-expectation

Note the following formula:

$\mathrm{E}[\ln |\mathbf{X}|]=\sum _{i=1}^p\psi (\frac{1}{2}(n+1-i))+p\ln (2)+\ln \left|\mathbf{V}\right|$

where ψ is the digamma function (the derivative of the log of the gamma function).

This plays a role in variational Bayes derivations for Bayes networks involving the Wishart distribution.

Entropy

The information entropy of the distribution has the following formula:

$\mathrm{H}[\mathbf{X}]=-\ln (B(\mathbf{V},n))-\frac{1}{2}(n-p-1)\mathrm{E}[\ln |\mathbf{X}|]+\frac{np}{2}$

where $B(\mathbf{V},n)$ is the normalizing constant of the distribution:

$B(\mathbf{V},n)=\frac{1}{{\left|\mathbf{V}\right|}^{\frac{n}{2}}{2}^{\frac{np}{2}}{\mathrm{\Gamma }}_p(\frac{n}{2})}$

This can be expanded as follows:

$\begin{array}{rl}\mathrm{H}[\mathbf{X}]& =\frac{n}{2}\ln \left|\mathbf{V}\right|+\frac{np}{2}\ln (2)+\ln ({\mathrm{\Gamma }}_p(\frac{n}{2}))-\frac{1}{2}(n-p-1)\mathrm{E}[\ln |\mathbf{X}|]+\frac{np}{2}\\ & =\frac{n}{2}\ln \left|\mathbf{V}\right|+\frac{np}{2}\ln (2)+\frac{1}{4}p(p-1)\ln (\pi )+\sum _{i=1}^p\ln \left(\mathrm{\Gamma }(\frac{n}{2}+\frac{1-i}{2})\right)\\ & -\frac{1}{2}(n-p-1)(\sum _{i=1}^p\psi (\frac{1}{2}(n+1-i))+p\ln (2)+\ln |\mathbf{V}\left|\right)+\frac{np}{2}\\ & =\frac{n}{2}\ln \left|\mathbf{V}\right|+\frac{np}{2}\ln (2)+\frac{1}{4}p(p-1)\ln (\pi )+\sum _{i=1}^p\ln \left(\mathrm{\Gamma }(\frac{n}{2}+\frac{1-i}{2})\right)\\ & -(\frac{1}{2}(n-p-1)\sum _{i=1}^p\psi (\frac{1}{2}(n+1-i))+\frac{1}{2}(n-p-1)p\ln (2)+\frac{1}{2}(n-p-1)\ln |\mathbf{V}\left|\right)+\frac{np}{2}\\ & =\frac{p+1}{2}\ln \left|\mathbf{V}\right|+\frac{1}{2}p(p+1)\ln (2)+\frac{1}{4}p(p-1)\ln (\pi )+\sum _{i=1}^p\ln \left(\mathrm{\Gamma }(\frac{n}{2}+\frac{1-i}{2})\right)-\frac{1}{2}(n-p-1)\sum _{i=1}^p\psi (\frac{1}{2}(n+1-i))+\frac{np}{2}\end{array}$

Characteristic function

The characteristic function of the Wishart distribution is

$\mathrm{\Theta }\mapsto {|\mathbf{I}-2i\mathrm{\Theta }\mathbf{V}|}^{-\frac{n}{2}}.$

In other words,

$\mathrm{\Theta }\mapsto \mathrm{E}\left[\mathrm{e}\mathrm{x}\mathrm{p}(i\mathrm{t}\mathrm{r}(\mathbf{X}\mathrm{\Theta }))\right]={|\mathbf{I}-2i\mathrm{\Theta }\mathbf{V}|}^{-\frac{n}{2}}$

where E[⋅] denotes expectation. (Here Θ and I are matrices the same size as V (I is the identity matrix); and i is the square root of −1).

Theorem

If $\mathbf{X}$ has a Wishart distribution with m degrees of freedom and variance matrix $\mathbf{V}$—write $\mathbf{X}\sim {\mathcal{𝒲}}_p(\mathbf{V},m)$—and $\mathbf{C}$ is a q × p matrix of rank q, then

$\mathbf{C}\mathbf{X}{\mathbf{C}}^T\sim {\mathcal{𝒲}}_q(\mathbf{C}\mathbf{V}{\mathbf{C}}^T,m).$

Corollary 1

If $\mathbf{z}$ is a nonzero $p\times 1$ constant vector, then ${\mathbf{z}}^T\mathbf{X}\mathbf{z}\sim {\sigma }_z^2{\chi }_m^2$.

In this case, ${\chi }_m^2$ is the chi-squared distribution and ${\sigma }_z^2={\mathbf{z}}^T\mathbf{V}\mathbf{z}$ (note that ${\sigma }_z^2$ is a constant; it is positive because $\mathbf{V}$ is positive definite).

Corollary 2

Consider the case where ${\mathbf{z}}^T=(0,\dots ,0,1,0,\dots ,0)$ (that is, the jth element is one and all others zero). Then corollary 1 above shows that

$w_{jj}\sim {\sigma }_{jj}{\chi }_m^2$

gives the marginal distribution of each of the elements on the matrix's diagonal.

Noted statistician George Seber points outPotter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park. that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefersPotter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park. to reserve the term multivariate for the case when all univariate marginals belong to the same family.

Estimator of the multivariate normal distribution

The Wishart distribution is the sampling distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. A derivation of the MLE uses the spectral theorem.

Bartlett decomposition

The Bartlett decomposition of a matrix $\mathbf{X}$ from a p-variate Wishart distribution with scale matrix V and n degrees of freedom is the factorization:

$\mathbf{X}=\mathbf{\text{<mi fromhbox="1">L</mi>}}\mathbf{\text{<mi fromhbox="1">A</mi>}}{\mathbf{\text{<mi fromhbox="1">A</mi>}}}^T{\mathbf{\text{<mi fromhbox="1">L</mi>}}}^T$

where L is the Cholesky decomposition of V, and:

$\mathbf{A}=\left(\begin{array}{ccccc}\sqrt{c_1}& 0& 0& \cdots & 0\\ n_{21}& \sqrt{c_2}& 0& \cdots & 0\\ n_{31}& n_{32}& \sqrt{c_3}& \cdots & 0\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ n_{p1}& n_{p2}& n_{p3}& \cdots & \sqrt{c_p}\end{array}\right)$

where $c_i\sim {\chi }_{n-i+1}^2$ and $n_{ij}\sim N(0,1)$ independently. This provides a useful method for obtaining random samples from a Wishart distribution.

The possible range of the shape parameter

It can be shown that the Wishart distribution can be defined if and only if the shape parameter n belongs to the set

${\mathrm{\Lambda }}_p:=\{0,\dots ,p-1\}\cup (p-1,\mathrm{\infty }).$

This set is named after Gindikin, who introduced it in the seventies in the context of gamma distributions on homogeneous cones. However, for the new parameters in the discrete spectrum of the Gindikin ensemble, namely,

${\mathrm{\Lambda }}_p^{*}:=\{0,\dots ,p-1\},$

the corresponding Wishart distribution has no Lebesgue density.

Relationships to other distributions

• The Wishart distribution is related to the Inverse-Wishart distribution, denoted by $W_p^{-1}$, as follows: If $\mathbf{X}\sim W_p(\mathbf{V},n)$ and if we do the change of variables $\mathbf{C}={\mathbf{X}}^{-1}$, then $\mathbf{C}\sim W_p^{-1}({\mathbf{V}}^{-1},n)$. This relationship may be derived by noting that the absolute value of the Jacobian determinant of this change of variables is $|\mathbf{C}{|}^{p+1}$, see for example equation (15.15) in.
• In Bayesian statistics, the Wishart distribution is a conjugate prior for the precision parameter of the multivariate normal distribution, when the mean parameter is known.
• A generalization is the multivariate gamma distribution.
• A different type of generalization is the normal-Wishart distribution, essentially the product of a multivariate normal distribution with a Wishart distribution.

See also

• Chi-squared distribution
• F-distribution
• Gamma distribution
• Hotelling's T-squared distribution
• Inverse-Wishart distribution
• Multivariate gamma distribution
• Student's t-distribution
• Wilks' lambda distribution

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

External links

• A C++ library for random matrix generator

55 yrs old Metal Polisher Records from Gypsumville, has interests which include owning an antique car, summoners war hack and spelunkering. Gets immense motivation from life by going to places such as Villa Adriana (Tivoli).

my web site - summoners war hack no survey ios