# Kronecker product

{{#invoke:Hatnote|hatnote}} In mathematics, the Kronecker product, denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors to matrices, and gives the matrix of the tensor product with respect to a standard choice of basis. The Kronecker product should not be confused with the usual matrix multiplication, which is an entirely different operation.

The Kronecker product is named after Leopold Kronecker, even though there is little evidence that he was the first to define and use it. Indeed, in the past the Kronecker product was sometimes called the Zehfuss matrix, after Johann Georg Zehfuss who in 1858 described the matrix operation we now know as the Kronecker product.[1]

## Definition

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product AB is the mp × nq block matrix:

${\displaystyle \mathbf {A} \otimes \mathbf {B} ={\begin{bmatrix}a_{11}\mathbf {B} &\cdots &a_{1n}\mathbf {B} \\\vdots &\ddots &\vdots \\a_{m1}\mathbf {B} &\cdots &a_{mn}\mathbf {B} \end{bmatrix}},}$

more explicitly:

${\displaystyle {\mathbf {A} \otimes \mathbf {B} }={\begin{bmatrix}a_{11}b_{11}&a_{11}b_{12}&\cdots &a_{11}b_{1q}&\cdots &\cdots &a_{1n}b_{11}&a_{1n}b_{12}&\cdots &a_{1n}b_{1q}\\a_{11}b_{21}&a_{11}b_{22}&\cdots &a_{11}b_{2q}&\cdots &\cdots &a_{1n}b_{21}&a_{1n}b_{22}&\cdots &a_{1n}b_{2q}\\\vdots &\vdots &\ddots &\vdots &&&\vdots &\vdots &\ddots &\vdots \\a_{11}b_{p1}&a_{11}b_{p2}&\cdots &a_{11}b_{pq}&\cdots &\cdots &a_{1n}b_{p1}&a_{1n}b_{p2}&\cdots &a_{1n}b_{pq}\\\vdots &\vdots &&\vdots &\ddots &&\vdots &\vdots &&\vdots \\\vdots &\vdots &&\vdots &&\ddots &\vdots &\vdots &&\vdots \\a_{m1}b_{11}&a_{m1}b_{12}&\cdots &a_{m1}b_{1q}&\cdots &\cdots &a_{mn}b_{11}&a_{mn}b_{12}&\cdots &a_{mn}b_{1q}\\a_{m1}b_{21}&a_{m1}b_{22}&\cdots &a_{m1}b_{2q}&\cdots &\cdots &a_{mn}b_{21}&a_{mn}b_{22}&\cdots &a_{mn}b_{2q}\\\vdots &\vdots &\ddots &\vdots &&&\vdots &\vdots &\ddots &\vdots \\a_{m1}b_{p1}&a_{m1}b_{p2}&\cdots &a_{m1}b_{pq}&\cdots &\cdots &a_{mn}b_{p1}&a_{mn}b_{p2}&\cdots &a_{mn}b_{pq}\end{bmatrix}}.}$

If A and B represent linear transformations V1W1 and V2W2, respectively, then AB represents the tensor product of the two maps, V1V2W1W2.

### Examples

${\displaystyle {\begin{bmatrix}1&2\\3&4\\\end{bmatrix}}\otimes {\begin{bmatrix}0&5\\6&7\\\end{bmatrix}}={\begin{bmatrix}1\cdot 0&1\cdot 5&2\cdot 0&2\cdot 5\\1\cdot 6&1\cdot 7&2\cdot 6&2\cdot 7\\3\cdot 0&3\cdot 5&4\cdot 0&4\cdot 5\\3\cdot 6&3\cdot 7&4\cdot 6&4\cdot 7\\\end{bmatrix}}={\begin{bmatrix}0&5&0&10\\6&7&12&14\\0&15&0&20\\18&21&24&28\end{bmatrix}}.}$

## Matrix equations

The Kronecker product can be used to get a convenient representation for some matrix equations. Consider for instance the equation AXB = C, where A, B and C are given matrices and the matrix X is the unknown. We can rewrite this equation as

${\displaystyle (\mathbf {B} ^{T}\otimes \mathbf {A} )\,\operatorname {vec} (\mathbf {X} )=\operatorname {vec} (\mathbf {AXB} )=\operatorname {vec} (\mathbf {C} ).}$

Here, vec(X) denotes the vectorization of the matrix X formed by stacking the columns of X into a single column vector. It now follows from the properties of the Kronecker product that the equation AXB = C has a unique solution if and only if A and B are nonsingular Template:Harv.

If X is row-ordered into the column vector x then AXB can be also be written as Template:Harv (ABT)x.

### Applications

For an example of the application of this formula, see the article on the Lyapunov equation. This formula also comes in handy in showing that the matrix normal distribution is a special case of the multivariate normal distribution.

## Related matrix operations {{safesubst:#invoke:anchor|main}}

Two related matrix operations are the Tracy–Singh and Khatri–Rao products which operate on partitioned matrices. Let the m × n matrix A be partitioned into the mi × nj blocks Aij and p × q matrix B into the pk × qTemplate:Ell blocks Bkl with of course Σi mi = m, Σj nj = n, Σk pk = p and ΣTemplate:Ell qTemplate:Ell = q.

### Tracy–Singh product

The Tracy–Singh product is defined as[2][3]

${\displaystyle \mathbf {A} \circ \mathbf {B} =(\mathbf {A} _{ij}\circ \mathbf {B} )_{ij}=((\mathbf {A} _{ij}\otimes \mathbf {B} _{kl})_{kl})_{ij}}$

which means that the (ij)-th subblock of the mp × nq product AB is the mi p × nj q matrix AijB, of which the (kTemplate:Ell)-th subblock equals the mi pk × nj qTemplate:Ell matrix AijBkTemplate:Ell. Essentially the Tracy-Singh product is the pairwise Kronecker product for each pair of partitions in the two matrices.

For example, if A and B both are 2 × 2 partitioned matrices e.g.:

${\displaystyle \mathbf {A} =\left[{\begin{array}{c | c}\mathbf {A} _{11}&\mathbf {A} _{12}\\\hline \mathbf {A} _{21}&\mathbf {A} _{22}\end{array}}\right]=\left[{\begin{array}{c c | c}1&2&3\\4&5&6\\\hline 7&8&9\end{array}}\right],\quad \mathbf {B} =\left[{\begin{array}{c | c}\mathbf {B} _{11}&\mathbf {B} _{12}\\\hline \mathbf {B} _{21}&\mathbf {B} _{22}\end{array}}\right]=\left[{\begin{array}{c | c c}1&4&7\\\hline 2&5&8\\3&6&9\end{array}}\right],}$

we get:

${\displaystyle \mathbf {A} \circ \mathbf {B} =\left[{\begin{array}{c | c}\mathbf {A} _{11}\circ \mathbf {B} &\mathbf {A} _{12}\circ \mathbf {B} \\\hline \mathbf {A} _{21}\circ \mathbf {B} &\mathbf {A} _{22}\circ \mathbf {B} \end{array}}\right]=\left[{\begin{array}{c | c | c | c }\mathbf {A} _{11}\otimes \mathbf {B} _{11}&\mathbf {A} _{11}\otimes \mathbf {B} _{12}&\mathbf {A} _{12}\otimes \mathbf {B} _{11}&\mathbf {A} _{12}\otimes \mathbf {B} _{12}\\\hline \mathbf {A} _{11}\otimes \mathbf {B} _{21}&\mathbf {A} _{11}\otimes \mathbf {B} _{22}&\mathbf {A} _{12}\otimes \mathbf {B} _{21}&\mathbf {A} _{12}\otimes \mathbf {B} _{22}\\\hline \mathbf {A} _{21}\otimes \mathbf {B} _{11}&\mathbf {A} _{21}\otimes \mathbf {B} _{12}&\mathbf {A} _{22}\otimes \mathbf {B} _{11}&\mathbf {A} _{22}\otimes \mathbf {B} _{12}\\\hline \mathbf {A} _{21}\otimes \mathbf {B} _{21}&\mathbf {A} _{21}\otimes \mathbf {B} _{22}&\mathbf {A} _{22}\otimes \mathbf {B} _{21}&\mathbf {A} _{22}\otimes \mathbf {B} _{22}\end{array}}\right]}$
${\displaystyle =\left[{\begin{array}{c c | c c c c | c | c c}1&2&4&7&8&14&3&12&21\\4&5&16&28&20&35&6&24&42\\\hline 2&4&5&8&10&16&6&15&24\\3&6&6&9&12&18&9&18&27\\8&10&20&32&25&40&12&30&48\\12&15&24&36&30&45&18&36&54\\\hline 7&8&28&49&32&56&9&36&63\\\hline 14&16&35&56&40&64&18&45&72\\21&24&42&63&48&72&27&54&81\end{array}}\right].}$

### Khatri–Rao product

The Khatri–Rao product is defined as[4][5]

${\displaystyle \mathbf {A} \ast \mathbf {B} =(\mathbf {A} _{ij}\otimes \mathbf {B} _{ij})_{ij}}$

in which the ij-th block is the mipi × njqj sized Kronecker product of the corresponding blocks of A and B, assuming the number of row and column partitions of both matrices is equal. The size of the product is then (Σi mipi) × (Σj njqj). Proceeding with the same matrices as the previous example we obtain:

${\displaystyle \mathbf {A} \ast \mathbf {B} =\left[{\begin{array}{c | c}\mathbf {A} _{11}\otimes \mathbf {B} _{11}&\mathbf {A} _{12}\otimes \mathbf {B} _{12}\\\hline \mathbf {A} _{21}\otimes \mathbf {B} _{21}&\mathbf {A} _{22}\otimes \mathbf {B} _{22}\end{array}}\right]=\left[{\begin{array}{c c | c c}1&2&12&21\\4&5&24&42\\\hline 14&16&45&72\\21&24&54&81\end{array}}\right].}$

This is a submatrix of the Tracy-Singh product of the two matrices (each partition in this example is a partition in a corner of the Tracy-Singh product).

A column-wise Kronecker product of two matrices may also be called the Khatri-Rao product. This product assumes the partitions of the matrices are their columns. In this case m1 = m, p1 = p, n = q and for each j: nj = pj = 1. The resulting product is a mp × n matrix of which each column is the Kronecker product of the corresponding columns of A and B. Using the matrices from the previous examples with the columns partitioned:

${\displaystyle \mathbf {C} =\left[{\begin{array}{c | c | c}\mathbf {C} _{1}&\mathbf {C} _{2}&\mathbf {C} _{3}\end{array}}\right]=\left[{\begin{array}{c | c | c}1&2&3\\4&5&6\\7&8&9\end{array}}\right],\quad \mathbf {D} =\left[{\begin{array}{c | c | c }\mathbf {D} _{1}&\mathbf {D} _{2}&\mathbf {D} _{3}\end{array}}\right]=\left[{\begin{array}{c | c | c }1&4&7\\2&5&8\\3&6&9\end{array}}\right],}$

so that:

${\displaystyle \mathbf {C} \ast \mathbf {D} =\left[{\begin{array}{c | c | c }\mathbf {C} _{1}\otimes \mathbf {D} _{1}&\mathbf {C} _{2}\otimes \mathbf {D} _{2}&\mathbf {C} _{3}\otimes \mathbf {D} _{3}\end{array}}\right]=\left[{\begin{array}{c | c | c }1&8&21\\2&10&24\\3&12&27\\4&20&42\\8&25&48\\12&30&54\\7&32&63\\14&40&72\\21&48&81\end{array}}\right].}$

## Notes

1. {{#invoke:citation/CS1|citation |CitationClass=citation }}
2. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
3. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
4. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
5. {{#invoke:citation/CS1|citation |CitationClass=citation }}

## References

• {{#invoke:citation/CS1|citation

|CitationClass=citation }}.

• {{#invoke:citation/CS1|citation

|CitationClass=citation }}.

• {{#invoke:citation/CS1|citation

|CitationClass=citation }}

• {{#invoke:citation/CS1|citation

|CitationClass=citation }}