View source for ISO week date

In [[control theory]], the '''linear-quadratic-Gaussian (LQG) control problem''' is one of the most fundamental [[optimal control]] problems. It concerns uncertain [[linear system]]s disturbed by [[additive white Gaussian noise]], having incomplete state information (i.e. not all the state variables are measured and available for feedback) and undergoing control subject to quadratic [[cost functional|cost]]s. Moreover the solution is unique and constitutes a linear dynamic feedback control law that is easily computed and implemented. Finally the LQG controller is also fundamental to the optimal control of perturbed non-linear systems.<ref name="Athans">{{cite journal |author=Athans M. |title=The role and use of the stochastic Linear-Quadratic-Gaussian problem in control system design |journal=IEEE Transaction on Automatic Control |volume=AC-16 |issue=6 |pages=529–552 |year=1971 |doi=10.1109/TAC.1971.1099818}}</ref>

The '''LQG controller''' is simply the combination of a [[Kalman filter]] i.e. a linear-quadratic estimator (LQE) with a [[linear-quadratic regulator]] (LQR). The [[separation principle]] guarantees that these can be designed and computed independently. LQG control applies to both [[LTI systems|linear time-invariant systems]] as well as linear time-varying systems. The application to linear time-invariant systems is well known. The application to [[Time-variant system|linear time-varying system]]s enables the design of linear feedback controllers for non-linear uncertain systems.

The LQG controller itself is a dynamic system like the system it controls. Both systems have the same state dimension. Therefore implementing the LQG controller may be problematic if the dimension of the system state is large. The '''reduced-order LQG problem''' (fixed-order LQG problem) overcomes this by fixing a-priori the number of states of the LQG controller. This problem is more difficult to solve because it is no longer separable. Also the solution is no longer unique. Despite these facts numerical algorithms are available<ref name="Wil1">{{cite journal |author=Van Willigenburg L.G., De Koning W.L. |title=Numerical algorithms and issues concerning the discrete-time optimal projection equations |journal=European Journal of Control |volume=6 |issue=1 |pages=93–100 |year=2000}} [http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=19948&objectType=file  Associated software download from Matlab Central].</ref><ref name="Wil2">{{cite journal |author=Van Willigenburg L.G., De Koning W.L. |title=Optimal reduced-order compensators for time-varying discrete-time systems with deterministic and white parameters |journal=Automatica |volume=35 |pages=129–138 |year=1999 |doi=10.1016/S0005-1098(98)00138-1}} [http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=20014&objectType=FILE  Associated software download from Matlab Central].</ref><ref name="Bern3">{{cite journal |author=Zigic D., Watson L.T., Collins E.G., Haddad W.M., Ying S. |title=Homotopy methods for solving the optimal projection equations for the H2 reduced order model problem |journal=International Journal of Control |volume=56 | issue=1 | pages=173–191 |year=1996 |doi=10.1080/00207179208934308}}</ref><ref name="Had1">{{cite journal |author=Collins Jr. E.G, Haddad W.M., Ying S. |title=A homotopy algorithm for reduced-order dynamic compensation using the Hyland-Bernstein optimal projection equations |journal=Journal of Guidance Control & Dynamics |volume=19 |pages=407–417 |year=1996 |doi=10.2514/3.21633 |issue=2}}</ref> to solve the associated [[optimal projection equations]]<ref name="Bern1">{{cite journal |author=Hyland D.C, Bernstein D.S. |title=The optimal projection equations for fixed order dynamic compensation |journal=IEEE Transaction on Automatic Control |volume=AC-29 | pages=1034–1037 |year=1984 |doi=10.1109/TAC.1984.1103418 |issue=11}}</ref><ref name="Bern2">{{cite journal |author=Bernstein D.S., Davis L.D., Hyland D.C. |title=The optimal projection equations for reduced-order discrete-time modeling estimation and control |journal=Journal of Guidance Control and Dynamics| volume=9 |issue=3 |pages=288–293 |year=1986 |doi=10.2514/3.20105}}</ref> which constitute necessary and sufficient conditions for a locally optimal reduced-order LQG controller.<ref name="Wil1"/>

Finally, a word of caution. LQG optimality does not automatically ensure good robustness properties.<ref>Green, Limebeer: Linear Robust Control, p. 27</ref> The robust stability of the closed loop system must be checked separately after the LQG controller has been designed. To promote robustness some of the system parameters may be assumed stochastic instead of deterministic. The associated more difficult control problem leads to a similar optimal controller of which only the controller parameters are different.<ref name="Wil2"/>

==Mathematical description of the problem and solution==

===Continuous time===
Consider the linear dynamic system,

: <math>\dot{\mathbf{x}}(t) = A(t) \mathbf{x}(t) + B(t) \mathbf{u}(t) +  \mathbf{v}(t)</math>
: <math>\mathbf{y}(t) = C(t) \mathbf{x}(t) + \mathbf{w}(t),</math>

where <math>{\mathbf{x}}</math> represents the vector of state variables of the system, <math>{\mathbf{u}}</math> the vector of control inputs and <math>{\mathbf{y}}</math> the vector of measured outputs available for feedback. Both additive white Gaussian system noise <math>\mathbf{v}(t)</math> and additive white Gaussian measurement noise <math>\mathbf{w}(t)</math> affect the system. Given this system the objective is to find the control input history <math>{\mathbf{u}}(t)</math> which at every time <math>{\mathbf{}}t</math> may depend only on the past measurements <math>{\mathbf{y}}(t'), 0 \leq t' < t</math> such that the following cost function is minimized,

: <math> J = E\left({\mathbf{x}^\mathrm T}(T)F{\mathbf{x}}(T)+ \int_{0}^{T} {\mathbf{x}^\mathrm T}(t)Q(t){\mathbf{x}}(t) + {\mathbf{u}^\mathrm T}(t)R(t){\mathbf{u}}(t)\,dt \right),</math>

: <math> F \ge 0,\quad Q(t) \ge 0,\quad R(t) > 0, </math>

where <math>{\mathbf{}}E</math> denotes the [[expected value]]. The final time (horizon) <math>{\mathbf{}}T</math> may be either finite or infinite. If the horizon tends to infinity the first term <math>{\mathbf{x}}^\mathrm T(T)F{\mathbf{x}}(T)</math> of the cost function becomes negligible and irrelevant to the problem. Also to keep the costs finite the cost function has to be taken to be <math>{\mathbf{}}J/T</math>. 

The LQG controller that solves the LQG control problem is specified by the following equations,

: <math> \dot{\hat{\mathbf{x}}}(t) = A(t)\hat{\mathbf{x}}(t) + B(t){\mathbf{u}}(t)+K(t) \left( {\mathbf{y}}(t)-C(t)\hat{\mathbf{x}}(t) \right),  \hat{\mathbf{x}}(0) = E \left( {\mathbf{x}}(0) \right) </math>

: <math> {\mathbf{u}}(t)= -L(t) \hat{\mathbf{x}}(t).</math>

The matrix <math>{\mathbf{}}K(t)</math> is called the '''Kalman gain''' of the associated [[Kalman filter]] represented by the first equation. At each time <math>{\mathbf{}}t</math> this filter generates estimates <math>\hat{\mathbf{x}}(t)</math> of the state <math>{\mathbf{x}}(t)</math> using the past measurements and inputs. The Kalman gain <math>{\mathbf{}}K(t)</math> is computed from the matrices <math>{\mathbf{}}A(t), C(t)</math>, the two intensity matrices <math>\mathbf{}V(t), W(t)</math> associated to the white Gaussian noises <math>\mathbf{v}(t)</math> and <math>\mathbf{w}(t)</math> and finally <math>E\left({\mathbf{x}}(0){\mathbf{x}}^\mathrm T(0) \right)</math>. These five matrices determine the Kalman gain through the following associated matrix Riccati differential equation,

: <math> \dot{P}(t) = A(t)P(t)+P(t)A^\mathrm T(t)-P(t)C^\mathrm T(t){\mathbf{}}W^{-1}(t)
C(t)P(t)+V(t),</math>

: <math> P(0)= E \left({\mathbf{x}}(0){\mathbf{x}}^\mathrm T(0) \right).</math>

Given the solution <math>P(t), 0 \leq t \leq T</math> the Kalman gain equals,

: <math> {\mathbf{}}K(t) = P(t)C^\mathrm T(t)W^{-1}(t) </math>

The matrix <math>{\mathbf{}}L(t)</math> is called the '''feedback gain''' matrix. This matrix is determined by the matrices <math>{\mathbf{}}A(t), B(t), Q(t), R(t)</math> and <math>{\mathbf{}}F</math> through the following associated matrix Riccati differential equation,

: <math> -\dot{S}(t) = A^\mathrm T(t)S(t)+S(t)A(t)-S(t)B(t)R^{-1}(t)B^\mathrm T(t)S(t)+Q(t),</math>

: <math>  {\mathbf{}}S(T) = F.</math>

Given the solution <math>{\mathbf{}}S(t), 0 \leq t \leq T</math> the feedback gain equals,

: <math> {\mathbf{}}L(t) = R^{-1}(t)B^\mathrm T(t)S(t).</math>

Observe the similarity of the two matrix Riccati differential equations, the first one running forward in time, the second one running backward in time. This similarity is called '''duality'''. The first matrix Riccati differential equation solves the linear-quadratic estimation problem (LQE). The second matrix Riccati differential equation solves the [[linear-quadratic regulator]] problem (LQR). These problems are dual and together they solve the linear-quadratic-Gaussian control problem (LQG). So the LQG problem separates into the LQE and LQR problem that can be solved independently. Therefore the LQG problem is called '''separable'''.

When <math>{\mathbf{}}A(t), B(t), C(t), Q(t), R(t)</math> and the noise intensity matrices <math>\mathbf{}V(t)</math>, <math>\mathbf{}W(t)</math> do not depend on <math>{\mathbf{}}t</math> and when <math>{\mathbf{}}T</math> tends to infinity the LQG controller becomes a time-invariant dynamic system. In that case both matrix Riccati differential equations may be replaced by the two associated [[algebraic Riccati equation]]s.

===Discrete time===
Since the discrete-time LQG control problem is similar to the one in continuous-time the description below focuses on the mathematical equations.

Discrete-time linear system equations:

: <math>{\mathbf{x}}_{i+1} = A_i\mathbf{x}_i + B_i \mathbf{u}_i +  \mathbf{v}_i</math>

: <math>\mathbf{y}_{i} = C_{i} \mathbf{x}_i + \mathbf{w}_i</math>

Here <math>\mathbf{}i</math> represents the discrete time index and <math>\mathbf{v}_{i}, \mathbf{w}_{i}</math> represent discrete-time Gaussian white noise processes with covariance matrices <math>\mathbf{}V_{i}, W_{i}</math> respectively.

The quadratic cost function to be minimized:

: <math> J = E\left({\mathbf{x}}^\mathrm T_{N}F{\mathbf{x}}_{N}+ \sum_{i=0}^{N-1} \mathbf{x}_i^\mathrm T Q_i \mathbf{x}_i + \mathbf{u}_i^\mathrm T R_i \mathbf{u}_i \right),</math>

: <math> F \ge 0, Q_i \ge 0, R_i > 0. \,  </math>

The discrete-time LQG controller:

:<math>\hat{\mathbf{x}}_{i+1}=A_i\hat{\mathbf{x}}_i+B_i{\mathbf{u}}_i+K_i \left({\mathbf{y}}_i-C_i{\hat{\mathbf{x}}}_i \right), \hat{\mathbf{x}}_0=E({\mathbf{x}}_0)</math>,

:<math> \mathbf{u}_i=-L_i\hat{\mathbf{x}}_i. \, </math>

The Kalman gain equals,

: <math> {\mathbf{}}K_i = P_iC ^\mathrm T _i(C_iP_iC ^\mathrm T _i + W_i)^{-1}, </math>

where <math>{\mathbf{}}P_i</math> is determined by the following matrix Riccati difference equation that runs forward in time,

: <math> P_{i+1} = A_i \left( P_i - P_i C ^\mathrm T _i \left( C_i P_i C ^\mathrm T _i+W_i \right)^{-1} C_i P_i \right) A ^\mathrm T _i+V_i, P_0=E \left( {\mathbf{x}}_0{\mathbf{x}}^\mathrm T_0 \right). </math>

The feedback gain matrix equals,

: <math> {\mathbf{}}L_i = (B^\mathrm T_iS_{i+1}B_i + R_i)^{-1}B^\mathrm T_iS_{i+1}A_i. </math>

where <math>{\mathbf{}}S_i</math> is determined by the following matrix Riccati difference equation that runs backward in time,

: <math> S_i = A^\mathrm T_i \left( S_{i+1} - S_{i+1}B_i \left( B^\mathrm T_iS_{i+1}B_i+R_i \right)^{-1} B^\mathrm T_i S_{i+1} \right) A_i+Q_i, S_N=F.</math>

If all the matrices in the problem formulation are time-invariant and if the horizon <math>{\mathbf{}}N</math> tends to infinity the discrete-time LQG controller becomes time-invariant. In that case the matrix Riccati difference equations may be replaced by their associated discrete-time [[algebraic Riccati equation]]s. These determine the time-invarant linear-quadratic estimator and the time-invariant [[linear-quadratic regulator]] in discrete-time. To keep the costs finite instead of <math>{\mathbf{}}J</math> one has to consider <math>{\mathbf{}}J/N</math> in this case.

==See also==

*[[Stochastic control]]
*[[Witsenhausen's counterexample]]

==References==
{{reflist}}

{{DEFAULTSORT:Linear-Quadratic-Gaussian Control}}
[[Category:Optimal control]]
[[Category:Control theory]]
[[Category:Stochastic control]]