View source for Trend stationary

In [[statistics]], [[econometrics]], [[epidemiology]] and related disciplines, the method of '''instrumental variables''' ('''IV''') is used to estimate [[causal relationships]]{{Cn|date=April 2013}} when [[controlled experiment]]s are not feasible or when a treatment is not successfully delivered to every unit in a randomized experiment.<ref name=Imbens:00>{{cite journal |last=Imbens |first=G. |first2=J. |last2=Angrist |year=1994 |title=Identification and estimation of local average treatment effects |journal=[[Econometrica]] |volume=62 |issue=2 |pages=467–476 |jstor=2951620 }}</ref>

Instrumental variable methods allow [[consistent estimator|consistent]] estimation when the [[dependent and independent variables|explanatory variables]] (covariates) are [[correlation|correlated]] with the [[errors and residuals in statistics|error terms]] of a [[regression analysis|regression]] relationship.   Such correlation may occur when the dependent variable causes at least one of the [[covariate]]s ("reverse" causation), when there are relevant explanatory variables which are omitted from the model, or when the covariates are subject to measurement error. In this situation, ordinary [[linear regression]] generally produces biased and inconsistent estimates.<ref name=Bullock:00>{{cite journal |last=Bullock |first=J. G. |last2=Green |first2=D. P. |last3=Ha |first3=S. E. |year=2010 |title=Yes, But What’s the Mechanism? (Don’t Expect an Easy Answer) |journal=[[Journal of Personality and Social Psychology]] |volume=98 |issue=4 |pages=550-558 |doi=10.1037/a0018933 }}</ref> However, if an ''instrument'' is available, consistent estimates may still be obtained. An instrument is a variable that does not itself belong in the explanatory equation and is correlated with the [[endogeneity (econometrics)|endogenous]] explanatory variables, conditional on the other covariates. In linear models, there are two main requirements for using an IV:
* The instrument must be correlated with the endogenous explanatory variables, conditional on the other covariates.
* The instrument cannot be correlated with the error term in the explanatory equation, that is, the instrument cannot suffer from the same problem as the original predicting variable.

==Definitions==
Formal definitions of instrumental variables, using counterfactuals and graphical criteria, are given by [[Judea Pearl|Pearl]] (2000).<ref name=Pearl:00>{{cite book |last=Pearl |first=J. |title=Causality: Models, Reasoning, and Inference |publisher=[[Cambridge University Press]] |year=2000 |location=New York |isbn=052189560X }}</ref>  Notions of causality in econometrics, and their relationship with instrumental variables and other methods, are discussed by [[James Heckman|Heckman]] (2008).<ref name=Heckman:00>{{cite journal |last=Heckman |first=J. |year=2008 |title=Econometric Causality |journal=[[International Statistical Review]] |volume=76 |issue=1 |pages=1–27 |doi=10.1111/j.1751-5823.2007.00024.x }}</ref>

The theory of instrumental variables was first derived by Philip G. Wright in his 1928 book ''The Tariff on Animal and Vegetable Oils''.<ref>{{cite journal |last1=Stock |first1=James H. |last2=Trebbi |first2=Francesco  |year=2003 |title=Retrospectives: Who Invented Instrumental Variable Regression?  |journal=[[Journal of Economic Perspectives]] |volume=17 |issue=3 |pages=177–194 |doi=10.1257/089533003769204416 }}</ref><!-- ref is helpful because the authorship of the appendix B was disputed and this references attempts to settle the dispute -->

==Example==

Informally, in attempting to estimate the causal effect of some variable ''x'' on another ''y'', an instrument is a third variable ''z'' which affects ''y'' only through its effect on ''x''.  For example, suppose a researcher wishes to estimate the causal effect of smoking on general health.<ref name=Leigh:00>{{cite journal |last=Leigh |first=J. P. |first2=M. |last2=Schembri |year=2004 |title=Instrumental Variables Technique: Cigarette Price Provided Better Estimate of Effects of Smoking on SF-12 |journal=[[Journal of Clinical Epidemiology]] |volume=57 |issue=3 |pages=284–293 |doi=10.1016/j.jclinepi.2003.08.006 }}</ref>  Correlation between health and smoking does not imply that smoking causes poor health because other variables may affect both health and smoking, or because health may affect smoking in addition to smoking causing health problems.  It is at best difficult and expensive to conduct controlled experiments on smoking status in the general population.  The researcher may proceed to attempt to estimate the causal effect of smoking on health from observational data by using the tax rate on tobacco products as an instrument for smoking in a causal analysis.  If tobacco taxes affect health ''only'' because they affect smoking (holding other variables in the model fixed), correlation between tobacco taxes and health is evidence that smoking causes changes in health.  An estimate of the effect of smoking on health can be made by also making use of the correlation between taxes and smoking patterns.

Because demonstrating that the third variable 'z' is causally related to 'y' ''exclusively'' via 'x' is an experimental impossibility, and because the same limitations that prevent an experiment to determine if there is a causal relationship between 'x' and 'y' will normally also prevent experiments determining if there is a causal relationship between 'z' and 'y' (assumed to be mediated through 'x'), correlational data is the only type of evidence that analysis by instrumental variable can provide, and causal inference is not licensed. The use of an instrumental variable simply produces additional evidence of a statistical relationship (in this case between 'z' and 'y'), without providing evidence of what type of relationship it is, and without providing evidence for the type of relationship between 'x' and 'y'.

==Applications==

IV methods are commonly used to estimate causal effects in contexts in which controlled experiments are not available.  Credibility of the estimates hinges on the selection of suitable instruments.  Good instruments are often created by policy changes. For example, the cancellation of a federal student-aid scholarship program may reveal the effects of aid on some students' outcomes.  Other [[natural experiment|natural and quasi-natural experiments]] of various types are commonly exploited, for example, Miguel, Satyanath, and Sergenti (2004) use weather shocks to identify the effect of changes in economic growth (i.e., declines) on civil conflict.<ref name=miguel:00>{{cite journal |last=Miguel |first=E. |last2=Satyanath |first2=S. |last3=Sergenti |first3=E. |year=2004 |title=Economic Shocks and Civil Conflict: An Instrumental Variable Approach |journal=[[Journal of Political Economy]] |volume=112 |issue=4 |pages=725–753 |doi=10.1086/421174 }}</ref>  [[Joshua Angrist|Angrist]] and [[Alan Krueger|Krueger]] (2001) present a survey of the history and uses of instrumental variable techniques.<ref name=angrist:00>{{cite journal |last=Angrist |first=J. |first2=A. |last2=Krueger |year=2001 |title=Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments |journal=[[Journal of Economic Perspectives]] |volume=15 |issue=4 |pages=69–85 |doi=10.1257/jep.15.4.69 }}</ref>

==Estimation==

Suppose the data are generated by a process of the form

: <math> y_i = \beta x_i + \varepsilon_i, </math>

where
* ''i'' indexes observations,
* <math>y_i</math> is the dependent variable,
* <math>x_i</math> is an independent variable,
* <math>\varepsilon_i</math> is an unobserved error term representing all causes of <math>y_i</math> other than <math>x_i</math>, and
* <math>\beta</math> is an unobserved scalar parameter.

The parameter <math>\beta</math> is the causal effect on <math>y_i</math> of a one unit change in <math>x_i</math>, holding all other causes of <math>y_i</math> constant.  The econometric goal is to estimate <math>\beta</math>.  For simplicity's sake assume the draws of <math>\varepsilon</math> are uncorrelated and that they are drawn from distributions with the same [[variance]], that is, that the errors are serially uncorrelated and [[homoskedastic]].

Suppose also that a regression model of nominally the same form is proposed.  Given a random sample of ''T'' observations from this process, the [[least squares|ordinary least squares]] estimator is

: <math> \widehat{\beta}_\mathrm{OLS} = \frac{ x^\mathrm{T} y }{ x^\mathrm{T}x} = \frac{ x^\mathrm{T}(x\beta + \varepsilon )}{ x^\mathrm{T}x} =
\beta + \frac{x^\mathrm{T} \varepsilon}{ x^\mathrm{T}x}.</math>

where ''x'', ''y'' and <math>\varepsilon</math> denote column vectors of length ''T''.  When ''x'' and <math> \varepsilon</math> are [[correlation|uncorrelated]], under certain regularity conditions the second term has an expected value conditional on ''x'' of zero and converges to zero in the limit, so the estimator is [[estimator bias|unbiased]] and consistent.  When ''x'' and the other unmeasured, causal variables collapsed into the <math> \varepsilon</math> term are correlated, however, the OLS estimator is generally biased and inconsistent for β.  In this case, it is valid to use the estimates to predict values of ''y'' given values of ''x'', but the estimate does not recover the causal effect of ''x'' on ''y''.

An instrumental variable ''z'' is one that is correlated with the independent variable but not with the [[errors and residuals in statistics|error]] term.  Using the [[method of moments (statistics)|method of moments]], take expectations conditional on ''z'' to find

:<math> E [ y | z ] = \beta E [ x | z ] + E [ \varepsilon | z ]. \, </math>

The second term on the right-hand side is zero by assumption.  Solve for <math>\beta</math> and write the resulting expression in terms of sample moments,

:<math> \widehat{\beta}_\mathrm{IV} = \frac{z^\mathrm{T} y}{ z^\mathrm{T} x } = \beta + \frac{z^\mathrm{T} \varepsilon}{z^\mathrm{T} x}. \, </math>

When ''z'' and <math> \varepsilon</math> are uncorrelated, the final term, under certain regularity conditions, approaches zero in the limit, providing a consistent estimator.  Put another way, the causal effect of ''x'' on ''y'' can be consistently estimated from these data even though ''x'' is not randomly assigned through experimental methods.

The approach generalizes to a model with multiple explanatory variables.  Suppose ''X'' is the ''T × K'' matrix of explanatory variables resulting from ''T'' observations on ''K'' variables.  Let ''Z'' be a ''T × K'' matrix of instruments.  Then it can be shown that the estimator

:<math> \widehat{\beta}_\mathrm{IV} = (Z^\mathrm{T} X)^{-1}Z^\mathrm{T} y \, </math>

is consistent under a multivariate generalization of the conditions discussed above.  If there are more instruments than there are covariates in the equation of interest so that ''Z'' is a ''T × M'' matrix with ''M > K'', the [[generalized method of moments]] (GMM) can be used and the resulting IV estimator is

:<math> \widehat{\beta}_\mathrm{GMM} = (X^\mathrm{T} P_Z X)^{-1}X^\mathrm{T} P_Z y,</math>

where <math>P_Z=Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T}</math>.

Note that the second expression collapses to the first when the number of instruments is equal to the number of covariates in the equation of interest (just-identified case).

{{Collapse top|title=Proof that  β<sub>GMM</sub> collapses to β<sub>IV</sub> in the just-identifed case}}
Developing the <math>\beta_{GMM}</math> expression:
:<math> \widehat{\beta}_\mathrm{GMM} = (X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} X)^{-1}X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y</math>

In the just-identified case, we have as many instruments as covariates, so that the dimension of X is the same of Z. Hence, <math>X^\mathrm{T} Z, Z^\mathrm{T} Z</math> and <math>Z^\mathrm{T}X</math> are all squared matrices of the same dimension. We can expand the inverse, using the fact that, for any invertible ''n''-by-''n'' matrices '''A''' and '''B''', ('''AB''')<sup>−1</sup> = '''B'''<sup>−1</sup>'''A'''<sup>−1</sup> (see [[Invertible matrix#Properties]]):
:<math>
\begin{align}
\widehat{\beta}_\mathrm{GMM} &= (Z^\mathrm{T} X)^{-1}(Z^\mathrm{T} Z)(X^\mathrm{T} Z)^{-1}X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y\\
&=  (Z^\mathrm{T} X)^{-1}(Z^\mathrm{T} Z)(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y\\
&=(Z^\mathrm{T} X)^{-1}Z^\mathrm{T}y \\
&=\widehat{\beta}_\mathrm{IV}
\end{align}
</math>
Reference: see Davidson and Mackinnnon (1993)<ref>{{cite book|last1=Davidson|first1=Russell|first2=James|last2=Mackinnon|title=Estimation and Inference in Econometrics|year=1993|publisher=Oxford University Press|location=New York|isbn=0-19-506011-3}}</ref>{{rp|218}}
{{Collapse bottom}}

==Interpretation as two-stage least squares==

One computational method which can be used to calculate IV estimates is two-stage least-squares (2SLS or TSLS).  In the first stage, each explanatory variable that is an endogenous covariate in the equation of interest is regressed on all of the exogenous variables in the model, including both exogenous covariates in the equation of interest and the excluded instruments.  The predicted values from these regressions are obtained.

'''Stage 1:''' Regress each column of '''X''' on '''Z''', (<math> X = Z \delta + \text{errors} </math>)

: <math>\widehat{\delta}=(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T}X, \,</math>

and save the predicted values:

: <math>\widehat{X}= Z\widehat{\delta} = Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T}X = P_Z X.\, </math>

In the second stage, the regression of interest is estimated as usual, except that in this stage each endogenous covariate is replaced with the predicted values from the first stage.

'''Stage 2:''' Regress '''Y''' on the predicted values from the first stage:

: <math> Y = \widehat X \beta + \mathrm{noise}.\,</math>

Which gives:
: <math> \beta_{2SLS} = \left(X^\mathrm{T}P_Z X\right)^{-1} X^\mathrm{T}P_ZY</math>
{{Collapse top|title=Proof: computation of the 2SLS estimator}}
Note that the usual OLS estimator is: <math> (\widehat X^\mathrm{T}\widehat X)^{-1}\widehat X^\mathrm{T}Y</math>.
Replacing <math> \widehat X^\mathrm{T} = P_Z X</math> and noting that <math>P_Z </math> is a symmetric and [[Idempotence|idempotent]] matrix, so that <math>  P_Z^\mathrm{T}P_Z=P_Z P_Z = P_Z</math>
: <math> \beta_{2SLS} = (\widehat X^\mathrm{T}\widehat X)^{-1}\widehat X^\mathrm{T} Y = \left(X^\mathrm{T}P_Z^\mathrm{T}P_Z X\right)^{-1} X^\mathrm{T}P_Z^\mathrm{T}Y=\left(X^\mathrm{T}P_Z X\right)^{-1} X^\mathrm{T}P_ZY</math>

{{Collapse bottom}}

The resulting estimator of <math>\beta</math> is numerically identical to the expression displayed above.   A small correction must be made to the sum-of-squared residuals in the second-stage fitted model in order that the covariance matrix of <math>\beta</math> is calculated correctly.

==Identification==

In the instrumental variable regression, if we have multiple endogenous regressors  <math>x_1 \dots x_k </math> and multiple instruments <math>z_1 \dots z_m </math> the coefficients on the endogenous regressors <math>\beta_1 \dots \beta_k </math>are said to be:

[[Identifiability|Exactly identified]] if ''m'' = ''k''.<br />
'''Overidentified''' if ''m'' > ''k''.<br />
[[Parameter identification problem|Underidentified]] if ''m'' < ''k''.

The parameters are underidentified (equivalently, not identified) if there are fewer instruments than there are covariates or, equivalently, if there are fewer excluded instruments than there are endogenous covariates in the equation of interest.

==Non-parametric analysis==

When the form of the structural equations is unknown, an instrumental variable <math>Z</math> can still be defined through the equations:
:<math>x = g(z,u)</math>
:<math>y = f(x,u)</math>
where <math>f</math> and <math>g</math> are two arbitrary functions and <math>Z</math> is independent of <math>U</math>. Unlike linear models, however, measurements of <math>Z, X</math> and <math>Y</math> do not allow for the identification of the average causal effect of <math>X</math> on <math>Y</math>, denoted ACE
:<math>\mbox{ACE} = \mbox{Pr}(y|\mbox{do}(x)) = \mbox{E}_u[f(x,u)].</math>
Balke and Pearl [1997] derived tight bounds on ACE and showed that these can provide valuable information on the sign and size of ACE.<ref name=balke:pearl97>{{cite journal |last=Balke |first=A. |last2=Pearl |first2=J. |title=Bounds on treatment effects from studies with imperfect compliance |journal=[[Journal of the American Statistical Association]] |volume=92 |issue=439 |pages=1172–1176 |year=1997 |doi=10.1080/01621459.1997.10474074 }}</ref>

In linear analysis, there is no test to falsify the assumption the <math>Z</math> is instrumental relative to the pair <math>(X,Y)</math>. This is not the case when <math>X</math> is discrete. Pearl (2000) has shown that, for all <math>f</math> and <math>g</math>, the following constraint, called "Instrumental Inequality" must hold whenever <math>Z</math> satisfies the two equations above:<ref name=Pearl:00 />
:<math>\max_x \sum_y [\max_z \Pr(y,x|z)]\leq 1.</math>

==On the interpretation of IV estimates==

The exposition above assumes that the causal effect of interest does not vary across observations, that is, that <math>\beta</math> is a constant.  Generally, different subjects will respond differently to changes in the "treatment" ''x''.  When this possibility is recognized, the average effect in the population of a change in ''x'' on ''y'' may differ from the effect in a given subpopulation.  For example, the average effect of a job training program may  substantially differ across the group of people who actually receive the training and the group which chooses not to receive training.  For these reasons, IV methods invoke implicit assumptions on behavioral response, or more generally assumptions over the correlation between the response to treatment and propensity to receive treatment.<ref>{{cite journal |last=Heckman |first=J. |year=1997 |title=Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations |journal=[[Journal of Human Resources]] |volume=32 |issue=3 |pages=441–462 |jstor=146178 }}</ref>

The standard IV estimator can recover [[local average treatment effects]] (LATE) rather than [[average treatment effects]] (ATE).<ref name=Imbens:00 />  Imbens and Angrist (1994) demonstrate that the linear IV estimate can be interpreted under weak conditions as a weighted average of local average treatment effects, where the weights depend on the elasticity of the endogenous regressor to changes in the instrumental variables. Roughly, that means that the effect of a variable is only revealed for the subpopulations affected by the observed changes in the instruments, and that subpopulations which respond most to changes in the instruments will have the largest effects on the magnitude of the IV estimate.

For example, if a researcher uses presence of a land-grant college as an instrument for college education in an earnings regression, she identifies the effect of college on earnings in the subpopulation which would obtain a college degree if a college is present but which would not obtain a degree if a college is not present.  This empirical approach does not, without further assumptions, tell the researcher anything about the effect of college among people who would either always or never get a college degree regardless of whether a local college exists.

==Potential problems==

Instrumental variables estimates are generally inconsistent if the instruments are correlated with the error term in the equation of interest.  Another problem is caused by the selection of "weak" instruments, instruments that are poor predictors of the endogenous question predictor in the first-stage equation. In this case, the prediction of the question predictor by the instrument will be poor and the predicted values will have very little variation. Consequently, they are unlikely to have much success in predicting the ultimate outcome when they are used to replace the question predictor in the second-stage equation.

In the context of the smoking and health example discussed above, tobacco taxes are weak instruments for smoking if smoking status is largely unresponsive to changes in taxes.  If higher taxes do not induce people to quit smoking (or not start smoking), then variation in tax rates tells us nothing about the effect of smoking on health.  If taxes affect health through channels other than through their effect on smoking, then the instruments are invalid and the instrumental variables approach may yield misleading results.  For example, places and times with relatively health-conscious populations may both implement high tobacco taxes and exhibit better health even holding smoking rates constant, so we would observe a correlation between health and tobacco taxes even if it were the case that smoking has no effect on health.  In this case, we would be mistaken to infer a causal effect of smoking on health from the observed correlation between tobacco taxes and health.

==Sampling properties and hypothesis testing==

When the covariates are exogenous, the small-sample properties of the OLS estimator can be derived in a straightforward manner by calculating moments of the estimator conditional on ''X''.  When some of the covariates are endogenous so that instrumental variables estimation is implemented, simple expressions for the moments of the estimator cannot be so obtained.  Generally, instrumental variables estimators only have desirable asymptotic, not finite sample, properties, and inference is based on asymptotic approximations to the sampling distribution of the estimator.  Even when the instruments are uncorrelated with the error in the equation of interest and when the instruments are not weak, the finite sample properties of the instrumental variables estimator may be poor.  For example, exactly identified models produce finite sample estimators with no moments, so the estimator can be said to be neither biased nor unbiased, the nominal size of test statistics may be substantially distorted, and the estimates may commonly be far away from the true value of the parameter.<ref name="Nelson 1990:00">{{cite journal |last=Nelson |first=C. R. |first2=R. |last2=Startz |year=1990 |title=Some Further Results on the Exact Small Sample Properties of the Instrumental Variable Estimator |journal=[[Econometrica]] |volume=58 |issue=4 |pages=967–976 |jstor=2938359 }}</ref>

==Testing instrument strength and overidentifying restrictions==

The strength of the instruments can be directly assessed because both the endogenous covariates and the instruments are observable.<ref name=Stock:00>{{cite journal |last=Stock |first=J. |first2=J. |last2=Wright |first3=M. |last3=Yogo |year=2002 |title=A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments |journal=[[Journal of the American Statistical Association]] |volume=20 |issue=4 |pages=518–529 |doi=10.1198/073500102288618658 }}</ref>  A common rule of thumb for models with one endogenous regressor is: the [[F-test|F-statistic]] against the [[Null hypothesis|null]] that the excluded instruments are irrelevant in the first-stage regression should be larger than 10.

The assumption that the instruments are not correlated with the error term in the equation of interest is not testable in exactly identified models.  If the model is overidentified, there is information available which may be used to test this assumption.  The most common test of these ''overidentifying restrictions'', called the [[Sargan test]], is based on the observation that the residuals should be uncorrelated with the set of exogenous variables if the instruments are truly exogenous.  The Sargan test statistic can be calculated as <math>TR^2</math> (the number of observations multiplied by the [[coefficient of determination]]) from the OLS regression of the residuals onto the set of exogenous variables.  This statistic will be asymptotically chi-squared with ''m''&nbsp;−&nbsp;''k'' degrees of freedom under the null that the error term is uncorrelated with the instruments.

==References==
{{Reflist|30em}}

==Further reading==
*{{cite journal |last=Imbens |first=Guido W. |first2=Jeffrey M. |last2=Wooldridge |year=2009 |title=Recent Developments in the Econometrics of Program Evaluation |journal=[[Journal of Economic Literature]] |volume=47 |issue=1 |pages=5–86 |doi=10.1257/jel.47.1.5 }}
*{{cite book |last=Wooldridge |first=Jeffrey M. |year=2013 |chapter=Instrumental Variables Estimation and Two-Stages Least Squares |title=Introductory Econometrics: A Modern Approach |location=Mason, OH |publisher=South-Western |edition=Fifth international |pages=490–528 |isbn=978-1-111-53439-4 }}

==External links==
* [http://www.urban.org/toolkit/data-methods/instrumental.cfm Layman's explanation] of instrumental variables.
* [http://emlab.berkeley.edu/users/mcfadden/e240b_f01/ch4.pdf Chapter] from [[Daniel McFadden]]'s textbook

[[Category:Regression analysis]]
[[Category:Simultaneous equation methods (econometrics)]]
[[Category:Mathematical and quantitative methods (economics)]]
[[Category:Econometrics]]