|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| In [[statistics]], the '''Pearson product-moment correlation coefficient''' ({{IPAc-en|ˈ|p|ɪər|s|ɨ|n}}) (sometimes referred to as the '''PPMCC''' or '''PCC''',<ref>"The human disease network", Albert Barabasi et al., Plos.org</ref> or '''Pearson's ''r''''') is a measure of the linear [[correlation]] (dependence) between two variables ''X'' and ''Y'', giving a value between +1 and −1 inclusive, where 1 is total positive correlation, 0 is no correlation, and −1 is total negative correlation. It is widely used in the sciences as a measure of the degree of linear dependence between two variables. It was developed by [[Karl Pearson]] from a related idea introduced by [[Francis Galton]] in the 1880s.<ref name="thirteenways">J. L. Rodgers and W. A. Nicewander. [http://www.jstor.org/stable/2685263 Thirteen ways to look at the correlation coefficient]. The American Statistician, 42(1):59–66, February 1988.</ref><ref>{{Cite journal| doi = 10.1214/ss/1177012580| last = Stigler | first = Stephen M. | title = Francis Galton's Account of the Invention of Correlation | journal = Statistical Science | volume=4 | issue=2 | pages = 73–79 | year = 1989 | jstor=2245329}}</ref>
| | If your computer is running slow, we have probably gone by the different stages of rage and frustration. Having such a terrific tool like a computer could seem like a curse along with a blessing simultaneously when this arises. It is ideal whenever it is running quickly and smooth, yet then when it starts acting weird and slows means down, frustration sets inside. How will something as fantastic because a computer make a individual so mad?<br><br>You require to recognize several convenient and cheap ways that could solve the problem of your computer plus speed it up. The earlier you fix it, the less damage the computer gets. I can tell about several helpful techniques which may aid you to accelerate you computer.<br><br>When you compare registry products you want a rapidly acting registry cleaning. It's no superior spending hours and your PC waiting for your registry cleaning to complete its task. We desire the cleaner to complete its task in minutes.<br><br>How to fix this problem is to initially reinstall the program(s) causing the mistakes. There are a lot of different programs which use this file, nevertheless one can have placed their own faulty variation of the file onto your program. By reinstalling any programs that are causing the error, we will not just enable a PC to run the system correctly, however a brand-new file may be placed onto a program - exiting a computer running as smoothly because possible again. If you try this, and find it does not work, then we should look to update your system & any software we have on the PC. This will likely update the Msvcr71.dll file, permitting the computer to read it properly again.<br><br>The final step is to make certain which we clean the registry of your computer. The "registry" is a large database that stores significant files, settings & choices, plus info. Windows reads the files it requirements in purchase for it to run programs by this database. If the registry gets damaged, afflicted, or clogged up, then Windows are not able to correctly access the files it needs for it to load up programs. As this arises, issues plus mistakes like the d3d9.dll error occur. To fix this and avoid future setbacks, you must download plus run a registry cleaning tool. The very suggested software is the "Frontline [http://bestregistrycleanerfix.com/registry-mechanic pc tools registry mechanic]".<br><br>Why this problem arises frequently? What are the causes of it? In fact, there are 3 main causes that can cause the PC freezing issue. To resolve the problem, we should take 3 procedures in the following paragraphs.<br><br>Maybe you may be asking how come these windows XP error messages appear. Well, for you to be capable to understand the fix, you need to first understand where those errors come from. There is this software called registry. A registry is software that shops everything on a PC from a normal configuration, setting, information, and logs of activities from installing to UN-installing, saving to deleting, and a lot more alterations we do in your system pass through it plus gets 'tagged' and saved because a simple file for recovery reasons. Imagine it as a big recorder, a registrar, of all your records inside the PC.<br><br>Ally Wood is a professional software reviewer and has worked in CNET. Now she is functioning for her own review software organization to give suggestions to the software creator plus has done deep test inside registry cleaner software. After reviewing the top registry cleaner, she has created complete review on a review site for you that can be accessed for free. |
| | |
| [[File:Correlation coefficient.gif|thumb|400px|right|Examples of scatter diagrams with different values of correlation coefficient (ρ)]]
| |
| [[Image:Correlation examples2.svg|thumb|400px|right|Several sets of (''x'', ''y'') points, with the correlation coefficient of ''x'' and ''y'' for each set. Note that the correlation reflects the non-linearity and direction of a linear relationship (top row), but not the slope of that relationship (middle), nor many aspects of nonlinear relationships (bottom). N.B.: the figure in the center has a slope of 0 but in that case the correlation coefficient is undefined because the variance of ''Y'' is zero.]]
| |
| | |
| ==Definition==
| |
| Pearson's correlation coefficient between two variables is defined as the [[covariance]] of the two variables divided by the product of their [[standard deviations]]. The form of the definition involves a "product moment", that is, the mean (the first moment about the origin) of the product of the mean-adjusted random variables; hence the modifier ''product-moment'' in the name.
| |
| | |
| ===For a population===
| |
| Pearson's correlation coefficient when applied to a population is commonly represented by the Greek letter ''ρ'' (rho) and may be referred to as the ''population correlation coefficient'' or the ''population Pearson correlation coefficient''. The formula for ''ρ'' is:
| |
| | |
| :<math> \rho_{X,Y}={\mathrm{cov}(X,Y) \over \sigma_X \sigma_Y} ={E[(X-\mu_X)(Y-\mu_Y)] \over \sigma_X\sigma_Y} </math>
| |
| | |
| where, <math> \mathrm{cov} </math> is the covariance, <math> \sigma_X </math> is the [[standard deviation]] of <math> X </math>, <math> \mu_X </math> is the [[mean]] of <math> X </math>, and <math> E </math> is the expectation.
| |
| | |
| ===For a sample===
| |
| Pearson's correlation coefficient when applied to a sample is commonly represented by the letter ''r'' and may be referred to as the ''sample correlation coefficient'' or the ''sample Pearson correlation coefficient''. We can obtain a formula for ''r'' by substituting estimates of the covariances and variances based on a [[statistical sample|sample]] into the formula above. That formula for ''r'' is:
| |
| | |
| :<math>r = \frac{\sum ^n _{i=1}(X_i - \bar{X})(Y_i - \bar{Y})}{\sqrt{\sum ^n _{i=1}(X_i - \bar{X})^2} \sqrt{\sum ^n _{i=1}(Y_i - \bar{Y})^2}}</math>
| |
| | |
| An equivalent expression gives the correlation coefficient as the mean of the products of the [[standard score]]s. Based on a [[Statistical sample|sample]] of paired data (''X''<sub>''i''</sub>, ''Y''<sub>''i''</sub>), the sample Pearson correlation coefficient is
| |
| | |
| :<math>r = \frac{1}{n-1} \sum ^n _{i=1} \left( \frac{X_i - \bar{X}}{s_X} \right) \left( \frac{Y_i - \bar{Y}}{s_Y} \right)</math>
| |
| | |
| where
| |
| | |
| :<math>\frac{X_i - \bar{X}}{s_X},\,\bar{X}=\frac{1}{n}\sum_{i=1}^n X_i, \text{ and } s_X=\sqrt{\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar{X})^2}</math>
| |
| | |
| are the [[standard score]], sample [[mean]], and sample [[standard deviation]], respectively.
| |
| | |
| ==Mathematical properties==
| |
| The absolute values of both the sample and population Pearson correlation coefficients are less than or equal to 1. Correlations equal to 1 or -1 correspond to data points lying exactly on a line (in the case of the sample correlation), or to a bivariate distribution entirely supported on a line (in the case of the population correlation). The Pearson correlation coefficient is symmetric: ''corr''(''X'',''Y'') = ''corr''(''Y'',''X'').
| |
| | |
| A key mathematical property of the Pearson correlation coefficient is that it is [[invariant estimator|invariant]] (up to a sign) to separate changes in location and scale in the two variables. That is, we may transform ''X'' to ''a'' + ''bX'' and transform ''Y'' to ''c'' + ''dY'', where ''a'', ''b'', ''c'', and ''d'' are constants, without changing the correlation coefficient (this fact holds for both the population and sample Pearson correlation coefficients). Note that more general linear transformations do change the correlation: see [[#Removing correlation|a later section]] for an application of this.
| |
| | |
| The Pearson correlation can be expressed in terms of uncentered moments. Since ''μ''<sub>''X''</sub> = E(''X''), ''σ''<sub>''X''</sub><sup>2</sup> = E[(''X'' − E(''X''))<sup>2</sup>] = E(''X''<sup>2</sup>) − E<sup>2</sup>(''X'') and
| |
| likewise for ''Y'', and since
| |
| | |
| : <math>E[(X-E(X))(Y-E(Y))]=E(XY)-E(X)E(Y),\,</math>
| |
| | |
| the correlation can also be written as
| |
| | |
| :<math>\rho_{X,Y}=\frac{E(XY)-E(X)E(Y)}{\sqrt{E(X^2)-(E(X))^2}~\sqrt{E(Y^2)- (E(Y))^2}}.</math>
| |
| | |
| Alternative formulae for the ''sample'' Pearson correlation coefficient are also available:
| |
| | |
| :<math>
| |
| r_{xy}=\frac{\sum x_iy_i-n \bar{x} \bar{y}}{(n-1) s_x s_y}=\frac{n\sum x_iy_i-\sum x_i\sum y_i}
| |
| {\sqrt{n\sum x_i^2-(\sum x_i)^2}~\sqrt{n\sum y_i^2-(\sum y_i)^2}}.
| |
| </math>
| |
| | |
| The second formula above needs to be corrected for a sample:
| |
| :<math>
| |
| r_{xy}=\frac{\sum x_iy_i-n \bar{x} \bar{y}}{(n-1) s_x s_y}=\frac{n\sum x_iy_i-\sum x_i\sum y_i}
| |
| {\sqrt{(n-1)\sum x_i^2-(\sum x_i)^2}~\sqrt{(n-1)\sum y_i^2-(\sum y_i)^2}}.
| |
| </math>
| |
| | |
| The above formula suggests a convenient single-pass algorithm for calculating sample correlations, but, depending on the numbers involved, it can sometimes be [[numerical stability|numerically unstable]].
| |
| | |
| ==Interpretation==
| |
| The correlation coefficient ranges from −1 to 1. A value of 1 implies that a linear equation describes the relationship between ''X'' and ''Y'' perfectly, with all data points lying on a [[line (mathematics)|line]] for which ''Y'' increases as ''X'' increases. A value of −1 implies that all data points lie on a line for which ''Y'' decreases as ''X'' increases. A value of 0 implies that there is no linear correlation between the variables.
| |
| | |
| More generally, note that (''X''<sub>''i''</sub> − <span style="text-decoration: overline;">''X''</span>)(''Y''<sub>''i''</sub> − <span style="text-decoration: overline;">''Y''</span>) is positive if and only if ''X''<sub>''i''</sub> and ''Y''<sub>''i''</sub> lie on the same side of their respective means. Thus the correlation coefficient is positive if ''X''<sub>''i''</sub> and ''Y''<sub>''i''</sub> tend to be simultaneously greater than, or simultaneously less than, their respective means. The correlation coefficient is negative if ''X''<sub>''i''</sub> and ''Y''<sub>''i''</sub> tend to lie on opposite sides of their respective means.
| |
| | |
| ===Geometric interpretation===
| |
| [[File:Regression lines.png|thumb|upright=1.5|Regression lines for y=g<sub>x</sub>(x) [red] and x=g<sub>y</sub>(y) [blue]]]
| |
| | |
| For uncentered data, it is possible to obtain a relation between correlation coefficient and the angle <math>\varphi</math> between both possible [[regression line]]s y=g<sub>x</sub>(x) and x=g<sub>y</sub>(y). One can show <ref>{{cite journal|last=Schmid Jr.|first=John|title=The Relationship between the Coefficient of Correlation and the Angle Included between Regression Lines|journal=The Journal of Educational Research|date=December 1947|volume= 41|issue= 4|url=http://www.jstor.org/stable/27528906}}</ref> that r = sec(<math>\ \theta</math>) -tan(<math>\ \theta</math>).
| |
| | |
| For centered data (i.e., data which have been shifted by the sample mean so as to have an average of zero), the correlation coefficient can also be viewed as the [[cosine]] of the [[angle]] <math>\ \theta</math> between the two [[Vector (geometry)|vectors]] of samples drawn from the two random variables (see below).
| |
| | |
| Both the uncentered (non-Pearson-compliant) and centered correlation coefficients can be determined for a dataset. As an example, suppose five countries are found to have gross national products of 1, 2, 3, 5, and 8 billion dollars, respectively. Suppose these same five countries (in the same order) are found to have 11%, 12%, 13%, 15%, and 18% poverty. Then let '''x''' and '''y''' be ordered 5-element vectors containing the above data: '''x''' = (1, 2, 3, 5, 8) and '''y''' = (0.11, 0.12, 0.13, 0.15, 0.18).
| |
| | |
| By the usual procedure for finding the angle <math>\ \theta</math> between two vectors (see [[dot product]]), the ''uncentered'' correlation coefficient is:
| |
| | |
| <!-- cos theta = (X dot Y) / ||X|| ||Y|| = 2.93 / sqrt(103 * 0.0983) = 0.920814711. -->
| |
| :<math> \cos \theta = \frac { \bold{x} \cdot \bold{y} } { \left\| \bold{x} \right\| \left\| \bold{y} \right\| } = \frac { 2.93 } { \sqrt { 103 } \sqrt { 0.0983 } } = 0.920814711. </math>
| |
| | |
| Note that the above data were deliberately chosen to be perfectly correlated: ''y'' = 0.10 + 0.01 ''x''. The Pearson correlation coefficient must therefore be exactly one. Centering the data (shifting '''x''' by E('''x''') = 3.8 and '''y''' by E('''y''') = 0.138) yields '''x''' = (−2.8, −1.8, −0.8, 1.2, 4.2) and '''y''' = (−0.028, −0.018, −0.008, 0.012, 0.042), from which
| |
| | |
| <!-- cos theta = (X dot Y) / ||X|| ||Y|| = 0.308 / sqrt(30.8 * 0.00308) = 1. -->
| |
| :<math> \cos \theta = \frac { \bold{x} \cdot \bold{y} } { \left\| \bold{x} \right\| \left\| \bold{y} \right\| } = \frac { 0.308 } { \sqrt { 30.8 } \sqrt { 0.00308 } } = 1 = \rho_{xy}, </math>
| |
| | |
| as expected.
| |
| | |
| Further interpretation of the correlation coefficient can be found in .<ref name="thirteenways"/>
| |
| | |
| ===Interpretation of the size of a correlation===
| |
| [[File:Pearson correlation and prediction intervals.svg|thumb|200px|right|This figure gives a sense of how the usefulness of a Pearson correlation for predicting values varies with its magnitude. Given jointly normal ''X'', ''Y'' with correlation ''ρ'', <math>1 - \sqrt{1 - \rho^2}</math> (plotted here as a function of ''ρ'') is the factor by which a given [[w:Prediction interval|prediction interval]] for ''Y'' may be reduced given the corresponding value of ''X''. For example, if ''ρ'' = .5, then the 95% prediction interval of ''Y''|''X'' will be about 13% smaller than the 95% prediction interval of ''Y''.]]
| |
| | |
| Several authors<ref name="Buda">A. Buda and A.Jarynowski (2010) ''Life-time of correlations and its applications vol.1'', Wydawnictwo Niezalezne: 5–21, December 2010, ISBN 978-83-915272-9-0</ref><ref name="Cohen88"/> have offered guidelines for the interpretation of a correlation coefficient. However, all such criteria are in some ways arbitrary and should not be observed too strictly.<ref name="Cohen88">Cohen, J. (1988). ''Statistical power analysis for the behavioral sciences'' (2nd ed.)</ref> The interpretation of a correlation coefficient depends on the context and purposes. A correlation of 0.8 may be very low if one is verifying a physical law using high-quality instruments, but may be regarded as very high in the social sciences where there may be a greater contribution from complicating factors.
| |
| | |
| ===Pearson’s distance===
| |
| A distance metric for two variables X and Y known as ''Pearson's distance'' can be defined from their correlation coefficient as<ref>Fulekar (Ed.), M.H. (2009) ''Bioinformatics: Applications in Life and Environmental Sciences'', Springer (pp. 110) ISBN 1-4020-8879-5</ref>
| |
| :<math>d_{X,Y}=1-\rho_{X,Y}.</math>
| |
| Considering that the Pearson correlation coefficient falls between [-1, 1], the Pearson distance lies in [0, 2].
| |
| | |
| ==Inference==
| |
| [[Image:correlation significance.svg|300px|right|thumb|A graph showing the minimum value of Pearson's correlation coefficient that is significantly different from zero at the 0.05 level, for a given sample size.]]Statistical inference based on Pearson's correlation coefficient often focuses on one of the following two aims:
| |
| * One aim is to test the [[null hypothesis]] that the true correlation coefficient ''ρ'' is equal to 0, based on the value of the sample correlation coefficient ''r''.
| |
| * The other aim is to construct a [[confidence interval]] around ''r'' that has a given probability of containing ''ρ''.
| |
| We discuss methods of achieving one or both of these aims below.
| |
| | |
| ===Use a permutation test===
| |
| [[Resampling (statistics)#Permutation tests|Permutation tests]] provide a direct approach to performing hypothesis tests and constructing confidence intervals. A permutation test for Pearson's correlation coefficient involves the following two steps:
| |
| # Using the original paired data (''x''<sub>''i''</sub>, ''y''<sub>''i''</sub>), randomly redefine the pairs to create a new data set (''x''<sub>''i''</sub>, ''y''<sub>''i′''</sub>), where the ''i′'' are a permutation of the set {1,...,''n''}. The permutation ''i′'' is selected randomly, with equal probabilities placed on all ''n''! possible permutations. This is equivalent to drawing the ''i′'' randomly "'''without replacement'''" from the set {1, ..., ''n''}. A closely related and equally justified ([[Bootstrapping (statistics)|bootstrapping]]) approach is to separately draw the ''i'' and the ''i′'' "'''with replacement'''" from {1, ..., ''n''};
| |
| # Construct a correlation coefficient ''r'' from the randomized data.
| |
| To perform the permutation test, repeat steps (1) and (2) a large number of times. The [[p-value]] for the permutation test is the proportion of the ''r'' values generated in step (2) that are larger than the Pearson correlation coefficient that was calculated from the original data. Here "larger" can mean either that the value is larger in magnitude, or larger in signed value, depending on whether a [[two-tailed test|two-sided]] or [[two-tailed test|one-sided]] test is desired.
| |
| | |
| ===Use a bootstrap===
| |
| The [[bootstrapping (statistics)|bootstrap]] can be used to construct confidence intervals for Pearson's correlation coefficient. In the "non-parametric" bootstrap, ''n'' pairs (''x''<sub>''i''</sub>, ''y''<sub>''i''</sub>) are resampled "with replacement" from the observed set of ''n'' pairs, and the correlation coefficient ''r'' is calculated based on the resampled data. This process is repeated a large number of times, and the empirical distribution of the resampled ''r'' values are used to approximate the [[sampling distribution]] of the statistic. A 95% [[confidence interval]] for ''ρ'' can be defined as the interval spanning from the 2.5<sup>''th''</sup> to the 97.5<sup>''th''</sup> [[percentile]] of the resampled ''r'' values.
| |
| | |
| ===Testing using Student's t-distribution===
| |
| For pairs from an uncorrelated [[bivariate normal distribution]], the [[sampling distribution]] of Pearson's correlation coefficient follows [[Student's t-distribution]] with degrees of freedom ''n'' − 2. Specifically, if the underlying variables have a bivariate normal distribution, the variable
| |
| | |
| :<math>t = r\sqrt{\frac{n-2}{1 - r^2}}</math>
| |
| | |
| has a Student's t-distribution in the null case (zero correlation).<ref>Rahman, N.A. (1968) ''A Course in Theoretical Statistics'', Charles Griffin and Company, 1968</ref> This also holds approximately even if the observed values are non-normal, provided sample sizes are not very small.<ref>Kendall, M.G., Stuart, A. (1973) ''The Advanced Theory of Statistics, Volume 2: Inference and Relationship'', Griffin. ISBN 0-85264-215-6 (Section 31.19)</ref> For determining the critical values for ''r'' the inverse of this transformation is also needed:
| |
| | |
| :<math>r = \frac{t}{\sqrt{n - 2 + t^2}}.</math>
| |
| | |
| Alternatively, large sample approaches can be used.
| |
| | |
| Early work on the distribution of the sample correlation coefficient was carried out by [[R. A. Fisher]]<ref>{{Cite journal
| |
| | last = Fisher | first = R.A.
| |
| | authorlink = R. A. Fisher
| |
| | title = Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population
| |
| | journal = [[Biometrika]]
| |
| | volume = 10
| |
| | issue = 4
| |
| | pages = 507–521
| |
| | year = 1915
| |
| | doi = 10.1093/biomet/10.4.507
| |
| }}</ref><ref>{{Cite journal
| |
| | last = Fisher | first = R.A.
| |
| | authorlink = R. A. Fisher
| |
| | title = On the probable error of a coefficient of correlation deduced from a small sample
| |
| | journal = [[Metron (journal)|Metron]]
| |
| | year = 1921
| |
| | volume = 1
| |
| | issue = 4
| |
| | pages = 3–32
| |
| | url = http://hdl.handle.net/2440/15169
| |
| | accessdate = 2009-03-25
| |
| | format = [[PDF]]
| |
| }}</ref>
| |
| and A. K. Gayen.<ref>{{Cite journal
| |
| | first = A.K. | last = Gayen
| |
| | title = The frequency distribution of the product moment correlation coefficient in random samples of any size draw from non-normal universes
| |
| | journal = [[Biometrika]]
| |
| | year = 1951
| |
| | volume = 38
| |
| | pages = 219–247
| |
| | doi = 10.1093/biomet/38.1-2.219
| |
| }}</ref>
| |
| Another early paper<ref>[[H. E. Soper|Soper, H.E.]], Young, A.W., Cave, B.M., Lee, A., Pearson, K. (1917). "On the distribution of the correlation coefficient in small samples. Appendix II to the papers of "Student" and R. A. Fisher. A co-operative study", ''[[Biometrika]]'', 11, 328-413. {{doi|10.1093/biomet/11.4.328}}</ref> provides graphs and tables for general values of ''ρ'', for small sample sizes, and discusses computational approaches.
| |
| | |
| ===Use the exact distribution===
| |
| For data that follows a [[bivariate normal distribution]], the exact density function for the sample correlation of a normal bivariate is<ref>Kenney, J. F. and Keeping, E. S., ''Mathematics of Statistics'', Pt. 2, 2nd ed. Princeton, NJ: Van Nostrand, 1951.</ref><ref>[http://mathworld.wolfram.com/CorrelationCoefficientBivariateNormalDistribution.html Correlation Coefficient - Bivariate Normal Distribution]</ref>
| |
| | |
| :<math>f\left(r\right) = \frac{\left(n - 2\right)\, \mathbf{\Gamma}\left(n - 1\right) \left(1 - \rho^2\right)^{\frac{n - 1}{2}} \left(1 - r^2\right)^{\frac{n - 4}{2}}}{\sqrt{2\pi}\, \mathbf{\Gamma}\left(n - \frac{1}{2}\right) \left(1 - \rho r\right)^{n - \frac{3}{2}}} \,\mathbf{_2F_1}\left(\frac{1}{2}, \frac{1}{2}; \frac{2n - 1}{2}; \frac{\rho r + 1}{2}\right)</math>
| |
| | |
| where <math>\mathbf{\Gamma}</math> is the [[gamma function]], <math>\,\mathbf{_2F_1}(a,b;c;z)</math> is the [[hypergeometric function|Gaussian hypergeometric function]]. In the special case when <math>\,\rho = 0</math>, the density can be written as:
| |
| | |
| :<math>f\left(r\right) = \frac{\left(1 - r^2\right)^{\frac{n - 4}{2}}}{\mathbf{B}\left(\frac{1}{2}, \frac{n - 2}{2}\right)},</math>
| |
| | |
| where <math>\mathbf{B}</math> is the [[beta function]], which is one way of writing the density of a Student's t-distribution, as above.
| |
| | |
| Note that{{citation needed|date=April 2012}} <math>E\left(r\right) = \rho - \frac{\rho \left(1 - \rho^2\right)}{2 \left(n - 1\right)} + \cdots </math>, therefore ''r'' is a biased estimator of <math>\,\rho</math>. An approximately unbiased estimator can be obtained by solving the equation <math>r = E\left(r\right) = \rho - \frac{\rho \left(1 - \rho^2\right)}{2 \left(n - 1\right)}</math> for <math>\,\rho</math>. However, the solution, <math>\breve{\rho} = r \left[1 + \frac{1 - r^2}{2\left(n - 1\right)}\right]</math>,{{citation needed|date=April 2012}} is suboptimal.{{citation needed|date=April 2012}} An approximately unbiased estimator,{{citation needed|date=April 2012}} with minimum variance for large values of ''n'', with a bias of order <math>\frac{1}{n - 1}</math>, can be obtained by maximizing <math>\log{f\left(r\right)}</math>, i.e. <math>\hat{\rho} = r \left[1 - \frac{1 - r^2}{2\left(n - 1\right)}\right]</math>.{{citation needed|date=April 2012}}
| |
| | |
| ===Use the Fisher transformation===
| |
| In practice, [[confidence intervals]] and [[hypothesis test]]s relating to ρ are usually carried out using the [[Fisher transformation]]:
| |
| | |
| : <math>F(r) = {1 \over 2}\ln{1 + r \over 1 - r} = \operatorname{arctanh}(r).</math>
| |
| | |
| If ''F''(''r'') is the Fisher transformation of ''r'', and ''n'' is the sample size, then ''F''(''r'') approximately follows a [[normal distribution]] with
| |
| | |
| :<math>\text{mean} = F(\rho) = \operatorname{arctanh}(\rho)</math> and standard error <math>\text{SE} = \frac{1}{\sqrt{n - 3}}.</math>
| |
| | |
| Thus, a [[standard score|z-score]] is
| |
| | |
| :<math>z = \frac{x - \text{mean}}{\text{SE}} = [F(r) - F(\rho_0)]\sqrt{n - 3}</math>
| |
| | |
| under the [[null hypothesis]] of that <math>\rho = \rho_0</math>, given the assumption that the sample pairs are [[independent and identically distributed]] and follow a [[bivariate normal distribution]]. Thus an approximate [[p-value]] can be obtained from a normal probability table. For example, if ''z'' = 2.2 is observed and a two-sided p-value is desired to test the null hypothesis that <math>\rho = 0</math>, the p-value is 2·Φ(−2.2) = 0.028, where Φ is the standard normal [[cumulative distribution function]].
| |
| | |
| To obtain a confidence interval for ρ, we first compute a confidence interval for ''F''(''<math>\rho</math>''):
| |
| | |
| :<math>100(1 - \alpha)%\text{CI}: \operatorname{arctanh}(\rho) \in [\operatorname{arctanh}(r) \pm z_{\alpha/2}SE]</math>
| |
| <br />
| |
| The inverse Fisher transformation bring the interval back to the correlation scale.
| |
| | |
| :<math>100(1 - \alpha)%\text{CI}: \rho \in [\operatorname{tanh}(\operatorname{arctanh}(r) - z_{\alpha/2}SE), \operatorname{tanh}(\operatorname{arctanh}(r) + z_{\alpha/2}SE)]</math>
| |
| <br />
| |
| For example, suppose we observe ''r'' = 0.3 with a sample size of ''n''=50, and we wish to obtain a 95% confidence interval for ρ. The transformed value is arctanh(''r'') = 0.30952, so the confidence interval on the transformed scale is 0.30952 ± 1.96/√47, or (0.023624, 0.595415). Converting back to the correlation scale yields (0.024, 0.534).
| |
| | |
| ==Pearson's correlation and least squares regression analysis==
| |
| The square of the sample correlation coefficient, typically denoted r<sup>2</sup> and called the [[coefficient of determination]], estimates the fraction of the variance in ''Y'' that is explained by ''X'' in a [[simple linear regression]]. As a starting point, the total variation in the ''Y''<sub>''i''</sub> around their average value can be decomposed as follows
| |
| | |
| :<math>
| |
| \sum_i (Y_i - \bar{Y})^2 = \sum_i (Y_i-\hat{Y}_i)^2 + \sum_i (\hat{Y}_i-\bar{Y})^2,
| |
| </math>
| |
| | |
| where the <math>\hat{Y}_i</math> are the fitted values from the regression analysis. This can be rearranged to give
| |
| | |
| :<math>
| |
| 1 = \frac{\sum_i (Y_i-\hat{Y}_i)^2}{\sum_i (Y_i - \bar{Y})^2} + \frac{\sum_i (\hat{Y}_i-\bar{Y})^2}{\sum_i (Y_i - \bar{Y})^2}.
| |
| </math>
| |
| | |
| The two summands above are the fraction of variance in ''Y'' that is explained by ''X'' (right) and that is unexplained by ''X'' (left).
| |
| | |
| Next, we apply a property of least square regression models, that the sample covariance between <math>\hat{Y}_i</math> and <math>Y_i-\hat{Y}_i</math> is zero. Thus, the sample correlation coefficient between the observed and fitted response values in the regression can be written
| |
| | |
| <math>
| |
| \begin{align}
| |
| r(Y,\hat{Y}) &= \frac{\sum_i(Y_i-\bar{Y})(\hat{Y}_i-\bar{Y})}{\sqrt{\sum_i(Y_i-\bar{Y})^2\cdot \sum_i(\hat{Y}_i-\bar{Y})^2}}\\
| |
| &= \frac{\sum_i(Y_i-\hat{Y}_i+\hat{Y}_i-\bar{Y})(\hat{Y}_i-\bar{Y})}{\sqrt{\sum_i(Y_i-\bar{Y})^2\cdot \sum_i(\hat{Y}_i-\bar{Y})^2}}\\
| |
| &= \frac{ \sum_i [(Y_i-\hat{Y}_i)(\hat{Y}_i-\bar{Y}) +(\hat{Y}_i-\bar{Y})^2 ]}{\sqrt{\sum_i(Y_i-\bar{Y})^2\cdot \sum_i(\hat{Y}_i-\bar{Y})^2}}\\
| |
| &= \frac{ \sum_i (\hat{Y}_i-\bar{Y})^2 }{\sqrt{\sum_i(Y_i-\bar{Y})^2\cdot \sum_i(\hat{Y}_i-\bar{Y})^2}}\\
| |
| | |
| &= \sqrt{\frac{\sum_i(\hat{Y}_i-\bar{Y})^2}{\sum_i(Y_i-\bar{Y})^2}}.
| |
| \end{align}
| |
| </math>
| |
| | |
| Thus
| |
| | |
| :<math>
| |
| r(Y,\hat{Y})^2 = \frac{\sum_i(\hat{Y}_i-\bar{Y})^2}{\sum_i(Y_i-\bar{Y})^2}
| |
| </math>
| |
| | |
| is the proportion of variance in ''Y'' explained by a linear function of ''X''.
| |
| | |
| ==Sensitivity to the data distribution==
| |
| | |
| ===Existence===
| |
| The population Pearson correlation coefficient is defined in terms of [[moment (mathematics)|moments]], and therefore exists for any bivariate [[probability distribution]] for which the [[statistical population|population]] [[covariance]] is defined and the [[marginal distribution|marginal]] [[population variance]]s are defined and are non-zero. Some probability distributions such as the [[Cauchy distribution]] have undefined variance and hence ρ is not defined if ''X'' or ''Y'' follows such a distribution. In some practical applications, such as those involving data suspected to follow a [[heavy-tailed distribution]], this is an important consideration. However, the existence of the correlation coefficient is usually not a concern; for instance, if the range of the distribution is bounded, ρ is always defined.
| |
| | |
| ===Large sample properties===
| |
| In the case of the bivariate [[normal distribution]] the population Pearson correlation coefficient characterizes the joint distribution as long as the marginal means and variances are known. For most other bivariate distributions this is not true. Nevertheless, the correlation coefficient is highly informative about the degree of linear dependence between two random quantities regardless of whether their joint distribution is normal.<ref name="thirteenways"/>
| |
| The sample correlation coefficient is the [[maximum likelihood estimate]] of the population correlation coefficient for bivariate normal data, and is [[asymptotic distribution|asymptotically]] [[bias of an estimator|unbiased]] and [[efficiency (statistics)|efficient]], which roughly means that it is impossible to construct a more accurate estimate than the sample correlation coefficient if the data are normal and the sample size is moderate or large. For non-normal populations, the sample correlation coefficient remains approximately unbiased, but may not be efficient. The sample correlation coefficient is a [[consistent estimator]] of the population correlation coefficient as long as the sample means, variances, and covariance are consistent (which is guaranteed when the [[law of large numbers]] can be applied).
| |
| | |
| ===Robustness===
| |
| Like many commonly used statistics, the sample statistic ''r'' is not [[robust statistics|robust]],<ref name="wilcox">{{Cite book| title=Introduction to robust estimation and hypothesis testing | last = Wilcox | first = Rand R. | publisher= Academic Press | year=2005}}</ref> so its value can be misleading if [[outlier]]s are present.<ref>{{Cite journal| title= Robust Estimation and Outlier Detection with Correlation Coefficients | last= Devlin | first = Susan J | coauthors = Gnanadesikan, R; Kettenring J.R. | journal= Biometrika | volume= 62 | issue= 3 |year=1975 | pages=531–545 | doi= 10.1093/biomet/62.3.531 | jstor=2335508}}</ref><ref>{{Cite book| title=Robust Statistics | last = Huber | first = Peter. J.| publisher= Wiley | year=2004}}{{Page needed|date=September 2010}}</ref> Specifically, the PMCC is neither distributionally robust,{{Citation needed|date=November 2009}} nor outlier resistant<ref name="wilcox"/> (see [[Robust statistics#Definition]]). Inspection of the [[scatterplot]] between ''X'' and ''Y'' will typically reveal a situation where lack of robustness might be an issue, and in such cases it may be advisable to use a robust measure of association. Note however that while most robust estimators of association measure [[statistical dependence]] in some way, they are generally not interpretable on the same scale as the Pearson correlation coefficient.
| |
| | |
| Statistical inference for Pearson's correlation coefficient is sensitive to the data distribution. Exact tests, and asymptotic tests based on the [[Fisher transformation]] can be applied if the data are approximately normally distributed, but may be misleading otherwise. In some situations, the [[bootstrapping (statistics)|bootstrap]] can be applied to construct confidence intervals, and [[resampling (statistics)|permutation tests]] can be applied to carry out hypothesis tests. These [[non-parametric statistics|non-parametric]] approaches may give more meaningful results in some situations where bivariate normality does not hold. However the standard versions of these approaches rely on [[exchangeable random variables|exchangeability]] of the data, meaning that there is no ordering or grouping of the data pairs being analyzed that might affect the behavior of the correlation estimate.
| |
| | |
| A stratified analysis is one way to either accommodate a lack of bivariate normality, or to isolate the correlation resulting from one factor while controlling for another. If ''W'' represents cluster membership or another factor that it is desirable to control, we can stratify the data based on the value of ''W'', then calculate a correlation coefficient within each stratum. The stratum-level estimates can then be combined to estimate the overall correlation while controlling for ''W''.<ref>Katz., Mitchell H. (2006) ''Multivariable Analysis - A Practical Guide for Clinicians''. 2nd Edition. Cambridge University Press. ISBN 978-0-521-54985-1. ISBN 0-521-54985-X {{DOI|10.2277/052154985X}}</ref>
| |
| | |
| ==Calculating a weighted correlation==
| |
| Suppose observations to be correlated have differing degrees of importance that can be expressed with a weight vector ''w''. To calculate the correlation between vectors ''x'' and ''y'' with the weight vector ''w'' (all of length ''n''),<ref>http://sci.tech-archive.net/Archive/sci.stat.math/2006-02/msg00171.html</ref><ref>[http://www.mathworks.com/matlabcentral/fileexchange/20846 A MATLAB Toolbox for computing Weighted Correlation Coefficients]</ref>
| |
| | |
| * Weighted mean:
| |
| | |
| :: <math>\operatorname{m}(x; w) = {\sum_i w_i x_i \over \sum_i w_i}.</math>
| |
| | |
| * Weighted covariance
| |
| :: <math>\operatorname{cov}(x,y;w) = {\sum_i w_i (x_i - \operatorname{m}(x; w)) (y_i - \operatorname{m}(y; w)) \over \sum_i w_i }.</math>
| |
| | |
| * Weighted correlation
| |
| :: <math>\operatorname{corr}(x,y;w) = {\operatorname{cov}(x,y;w) \over \sqrt{\operatorname{cov}(x,x;w) \operatorname{cov}(y,y;w)}}.</math>
| |
| | |
| ==Removing correlation==
| |
| It is always possible to remove the correlation between random variables with a linear transformation, even if the relationship between the variables is nonlinear. A presentation of this result for population distributions is given by Cox & Hinkley.<ref>Cox, D.R., Hinkley, D.V. (1974) ''Theoretical Statistics'', Chapman & Hall (Appendix 3) ISBN 0-412-12420-3</ref>
| |
| | |
| A corresponding result exists for sample correlations, in which the sample correlation is reduced to zero. Suppose a vector of ''n'' random variables is sampled ''m'' times. Let ''X'' be a matrix where <math>X_{i,j}</math> is the ''j''th variable of sample ''i''. Let <math>Z_{m,m}</math> be an ''m'' by ''m'' square matrix with every element 1. Then ''D'' is the data transformed so every random variable has zero mean, and ''T'' is the data transformed so all variables have zero mean and zero correlation with all other variables - the sample covariance matrix of ''T'' will be the identity matrix. This has to be further divided by the standard deviation to get unit variance. The transformed variables will be uncorrelated, even though they may not be [[Statistical independence|independent]].
| |
| | |
| :<math>D = X -\frac{1}{m} Z_{m,m} X</math>
| |
| | |
| <!-- extra blank line between two lines of "displayed" [[TeX]], for legibility -->
| |
| | |
| :<math>T = D (D^T D)^{-\frac{1}{2}},</math>
| |
| | |
| where an exponent of -1/2 represents the [[matrix square root]] of the [[matrix inverse|inverse]] of a matrix. The covariance matrix of ''T'' will be the identity matrix. If a new data sample ''x'' is a row vector of ''n'' elements, then the same transform can be applied to ''x'' to get the transformed vectors ''d'' and ''t'':
| |
| | |
| :<math>d = x - \frac{1}{m} Z_{1,m} X,</math>
| |
| | |
| <!-- extra blank line between two lines of "displayed" [[TeX]], for legibility -->
| |
| | |
| :<math>t = d (D^T D)^{-\frac{1}{2}}.</math>
| |
| This decorrelation is related to [[principal components analysis]] for multivariate data.
| |
| | |
| ==Reflective correlation==
| |
| The reflective correlation is a variant of Pearson's correlation in which the data are not centered around their mean values.{{Citation needed|date=January 2011}} The population reflective correlation is
| |
| | |
| :<math>
| |
| \text{Corr}_r(X,Y) = \frac{E[XY]}{\sqrt{EX^2\cdot EY^2}}.
| |
| </math>
| |
| | |
| The reflective correlation is symmetric, but it is not invariant under translation:
| |
| | |
| :<math>
| |
| \text{Corr}_r(X, Y) = \text{Corr}_r(Y, X) = \text{Corr}_r(X, bY) \neq \text{Corr}_r(X, a + b Y), \quad a \neq 0, b > 0.
| |
| </math>
| |
| | |
| The sample reflective correlation is
| |
| | |
| :<math>
| |
| rr_{xy} = \frac{\sum x_i y_i}{\sqrt{(\sum x_i^2)(\sum y_i^2)}}.
| |
| </math>
| |
| | |
| The weighted version of the sample reflective correlation is
| |
| | |
| :<math>
| |
| rr_{xy, w} = \frac{\sum w_i x_i y_i}{\sqrt{(\sum w_i x_i^2)(\sum w_i y_i^2)}}.
| |
| </math>
| |
| | |
| ==Scaled correlation==
| |
| {{Main|Scaled correlation}}
| |
| | |
| Scaled correlation is a variant of Pearson's correlation in which the range of the data is restricted intentionally and in a controlled manner to reveal correlations between fast components in time series.<ref name = "Nikolicetal">Nikolić D, Muresan RC, Feng W, Singer W (2012) Scaled correlation analysis: a better way to compute a cross-correlogram. ''European Journal of Neuroscience'', pp. 1–21, {{doi|10.1111/j.1460-9568.2011.07987.x}} http://www.danko-nikolic.com/wp-content/uploads/2012/03/Scaled-correlation-analysis.pdf</ref> Scaled correlation is defined as average correlation across short segments of data.
| |
| | |
| Let <math>K</math> be the number of segments that can fit into the total length of the signal <math>T</math> for a given scale <math>s</math>:
| |
| | |
| :<math>K = \operatorname{round}\left(\frac{T}{s}\right).</math>
| |
| | |
| The scaled correlation across the entire signals <math>\bar{r}_s</math> is then computed as
| |
| | |
| :<math>\bar{r}_s = \frac{1}{K} \sum\limits_{k=1}^K r_k,</math>
| |
| | |
| where <math>r_k</math> is Pearson's coefficient of correlation for segment <math>k</math>.
| |
| | |
| By choosing the parameter <math>s</math>, the range of values is reduced and the correlations on long time scale are filtered out, only the correlations on short time scales being revealed. Thus, the contributions of slow components are removed and those of fast components are retained.
| |
| | |
| ==Heavy noise conditions==
| |
| | |
| Under heavy noise conditions, extracting the correlation coefficient between two sets of stochastic variables is nontrivial, in particular where Canonical Correlation Analysis reports on degraded correlation values due to the heavy noise contributions. A generalization of the approach is given elsewhere.<ref>{{Cite journal
| |
| | first = N. | last = Moriya
| |
| | title = Noise-Related Multivariate Optimal Joint-Analysis in Longitudinal Stochastic Processes in Progress in applied mathematical modeling
| |
| | booktitle = [[Progress in Applied Mathematical Modeling]]
| |
| | editor = Fengshan Yang
| |
| | publisher = Nova Science Publishers, Inc.
| |
| | year = 2008
| |
| | pages = 223–260
| |
| | ISBN = 978-1-60021-976-4
| |
| }}</ref>
| |
| | |
| ==See also==
| |
| {{Portal|Statistics}}
| |
| * [[Association (statistics)]]
| |
| * [[Correlation and dependence]]
| |
| * [[Disattenuation]]
| |
| * [[Maximal information coefficient]]
| |
| * [[Normally distributed and uncorrelated does not imply independent]]
| |
| * [[Quadrant Count Ratio]]
| |
| * [[Spearman's rank correlation coefficient]]
| |
| | |
| ==References==
| |
| {{Reflist|30em}}
| |
| | |
| ==External links==
| |
| {{Wikiversity|Linear correlation}}
| |
| * [http://nagysandor.eu/AsimovTeka/correlation_en/index.html Interactive Flash simulation on the correlation of two normally distributed variables.] Author: Juha Puranen.
| |
| * [http://www.hackmath.net/en/calculator/linear-regression Correlation coefficient calculator - linear regression]
| |
| * {{PDFlink | [http://frank.mtsu.edu/~dkfuller/tables/correlationtable.pdf Critical values for Pearson's correlation coefficient (large table)] }}
| |
| | |
| {{Statistics|descriptive}}
| |
| | |
| {{Use dmy dates|date=September 2010}}
| |
| | |
| {{DEFAULTSORT:Pearson Product-Moment Correlation Coefficient}}
| |
| [[Category:Covariance and correlation]]
| |
| [[Category:Parametric statistics]]
| |
| [[Category:Statistical ratios]]
| |
| [[Category:Statistical dependence]]
| |
| | |
| [[ru:Корреляция#Линейный коэффициент корреляции]]
| |