|
|
| Line 1: |
Line 1: |
| In [[statistics]], a [[statistic]] is ''sufficient'' with respect to a [[statistical model]] and its associated unknown [[parameter]] if "no other statistic that can be calculated from the same [[sample (statistics)|sample]] provides any additional information as to the value of the parameter".<ref name=Fisher1922>{{cite journal | | Usually it's the other way around: the engine dies well before the frame is worn out. In this analysis, a soil sample is heated, and the point at which the soil breaks down, melts, or boils is recorded. This technology is typically an infrared testing process that involves non-destructive, non-invasive inspection procedures by making use of thermography cameras. Even with a new filter and oil, you're not getting complete engine protection. By Ms.Sunita : A how to tutorial about Market Forecast to 2020, Solar Thermal Power Market, Research, Business with step by step guide from Ms.Sunita. We get the facts from over 22,000 news and information sources, a huge database of key industry participants and draw on our relationships with more than 900 market research companies across the world. Thermal requirements may be significantly affected due to small amounts of water being evaporated or condensed. This technique was used by Somiya and Hirano in as early as 1930 for determining the volatile yield of coals, the prime fossil fuel of that time. As the gas engine heat pump heating in winter when the introduction of natural gas engine cylinder, and exhaust heat, thus, in the heating mode, gas engine heat pump and conventional electric-driven heat pump with a larger difference. Prior to first introduction pf B100m it is suggested to clear tanks and gasoline systems. But some people think that the cost of heat transfer is too expensive. People make use of whatever will function for them. To maintain overall thermal balance, a control volume should be considered and defined. Frank Irby Plumbing Company is located in Macon, Georgia and specializes in a wide range of plumbing services, which include water heater and sprinkler system installation, sewer and drain clearing and plumbing repair work. Of course, this beast of a processor is designed for running heavy applications and using it for mundane computing jobs would be more than just overkill. Cover pots to conserve heat and adjust the level of heat on the burner. Furthermore, mini 1000 battery thermogravimetric analysis could also examine a variety of additives on thermal stability. In the end, the oil settles back down into the oil pan where it gets sucked back up by the oil pump, and the process begins anew. The temperature limits of all materials should be considered and should be adequate. After publicity to heat over a period of time, fuels can resist breakdown depending on their thermal stability. Crabby cold oil means bad lubrication. When you are baking something for a long time, you could turn off the oven a few minutes before the time is up, to take advantage of the accumulated heat. It is important to attend these meetings not only to keep current on changes but also to establish contacts with other university researchers with experiments on the same satellite. When you install a residential demand controller you can receive a $300 rebate.<br><br>This will usually proceed in a number of steps that will likely culminate near the melting point of the polymer itself. A individual might believe of using combinations of B20 or less to combat this instabilities. Results on the surface interpolation and linear fitting way to get other non-physical quantity under test conditions point to obtain the data needed to calculate the engine heat. I believe that crop circles (or formations) are among these 'signs' and, in fact, the actual creation and nature of them have been denied from the public for years! Pure sci-fi or fantasy, right? Contaminated drinking water is an enormous problem overseas, with many families having zero access to a pure source. I told him, nope - original engine. Keep the door closed as much as possible. Burning is not well with oxidized biodiesel. So you won't have to worry about this if you are an typical client. When your dryer has an automatic sensor to detect when the clothes are dry, use it. Many electrical energy companies offer this service and have teams of specialized professionals who can detect where and how you are losing or wasting energy in your home. Act accordingly. Now, at 130,000 miles, I continue to drive my car without the symptoms of an older engine: - No loss of power due to lower compression (due to worn pistons and piston rings). This is an opportunity for thousands, hundreds of thousands, of new jobs and new construction, leading to an export commodity that is in demand everywhere, solar Photo-Voltaic panels. Sometimes the actual physical properties of materials may differ from the values given in the handbook. The newest plants incorporate technology to streamline the process and save energy and money. the snow will drift. Thermal images can be a first line of visualization of underlying practical abnormalities. Before initial introduction pf B100m it is recommended to clean tanks and gasoline systems. Snow will accumulate aside / behind parapets, and drift up in front of and behind obstructions, including steps in the roof surface. Windows Windows are also important in conserving heating and air conditioning. Likely because no one of that time would (or could) understand the science of [http://www.Adobe.com/cfusion/search/index.cfm?term=&Replication&loc=en_us&siteSection=home Replication] or Alchemy or whatever it was.<br><br>If you have any inquiries regarding where and how you can make use of [http://keficalaemcasa.com//index.php?a=profile&u=mig54 Thorn Thermal Analysis], you could call us at the web site. |
| | last=Fisher | first=R.A. |authorlink=Ronald Fisher
| |
| | journal= Philosophical Transactions of the Royal Society A
| |
| | title=On the mathematical foundations of theoretical statistics
| |
| | volume=222 | year=1922 | pages=309–368
| |
| | url=http://digital.library.adelaide.edu.au/dspace/handle/2440/15172
| |
| | jstor=91208 | jfm = 48.1280.02 |doi=10.1098/rsta.1922.0009
| |
| }}</ref> In particular, a statistic is '''sufficient''' for a [[Parametric family|family]] of [[probability distribution]]s if the sample from which it is calculated gives no additional information than does the statistic, as to which of those probability distributions is that of the population from which the sample was taken.
| |
| | |
| Roughly, given a set <math> \mathbf{X}</math> of [[independent identically distributed]] data conditioned on an unknown parameter <math>\theta</math>, a sufficient statistic is a function <math>T(\mathbf{X})</math> whose value contains all the information needed to compute any estimate of the parameter (e.g. a [[maximum likelihood]] estimate). Due to the factorization theorem (see below), for a sufficient statistic <math>T(\mathbf{X})</math>, the [[joint distribution]] can be written as <math>p(\mathbf{X}) = h(\mathbf{X}) \, g(\theta, T(\mathbf{X}))\,</math>. From this factorization, it can easily be seen that the maximum likelihood estimate of <math>\theta</math> will interact with <math>\mathbf{X}</math> only through <math>T(\mathbf{X})</math>. Typically, the sufficient statistic is a simple function of the data, e.g. the sum of all the data points.
| |
| | |
| More generally, the "unknown parameter" may represent a [[Euclidean vector|vector]] of unknown quantities or may represent everything about the model that is unknown or not fully specified. In such a case, the sufficient statistic may be a set of functions, called a ''jointly sufficient statistic''. Typically, there are as many functions as there are parameters. For example, for a [[Gaussian distribution]] with unknown [[mean]] and [[variance]], the jointly sufficient statistic, from which maximum likelihood estimates of both parameters can be estimated, consists of two functions, the sum of all data points and the sum of all squared data points (or equivalently, the [[sample mean]] and [[sample variance]]).
| |
| | |
| The concept, due to [[Ronald Fisher]], is equivalent to the statement that, [[Conditional probability distribution|conditional]] on the value of a sufficient statistic for a parameter, the [[joint probability distribution]] of the data does not depend on that parameter. Both the statistic and the underlying parameter can be vectors.
| |
| | |
| A related concept is that of '''linear sufficiency''', which is weaker than ''sufficiency'' but can be applied in some cases where there is no sufficient statistic, although it is restricted to linear estimators.<ref>Dodge, Y. (2003) — entry for linear sufficiency</ref> The [[Kolmogorov structure function]] deals with individual finite data, the related notion there is the algorithmic sufficient statistic.
| |
| | |
| The concept of sufficiency has fallen out of favor in [[descriptive statistics]] because of the strong dependence on an assumption of the distributional form (see [[#Exponential family|Pitman–Koopman–Darmois theorem]] below), but remains very important in theoretical work.<ref name=Stigler1973>{{cite journal
| |
| | last = Stigler
| |
| | first = Stephen
| |
| | authorlink = Stephen Stigler
| |
| | date =
| |
| | year = 1973
| |
| | month = December
| |
| | title = Studies in the History of Probability and Statistics. XXXII: Laplace, Fisher and the Discovery of the Concept of Sufficiency
| |
| | journal = Biometrika
| |
| | volume = 60
| |
| | issue = 3
| |
| | pages = 439–445
| |
| | doi = 10.1093/biomet/60.3.439
| |
| | mr = 0326872 | jstor = 2334992
| |
| }}</ref>
| |
| | |
| ==Mathematical definition==
| |
| | |
| A statistic ''T''(''X'') is '''sufficient for underlying parameter ''θ''''' precisely if the conditional [[probability distribution]] of the data ''X'', given the statistic ''T''(''X''), does not depend on the parameter ''θ'',<ref name="CasellaBerger">{{cite book | last = Casella | first = George | coauthors = Berger, Roger L. | title = Statistical Inference, 2nd ed | publisher=Duxbury Press | year = 2002}}</ref> i.e.
| |
| | |
| :<math>\Pr(X=x|T(X)=t,\theta) = \Pr(X=x|T(X)=t), \,</math>
| |
| or in shorthand
| |
| :<math>\Pr(x|t,\theta) = \Pr(x|t).\,</math>
| |
| <!--
| |
| [However, it seems that a proper definition should strictly speaking also address what happens in the case that for some values of the parameter, the event T(X)=t has probability zero.]-->
| |
| | |
| Instead of this last expression, the definition still holds if one uses either of the equivalent expressions:
| |
| :<math>\Pr(\theta|t,x) = \Pr(\theta|t),\,</math> or
| |
| :<math>\Pr(\theta, x|t) = \Pr(\theta|t) \Pr(x|t),\,</math>
| |
| which indicate, respectively, that the conditional probability of the parameter ''θ'', given the sufficient statistic ''t'', does not depend on the data ''x''; and that the conditional probability of the parameter ''θ'' given the sufficient statistic ''t'' and the conditional probability of the data ''x'' given the sufficient statistic ''t'' are [[statistically independent]].
| |
| | |
| ===Example===
| |
| As an example, the sample mean is sufficient for the mean (μ) of a [[normal distribution]] with known variance. Once the sample mean is known, no further information about μ can be obtained from the sample itself. On the other hand, the [[median]] is not sufficient for the mean: even if the median of the sample is known, knowing the sample itself would provide further information about the population mean. For example, if the observations that are less than the median are only slightly less, but observations exceeding the median exceed it by a large amount, then this would have a bearing on one's inference about the population mean.
| |
| | |
| ==Fisher–Neyman factorization theorem==
| |
| | |
| ''[[Ronald Fisher|Fisher's]] factorization theorem'' or ''factorization criterion'' provides a convenient '''characterization''' of a sufficient statistic. If the [[probability density function]] is ƒ<sub>''θ''</sub>(''x''), then ''T'' is sufficient for ''θ'' [[if and only if]] nonnegative functions ''g'' and ''h'' can be found such that
| |
| | |
| :<math> f_\theta(x)=h(x) \, g_\theta(T(x)), \,\!</math>
| |
| | |
| i.e. the density ƒ can be factored into a product such that one factor, ''h'', does not depend on ''θ'' and the other factor, which does depend on ''θ'', depends on ''x'' only through ''T''(''x'').
| |
| | |
| ===Likelihood principle interpretation===
| |
| An implication of the theorem is that when using likelihood-based inference, two sets of data yielding the same value for the sufficient statistic ''T''(''X'') will always yield the same inferences about θ. By the factorization criterion, the likelihood's dependence on θ is only in conjunction with ''T''(''X''). As this is the same in both cases, the dependence on θ will be the same as well, leading to identical inferences.
| |
| | |
| ===Proof===
| |
| Due to Hogg and Craig.<ref name="HoggCraig">{{cite book | last = Hogg | first = Robert V. | coauthors = Craig, Allen T. | title = Introduction to Mathematical Statistics | publisher=Prentice Hall | year = 1995 | ISBN=978-0-02-355722-4}}</ref> Let <math>X_1, X_2, \ldots, X_n</math>, denote a random sample from a distribution having the [[Probability density function|pdf]] ''f''(''x'', ''θ'') for ''ι'' < ''θ'' < ''δ''. Let ''Y''<sub>1</sub> = ''u''<sub>1</sub>(''X''<sub>1</sub>, ''X''<sub>2</sub>, ..., ''X''<sub>''n''</sub>) be a statistic whose pdf is ''g''<sub>1</sub>(''y''<sub>1</sub>; ''θ''). Then ''Y''<sub>1</sub> = ''u''<sub>1</sub>(''X''<sub>1</sub>, ''X''<sub>2</sub>, ..., ''X''<sub>''n''</sub>) is a sufficient statistic for ''θ'' if and only if, for some function ''H'',
| |
| | |
| :<math> \prod_{i=1}^n f(x_i; \theta) = g_1 \left[u_1 (x_1, x_2, \dots, x_n); \theta \right] H(x_1, x_2, \dots, x_n). \,</math>
| |
| | |
| First, suppose that
| |
| :<math> \prod_{i=1}^n f(x_i; \theta) = g_1 \left[u_1 (x_1, x_2, \dots, x_n); \theta \right] H(x_1, x_2, \dots, x_n). \,</math>
| |
| | |
| We shall make the transformation ''y''<sub>''i''</sub> = ''u''<sub>i</sub>(''x''<sub>1</sub>, ''x''<sub>2</sub>, ..., ''x''<sub>''n''</sub>), for ''i'' = 1, ..., ''n'', having inverse functions ''x''<sub>''i''</sub> = ''w''<sub>''i''</sub>(''y''<sub>1</sub>, ''y''<sub>2</sub>, ..., ''y''<sub>''n''</sub>), for ''i'' = 1, ..., ''n'', and [[Jacobian matrix and determinant|Jacobian]] <math> J = \left[w_i/y_j \right] </math>. Thus,
| |
| | |
| :<math>
| |
| \prod_{i=1}^n f \left[ w_i(y_1, y_2, \dots, y_n); \theta \right] =
| |
| |J| g_1 (y_1; \theta) H \left[ w_1(y_1, y_2, \dots, y_n), \dots, w_n(y_1, y_2, \dots, y_n) \right].
| |
| </math>
| |
| | |
| The left-hand member is the joint pdf ''g''(''y''<sub>1</sub>, ''y''<sub>2</sub>, ..., ''y''<sub>''n''</sub>; θ) of ''Y''<sub>1</sub> = ''u''<sub>1</sub>(''X''<sub>1</sub>, ..., ''X''<sub>''n''</sub>), ..., ''Y''<sub>''n''</sub> = ''u''<sub>''n''</sub>(''X''<sub>1</sub>, ..., ''X''<sub>''n''</sub>). In the right-hand member, <math>g_1(y_1;\theta)</math> is the pdf of <math>Y_1</math>, so that <math>H[ w_1, \dots , w_n] |J|</math> is the quotient of <math>g(y_1,\dots,y_n;\theta)</math> and <math>g_1(y_1;\theta)</math>; that is, it is the conditional pdf <math>h(y_2, \dots, y_n | y_1; \theta)</math> of <math>Y_2,\dots,Y_n</math> given <math>Y_1=y_1</math>.
| |
| | |
| But <math>H(x_1,x_2,\dots,x_n)</math>, and thus <math>H\left[w_1(y_1,\dots,y_n), \dots, w_n(y_1, \dots, y_n))\right]</math>, was given not to depend upon <math>\theta</math>. Since <math>\theta</math> was not introduced in the transformation and accordingly not in the Jacobian <math>J</math>, it follows that <math>h(y_2, \dots, y_n | y_1; \theta)</math> does not depend upon <math>\theta</math> and that <math>Y_1</math> is a sufficient statistics for <math>\theta</math>.
| |
| | |
| The converse is proven by taking:
| |
| | |
| :<math>g(y_1,\dots,y_n;\theta)=g_1(y_1; \theta) h(y_2, \dots, y_n | y_1),\,</math>
| |
| | |
| where <math>h(y_2, \dots, y_n | y_1)</math> does not depend upon <math>\theta</math> because <math>Y_2 ... Y_n</math> depend only upon <math>X_1 ... X_n</math>, which are independent on <math>\Theta</math> when conditioned by <math>Y_1</math>, a sufficient statistics by hypothesis. Now divide both members by the absolute value of the non-vanishing Jacobian <math>J</math>, and replace <math>y_1, \dots, y_n</math> by the functions <math>u_1(x_1, \dots, x_n), \dots, u_n(x_1,\dots, x_n)</math> in <math>x_1,\dots, x_n</math>. This yields
| |
| | |
| :<math>\frac{g\left[ u_1(x_1, \dots, x_n), \dots, u_n(x_1, \dots, x_n); \theta \right]}{|J*|}=g_1\left[u_1(x_1,\dots,x_n); \theta\right] \frac{h(u_2, \dots, u_n | u_1)}{|J*|}</math>
| |
| | |
| where <math>J*</math> is the Jacobian with <math>y_1,\dots,y_n</math> replaced by their value in terms <math>x_1, \dots, x_n</math>. The left-hand member is necessarily the joint pdf <math>f(x_1;\theta)\cdots f(x_n;\theta)</math> of <math>X_1,\dots,X_n</math>. Since <math>h(y_2,\dots,y_n|y_1)</math>, and thus <math>h(u_2,\dots,u_n|u_1)</math>, does not depend upon <math>\theta</math>, then
| |
| | |
| :<math>H(x_1,\dots,x_2)=\frac{h(u_2,\dots,u_n|u_1)}{|J*|}</math>
| |
| | |
| is a function that does not depend upon <math>\theta</math>.
| |
| | |
| ===Another proof===
| |
| A simpler more illustrative proof is as follows, although it applies only in the discrete case.
| |
| | |
| We use the shorthand notation to denote the joint probability of <math>(X, T(X))</math> by <math>f_\theta(x,t)</math>. Since <math>T</math> is a function of <math>X</math>, we have <math>f_\theta(x,t) = f_\theta(x)</math> (only when <math> t = T(x) </math> and zero otherwise) and thus:
| |
| | |
| :<math>f_\theta(x) = f_\theta(x,t) = f_{\theta | t}(x) f_\theta(t) </math>
| |
| | |
| with the last equality being true by the definition of [[conditional probability distribution]]s. Thus <math>f_\theta(x)=a(x) b_\theta(t)</math> with <math>a(x) = f_{\theta | t}(x)</math> and <math>b_\theta(t) = f_\theta(t)</math>.
| |
| | |
| Reciprocally, if <math>f_\theta(x)=a(x) b_\theta(t)</math>, we have
| |
| | |
| :<math>
| |
| \begin{align}
| |
| f_\theta(t) & = \sum _{x : T(x) = t} f_\theta(x, t) \\
| |
| & = \sum _{x : T(x) = t} f_\theta(x) \\
| |
| & = \sum _{x : T(x) = t} a(x) b_\theta(t) \\
| |
| & = \left( \sum _{x : T(x) = t} a(x) \right) b_\theta(t).
| |
| \end{align}</math>
| |
| | |
| With the first equality by the [[Probability density function#Probability functions associated with multiple variables|definition of pdf for multiple variables]], the second by the remark above, the third by hypothesis, and the fourth because the summation is not over <math>t</math>.
| |
| | |
| Thus, the conditional probability distribution is:
| |
| :<math>
| |
| \begin{align}
| |
| f_{\theta|t}(x)
| |
| & = \frac{f_\theta(x, t)}{f_\theta(t)} \\
| |
| & = \frac{f_\theta(x)}{f_\theta(t)} \\
| |
| & = \frac{a(x) b_\theta(t)}{\left( \sum _{x : T(x) = t} a(x) \right) b_\theta(t)} \\
| |
| & = \frac{a(x)}{\sum _{x : T(x) = t} a(x)}.
| |
| \end{align}</math>
| |
| | |
| With the first equality by definition of conditional probability density, the second by the remark above, the third by the equality proven above, and the fourth by simplification. This expression does not depend on <math>\theta</math> and thus <math>T</math> is a sufficient statistic.<ref>{{cite web | url=http://cnx.org/content/m11480/1.6/ | title=The Fisher–Neyman Factorization Theorem}}. Webpage at Connexions (cnx.org)</ref>
| |
| | |
| ==Minimal sufficiency==
| |
| A sufficient statistic is '''minimal sufficient''' if it can be represented as a function of any other sufficient statistic. In other words, ''S''(''X'') is '''minimal sufficient''' if and only if<ref>Dodge (2003) — entry for minimal sufficient statistics</ref>
| |
| #''S''(''X'') is sufficient, and
| |
| #if ''T''(''X'') is sufficient, then there exists a function ''f'' such that ''S''(''X'') = ''f''(''T''(''X'')).
| |
| | |
| Intuitively, a minimal sufficient statistic ''most efficiently'' captures all possible information about the parameter ''θ''.
| |
| | |
| A useful characterization of minimal sufficiency is that when the density ''f''<sub>θ</sub> exists, ''S''(''X'') is '''minimal sufficient''' if and only if
| |
| :<math>\frac{f_\theta(x)}{f_\theta(y)}</math> is independent of ''θ'' :<math>\Longleftrightarrow</math> ''S''(''x'') = ''S''(''y'')
| |
| | |
| This follows as a direct consequence from [[#Fisher–Neyman factorization theorem|Fisher's factorization theorem]] stated above.
| |
| | |
| A case in which there is no minimal sufficient statistic was shown by Bahadur, 1954.<ref>Lehmann and Casella (1998), ''Theory of Point Estimation'', 2nd Edition, Springer, p 37</ref> However, under mild conditions, a minimal sufficient statistic does always exist. In particular, in Euclidean space, these conditions always hold if the random variables (associated with <math>P_\theta</math> ) are all discrete or are all continuous.
| |
| | |
| If there exists a minimal sufficient statistic, and this is usually the case, then every [[Completeness (statistics)|complete]] sufficient statistic is necessarily minimal sufficient<ref>Lehmann and Casella (1998), ''Theory of Point Estimation'', 2nd Edition, Springer, page 42</ref>(note that this statement does not exclude the option of a pathological case in which a complete sufficient exists while there is no minimal sufficient statistic). While it is hard to find cases in which a minimal sufficient statistic does not exist, it is not so hard to find cases in which there is no complete statistic.
| |
| | |
| The collection of likelihood ratios <math>\left\{\frac{L(\theta_1|X)}{L(\theta_2|X)}\right\}</math> is a minimal sufficient statistic if <math>P(X|\theta)</math> is discrete or has a density function.
| |
| | |
| ==Examples==
| |
| ===Bernoulli distribution===
| |
| | |
| If ''X''<sub>1</sub>, ...., ''X''<sub>''n''</sub> are independent [[Bernoulli trial|Bernoulli-distributed]] random variables with expected value ''p'', then the sum ''T''(''X'') = ''X''<sub>1</sub> + ... + ''X''<sub>''n''</sub> is a sufficient statistic for ''p'' (here 'success' corresponds to ''X''<sub>''i''</sub> = 1 and 'failure' to ''X''<sub>''i''</sub> = 0; so ''T'' is the total number of successes)
| |
| | |
| This is seen by considering the joint probability distribution:
| |
| | |
| :<math> \Pr\{X=x\}=\Pr\{X_1=x_1,X_2=x_2,\ldots,X_n=x_n\}.</math>
| |
| | |
| Because the observations are independent, this can be written as
| |
| | |
| :<math>
| |
| p^{x_1}(1-p)^{1-x_1} p^{x_2}(1-p)^{1-x_2}\cdots p^{x_n}(1-p)^{1-x_n} \,\!</math>
| |
| | |
| and, collecting powers of ''p'' and 1 − ''p'', gives
| |
| | |
| :<math>
| |
| p^{\sum x_i}(1-p)^{n-\sum x_i}=p^{T(x)}(1-p)^{n-T(x)} \,\!
| |
| </math>
| |
| | |
| which satisfies the factorization criterion, with ''h''(''x'') = 1 being just a constant.
| |
| | |
| Note the crucial feature: the unknown parameter ''p'' interacts with the data ''x'' only via the statistic ''T''(''x'') = Σ ''x''<sub>''i''</sub>.
| |
| | |
| ===Uniform distribution===
| |
| {{see also|German tank problem}}
| |
| If ''X''<sub>1</sub>, ...., ''X''<sub>''n''</sub> are independent and [[uniform distribution (continuous)|uniformly distributed]] on the interval [0,''θ''], then ''T''(''X'') = max(''X''<sub>1</sub>, ..., ''X''<sub>''n''</sub>) is sufficient for θ — the [[sample maximum]] is a sufficient statistic for the population maximum.
| |
| | |
| To see this, consider the joint [[probability density function]] of ''X''=(''X''<sub>1</sub>,...,''X''<sub>''n''</sub>). Because the observations are independent, the pdf can be written as a product of individual densities
| |
| | |
| :<math>\begin{align}
| |
| f_X(x_1,\ldots,x_n)
| |
| &= \frac{1}{\theta}\mathbf{1}_{\{0\leq x_1\leq\theta\}} \cdots
| |
| \frac{1}{\theta}\mathbf{1}_{\{0\leq x_n\leq\theta\}} \\
| |
| &= \frac{1}{\theta^n}\mathbf{1}_{\{0\leq\min\{x_i\}\}}\mathbf{1}_{\{\max\{x_i\}\leq\theta\}}
| |
| \end{align}</math>
| |
| | |
| where '''1'''<sub>{''...''}</sub> is the [[indicator function]]. Thus the density takes form required by the Fisher–Neyman factorization theorem, where ''h''(''x'') = '''1'''<sub>{min{''x<sub>i</sub>''}≥0}</sub>, and the rest of the expression is a function of only ''θ'' and ''T''(''x'') = max{''x<sub>i</sub>''}.
| |
| | |
| In fact, the [[minimum-variance unbiased estimator]] (MVUE) for ''θ'' is
| |
| | |
| :<math> \frac{n+1}{n}T(X). </math>
| |
| | |
| This is the sample maximum, scaled to correct for the [[bias of an estimator|bias]], and is MVUE by the [[Lehmann–Scheffé theorem]]. Unscaled sample maximum ''T''(''X'') is the [[maximum likelihood estimator]] for ''θ''.
| |
| | |
| ===Uniform distribution (with two parameters)===
| |
| | |
| If <math>X_1,...,X_n\,</math> are independent and [[Uniform distribution (continuous)|uniformly distributed]] on the interval <math>[\alpha, \beta]\,</math> (where <math>\alpha\,</math> and <math>\beta\,</math> are unknown parameters), then <math>T(X_1^n)=\left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right)\,</math> is a two-dimensional sufficient statistic for <math>(\alpha\, , \, \beta)</math>.
| |
| | |
| To see this, consider the joint [[probability density function]] of <math>X_1^n=(X_1,\ldots,X_n)</math>. Because the observations are independent, the pdf can be written as a product of individual densities, i.e.
| |
| | |
| :<math>\begin{align}
| |
| f_{X_1^n}(x_1^n)
| |
| &= \prod_{i=1}^n \left({1 \over \beta-\alpha}\right) \mathbf{1}_{ \{ \alpha \leq x_i \leq \beta \} }
| |
| = \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \leq x_i \leq \beta, \, \forall \, i = 1,\ldots,n\}} \\
| |
| &= \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \, \leq \, \min_{1 \leq i \leq n}X_i \} } \mathbf{1}_{ \{ \max_{1 \leq i \leq n}X_i \, \leq \, \beta \} }.
| |
| \end{align}</math>
| |
| | |
| The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting
| |
| | |
| :<math>\begin{align}
| |
| h(x_1^n)= 1, \quad
| |
| g_{(\alpha, \beta)}(x_1^n)= \left({1 \over \beta-\alpha}\right)^n \mathbf{1}_{ \{ \alpha \, \leq \, \min_{1 \leq i \leq n}X_i \} } \mathbf{1}_{ \{ \max_{1 \leq i \leq n}X_i \, \leq \, \beta \} }.
| |
| \end{align}</math>
| |
| | |
| Since <math>h(x_1^n)</math> does not depend on the parameter <math>(\alpha, \beta)</math> and <math>g_{(\alpha \, , \, \beta)}(x_1^n)</math> depends only on <math>x_1^n</math> through the function <math>T(X_1^n)= \left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right),\,</math>
| |
| | |
| the Fisher–Neyman factorization theorem implies <math>T(X_1^n) = \left(\min_{1 \leq i \leq n}X_i,\max_{1 \leq i \leq n}X_i\right)\,</math> is a sufficient statistic for <math>(\alpha\, , \, \beta)</math>.
| |
| | |
| ===Poisson distribution===
| |
| | |
| If ''X''<sub>1</sub>, ...., ''X''<sub>''n''</sub> are independent and have a [[Poisson distribution]] with parameter ''λ'', then the sum ''T''(''X'') = ''X''<sub>1</sub> + ... + ''X''<sub>''n''</sub> is a sufficient statistic for ''λ''.
| |
| | |
| To see this, consider the joint probability distribution:
| |
| | |
| :<math>
| |
| \Pr(X=x)=P(X_1=x_1,X_2=x_2,\ldots,X_n=x_n). \,
| |
| </math>
| |
| | |
| Because the observations are independent, this can be written as
| |
| | |
| :<math>
| |
| {e^{-\lambda} \lambda^{x_1} \over x_1 !} \cdot
| |
| {e^{-\lambda} \lambda^{x_2} \over x_2 !} \cdots
| |
| {e^{-\lambda} \lambda^{x_n} \over x_n !} \,
| |
| </math>
| |
| | |
| which may be written as
| |
| | |
| :<math>
| |
| e^{-n\lambda} \lambda^{(x_1+x_2+\cdots+x_n)} \cdot
| |
| {1 \over x_1 ! x_2 !\cdots x_n ! } \,
| |
| </math>
| |
| | |
| which shows that the factorization criterion is satisfied, where ''h''(''x'') is the reciprocal of the product of the factorials. Note the parameter λ interacts with the data only through its sum ''T''(''X'').
| |
| | |
| ===Normal distribution===
| |
| | |
| If <math>X_1,\dots,X_n</math> are independent and [[Normal Distribution|normally distributed]] with expected value ''θ'' (a parameter) and known finite variance <math>\sigma^{2}</math>, then <math>T(X_1^n)=\overline{X}=\frac1n\sum_{i=1}^nX_i</math> is a sufficient statistic for θ.
| |
| | |
| To see this, consider the joint [[probability density function]] of <math>X_1^n=(X_1,\dots,X_n)</math>. Because the observations are independent, the pdf can be written as a product of individual densities, i.e. -
| |
| | |
| :<math>\begin{align}
| |
| f_{X_1^n}(x_1^n)
| |
| & = \prod_{i=1}^n \tfrac{1}{\sqrt{2\pi\sigma^2}}\, e^{-(x_i-\theta)^2/(2\sigma^2)}
| |
| = (2\pi\sigma^2)^{-n/2}\, e^{ -\sum_{i=1}^n(x_i-\theta)^2/(2\sigma^2)} \\
| |
| & = (2\pi\sigma^2)^{-n/2}\, e^{ -\sum_{i=1}^n( (x_i-\overline{x}) - (\theta-\overline{x}) )^2/(2\sigma^2)} \\
| |
| & = (2\pi\sigma^2)^{-n/2}\, \exp \left( {-1\over2\sigma^2} \left(\sum_{i=1}^n(x_i-\overline{x})^2 + \sum_{i=1}^n(\theta-\overline{x})^2 -2\sum_{i=1}^n(x_i-\overline{x})(\theta-\overline{x})\right) \right).
| |
| \end{align}</math>
| |
| | |
| Then, since <math>\sum_{i=1}^n(x_i-\overline{x})(\theta-\overline{x})=0</math>, which can be shown simply by expanding this term,
| |
| | |
| :<math>\begin{align}
| |
| f_{X_1^n}(x_1^n)
| |
| &= (2\pi\sigma^2)^{-n\over2}\, e^{ {-1\over2\sigma^2} (\sum_{i=1}^n(x_i-\overline{x})^2 + n(\theta-\overline{x})^2) }
| |
| &= (2\pi\sigma^2)^{-n\over2}\, e^{ {-1\over2\sigma^2} \sum_{i=1}^n(x_i-\overline{x})^2}\, e^{ {-n\over2\sigma^2}(\theta-\overline{x})^2 }.
| |
| \end{align}</math>
| |
| | |
| The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting
| |
| | |
| :<math>\begin{align}
| |
| h(x_1^n)= (2\pi\sigma^2)^{-n\over2}\, e^{ {-1\over2\sigma^2} \sum_{i=1}^n(x_i-\overline{x})^2},\,\,\,
| |
| g_{\theta}(x_1^n)= e^{ {-n\over2\sigma^2}(\theta-\overline{x})^2 }.
| |
| \end{align}</math>
| |
| | |
| Since <math>h(x_1^n)</math> does not depend on the parameter <math>\theta</math> and <math>g_{\theta}(x_1^n)</math> depends only on <math>x_1^n</math> through the function <math>T(X_1^n)=\overline{X}=\frac1n\sum_{i=1}^nX_i,</math>
| |
| | |
| the Fisher–Neyman factorization theorem implies <math>T(X_1^n)=\overline{X}=\frac1n\sum_{i=1}^nX_i</math> is a sufficient statistic for <math>\theta</math>.
| |
| | |
| ===Exponential distribution===
| |
| | |
| If <math>X_1,\dots,X_n</math> are independent and [[Exponential distribution|exponentially distributed]] with expected value ''θ'' (an unknown real-valued positive parameter), then <math>T(X_1^n)=\sum_{i=1}^nX_i</math> is a sufficient statistic for θ.
| |
| | |
| To see this, consider the joint [[probability density function]] of <math>X_1^n=(X_1,\dots,X_n)</math>. Because the observations are independent, the pdf can be written as a product of individual densities, i.e. -
| |
| | |
| :<math>\begin{align}
| |
| f_{X_1^n}(x_1^n)
| |
| &= \prod_{i=1}^n {1 \over \theta} \, e^{ {-1 \over \theta}x_i }
| |
| = {1 \over \theta^n}\, e^{ {-1 \over \theta} \sum_{i=1}^nx_i }.
| |
| \end{align}</math>
| |
| | |
| The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting
| |
| | |
| :<math>\begin{align}
| |
| h(x_1^n)= 1,\,\,\,
| |
| g_{\theta}(x_1^n)= {1 \over \theta^n}\, e^{ {-1 \over \theta} \sum_{i=1}^nx_i }.
| |
| \end{align}</math>
| |
| | |
| Since <math>h(x_1^n)</math> does not depend on the parameter <math>\theta</math> and <math>g_{\theta}(x_1^n)</math> depends only on <math>x_1^n</math> through the function <math>T(X_1^n)=\sum_{i=1}^nX_i</math>
| |
| | |
| the Fisher–Neyman factorization theorem implies <math>T(X_1^n)=\sum_{i=1}^nX_i</math> is a sufficient statistic for <math>\theta</math>.
| |
| | |
| ===Gamma distribution===
| |
| | |
| If <math>X_1,\dots,X_n\,</math> are independent and distributed as a [[Gamma distribution|<math>\Gamma(\alpha \, , \, \beta) \,\,</math>]], where <math>\alpha\,</math> and <math>\beta\,</math> are unknown parameters of a [[Gamma distribution]], then <math>T(X_1^n) = \left( \prod_{i=1}^n{x_i} , \sum_{i=1}^n x_i \right)\,</math> is a two-dimensional sufficient statistic for <math>(\alpha, \beta)</math>.
| |
| | |
| To see this, consider the joint [[probability density function]] of <math>X_1^n=(X_1,\dots,X_n)</math>. Because the observations are independent, the pdf can be written as a product of individual densities, i.e. -
| |
| | |
| :<math>\begin{align}
| |
| f_{X_1^n}(x_1^n)
| |
| &= \prod_{i=1}^n \left({1 \over \Gamma(\alpha) \beta^{\alpha}}\right) x_i^{\alpha -1} e^{{-1 \over \beta}x_i}
| |
| &= \left({1 \over \Gamma(\alpha) \beta^{\alpha}}\right)^n \left(\prod_{i=1}^n x_i\right)^{\alpha-1} e^{{-1 \over \beta} \sum_{i=1}^n{x_i}}.
| |
| \end{align}</math>
| |
| | |
| The joint density of the sample takes the form required by the Fisher–Neyman factorization theorem, by letting
| |
| | |
| :<math>\begin{align}
| |
| h(x_1^n)= 1,\,\,\,
| |
| g_{(\alpha \, , \, \beta)}(x_1^n)= \left({1 \over \Gamma(\alpha) \beta^{\alpha}}\right)^n \left(\prod_{i=1}^n x_i\right)^{\alpha-1} e^{{-1 \over \beta} \sum_{i=1}^n{x_i}}.
| |
| \end{align}</math>
| |
| | |
| Since <math>h(x_1^n)</math> does not depend on the parameter <math>(\alpha\, , \, \beta)</math> and <math>g_{(\alpha \, , \, \beta)}(x_1^n)</math> depends only on <math>x_1^n</math> through the function <math>T(X_1^n)= \left( \prod_{i=1}^n{x_i} , \sum_{i=1}^n{x_i} \right),</math>
| |
| | |
| the Fisher–Neyman factorization theorem implies <math>T(X_1^n)= \left( \prod_{i=1}^n{x_i} , \sum_{i=1}^n{x_i} \right)</math> is a sufficient statistic for <math>(\alpha\, , \, \beta).</math>
| |
| | |
| ==Rao–Blackwell theorem==
| |
| | |
| '''Sufficiency''' finds a useful application in the [[Rao–Blackwell theorem]], which states that if ''g''(''X'') is any kind of estimator of ''θ'', then typically the conditional expectation of ''g''(''X'') given sufficient statistic ''T''(''X'') is a better estimator of ''θ'', and is never worse. Sometimes one can very easily construct a very crude estimator ''g''(''X''), and then evaluate that conditional expected value to get an estimator that is in various senses optimal.
| |
| | |
| ==Exponential family==
| |
| {{main|Exponential family}}
| |
| According to the '''Pitman–Koopman–Darmois theorem,''' among families of probability distributions whose domain does not vary with the parameter being estimated, only in [[exponential family|exponential families]] is there a sufficient statistic whose dimension remains bounded as sample size increases. Less tersely, suppose <math>X_n, n = 1, 2, 3, \dots</math> are [[independent identically distributed]] random variables whose distribution is known to be in some family of probability distributions. Only if that family is an exponential family is there a (possibly vector-valued) sufficient statistic <math>T(X_1, \dots, X_n)</math> whose number of scalar components does not increase as the sample size ''n'' increases.
| |
| | |
| This theorem shows that sufficiency (or rather, the existence of a scalar or vector-valued of bounded dimension sufficient statistic) sharply restricts the possible forms of the distribution.
| |
| | |
| ==Other types of sufficiency==
| |
| ===Bayesian sufficiency===
| |
| | |
| An alternative formulation of the condition that a statistic be sufficient, set in a Bayesian context, involves the posterior distributions obtained by using the full data-set and by using only a statistic. Thus the requirement is that, for almost every x,
| |
| | |
| :<math>\Pr(\theta|X=x) = \Pr(\theta|T(X)=t(x)). \,</math>
| |
| | |
| It turns out that this "Bayesian sufficiency" is a consequence of the formulation above,<ref>{{cite book
| |
| |last1=Bernardo |first1=J.M. |authorlink1=José-Miguel Bernardo
| |
| |last2=Smith |first2=A.F.M. |authorlink2=Adrian Smith (academic)
| |
| |year=1994
| |
| |title=Bayesian Theory
| |
| |publisher=Wiley
| |
| |isbn=0-471-92416-4
| |
| |chapter=Section 5.1.4
| |
| }}</ref> however they are not directly equivalent in the infinite-dimensional case.<ref>{{cite journal
| |
| |last1=Blackwell |first1=D. |authorlink1=David Blackwell
| |
| |last2=Ramamoorthi |first2=R. V.
| |
| |title=A Bayes but not classically sufficient statistic.
| |
| |journal=[[Annals of Statistics]]
| |
| |volume=10 |year=1982 |issue=3 |pages=1025–1026
| |
| |doi=10.1214/aos/1176345895 |mr=663456 | zbl = 0485.62004
| |
| }}</ref> A range of theoretical results for sufficiency in a Bayesian context is available.<ref>{{cite journal
| |
| |last1=Nogales |first1=A.G.
| |
| |last2=Oyola |first2=J.A.
| |
| |last3=Perez |first3=P.
| |
| |year=2000
| |
| |title=On conditional independence and the relationship between sufficiency and invariance under the Bayesian point of view
| |
| |journal=Statistics & Probability Letters
| |
| |volume=46 |issue=1 |pages=75–84
| |
| |doi=10.1016/S0167-7152(99)00089-9 |mr=1731351 | zbl = 0964.62003
| |
| }}</ref>
| |
| | |
| ===Linear sufficiency===
| |
| | |
| A concept called "linear sufficiency" can be formulated in a Bayesian context,<ref>{{cite journal |first=M. |last=Goldstein |first2=A. |last2=O'Hagan |year=1996 |title=Bayes Linear Sufficiency and Systems of Expert Posterior Assessments |journal=[[Journal of the Royal Statistical Society]] |series=Series B |volume=58 |issue=2 |pages=301–316 |jstor=2345978 }}</ref> and more generally.<ref>{{cite journal |last=Godambe |first=V. P. |year=1966 |title=A New Approach to Sampling from Finite Populations. II Distribution-Free Sufficiency |journal=[[Journal of the Royal Statistical Society]] |series=Series B |volume=28 |issue=2 |pages=320–328 |jstor=2984375 }}</ref> First define the best linear predictor of a vector ''Y'' based on ''X'' as <math>\hat E[Y|X]</math>. Then a linear statistic ''T''(''x'') is linear sufficient<ref>{{cite journal |last=Witting |first=T. |year=1987 |title=The linear Markov property in credibility theory |journal=ASTIN Bulletin |volume=17 |issue=1 |pages=71–84 |doi= }}</ref> if
| |
| | |
| :<math>\hat E[\theta|X]= \hat E[\theta|T(X)] . </math>
| |
| | |
| ==See also==
| |
| *[[Completeness (statistics)|Completeness]] of a statistic
| |
| *[[Basu's theorem]] on independence of complete sufficient and ancillary statistics
| |
| *[[Lehmann–Scheffé theorem]]: a complete sufficient estimator is the best estimator of its expectation
| |
| *[[Rao–Blackwell theorem]]
| |
| *[[Sufficient dimension reduction]]
| |
| *[[Ancillary statistic]]
| |
| | |
| ==Notes==
| |
| {{reflist|30em}}
| |
| | |
| ==References==
| |
| * {{Springer|title=Sufficient statistic|id=S/s091070|first=A.S.|last=Kholevo}}
| |
| * {{cite book
| |
| | last = Lehmann
| |
| | first = E. L.
| |
| | coauthors = Casella, G.
| |
| | title = Theory of Point Estimation
| |
| | year = 1998
| |
| | publisher = Springer
| |
| | isbn = 0-387-98502-6
| |
| | edition = 2nd
| |
| | pages = Chapter 4
| |
| | nopp = TRUE }}
| |
| *Dodge, Y. (2003) ''The Oxford Dictionary of Statistical Terms'', OUP. ISBN 0-19-920613-9
| |
| | |
| {{Statistics}}
| |
| | |
| {{DEFAULTSORT:Sufficient Statistic}}
| |
| [[Category:Statistical theory]]
| |
| [[Category:Statistical principles]]
| |
| [[Category:Articles containing proofs]]
| |