Basu's theorem

In statistics, Basu's theorem states that any boundedly complete minimal sufficient statistic is independent of any ancillary statistic. This is a 1955 result of Debabrata Basu.^[1]

It is often used in statistics as a tool to prove independence of two statistics, by first demonstrating one is complete sufficient and the other is ancillary, then appealing to the theorem.^[2] An example of this is to show that the sample mean and sample variance of a normal distribution are independent statistics, which is done in the Example section below. This property (independence of sample mean and sample variance) characterizes normal distributions.

Statement

Let ${\displaystyle (P_{\theta };\theta \in \Theta )}$ be a family of distributions on a measurable space ${\displaystyle (X,{\mathcal {A}})}$ and a statistic ${\displaystyle T}$ maps from ${\displaystyle (X,{\mathcal {A}})}$ to some measurable space ${\displaystyle (Y,{\mathcal {B}})}$. If ${\displaystyle T}$ is a boundedly complete sufficient statistic for ${\displaystyle \theta }$, and ${\displaystyle A}$ is ancillary to ${\displaystyle \theta }$, then conditional on ${\displaystyle \theta }$, ${\displaystyle T}$ is independent of ${\displaystyle A}$. That is, ${\displaystyle T\perp \!\!\!\perp A|\theta }$.

Proof

Let ${\displaystyle P_{\theta }^{T}}$ and ${\displaystyle P_{\theta }^{A}}$ be the marginal distributions of ${\displaystyle T}$ and ${\displaystyle A}$ respectively.

Denote by ${\displaystyle A^{-1}(B)}$ the preimage of a set ${\displaystyle B}$ under the map ${\displaystyle A}$. For any measurable set ${\displaystyle B\in {\mathcal {B}}}$ we have

${\displaystyle P_{\theta }^{A}(B)=P_{\theta }(A^{-1}(B))=\int _{Y}P_{\theta }(A^{-1}(B)\mid T=t)\ P_{\theta }^{T}(dt).}$

The distribution ${\displaystyle P_{\theta }^{A}}$ does not depend on ${\displaystyle \theta }$ because ${\displaystyle A}$ is ancillary. Likewise, ${\displaystyle P_{\theta }(\cdot \mid T=t)}$ does not depend on ${\displaystyle \theta }$ because ${\displaystyle T}$ is sufficient. Therefore

${\displaystyle \int _{Y}{\big [}P(A^{-1}(B)\mid T=t)-P^{A}(B){\big ]}\ P_{\theta }^{T}(dt)=0.}$

Note the integrand (the function inside the integral) is a function of ${\displaystyle t}$ and not ${\displaystyle \theta }$. Therefore, since ${\displaystyle T}$ is boundedly complete the function

${\displaystyle g(t)=P(A^{-1}(B)\mid T=t)-P^{A}(B)}$

is zero for ${\displaystyle P_{\theta }^{T}}$ almost all values of ${\displaystyle t}$ and thus

${\displaystyle P(A^{-1}(B)\mid T=t)=P^{A}(B)}$

for almost all ${\displaystyle t}$. Therefore, ${\displaystyle A}$ is independent of ${\displaystyle T}$.

Example

Independence of sample mean and sample variance of a normal distribution

Let X₁, X₂, ..., X_n be independent, identically distributed normal random variables with mean μ and variance σ².

Then with respect to the parameter μ, one can show that

${\displaystyle {\widehat {\mu }}={\frac {\sum X_{i}}{n}},}$

the sample mean, is a complete and sufficient statistic – it is all the information one can derive to estimate μ, and no more – and

${\displaystyle {\widehat {\sigma }}^{2}={\frac {\sum \left(X_{i}-{\bar {X}}\right)^{2}}{n-1}},}$

the sample variance, is an ancillary statistic – its distribution does not depend on μ.

Therefore, from Basu's theorem it follows that these statistics are independent conditional on ${\displaystyle \mu }$, conditional on ${\displaystyle \sigma ^{2}}$.

This independence result can also be proven by Cochran's theorem.

Further, this property (that the sample mean and sample variance of the normal distribution are independent) characterizes the normal distribution – no other distribution has this property.^[3]

Notes

References

• Basu, D. (1955). "On Statistics Independent of a Complete Sufficient Statistic". Sankhyā. 15 (4): 377–380. JSTOR 25048259. MR 0074745. Zbl 0068.13401.
• Mukhopadhyay, Nitis (2000). Probability and Statistical Inference. Statistics: A Series of Textbooks and Monographs. 162. Florida: CRC Press USA. ISBN 0-8247-0379-0.
• Boos, Dennis D.; Oliver, Jacqueline M. Hughes (Aug 1998). "Applications of Basu's Theorem". The American Statistician. 52 (3): 218–221. doi:10.2307/2685927. JSTOR 2685927. MR 1650407.
• Ghosh, Malay (October 2002). "Basu's Theorem with Applications: A Personalistic Review". Sankhyā: The Indian Journal of Statistics, Series A. 64 (3): 509–531. JSTOR 25051412. MR 1985397.