Estimation

In statistical inference, we use sample data to estimate unknown population parameters or construct intervals using a stated procedure. Examples include the mean compressive strength of a concrete mix ($\mu$) and the proportion of defective rivets in a bridge ($\pi$).

A point estimate is a single value calculated from sample data. For example, the sample mean $\bar{x}$ is a point estimate of the population mean $\mu$. The sample variance $s^2$ is a point estimate of the population variance $\sigma^2$.

Not all estimators are created equal. A good estimator should be:

• Unbiased: The expected value of the estimator equals the true population parameter (e.g., $E[\bar{X}] = \mu$). If we take many samples, the average of our estimates will center exactly on the true value.
• Minimum Variance (Efficient): Among all unbiased estimators, the one with the smallest variance (tightest spread) is preferred. It consistently provides estimates closer to the true value.

How do statisticians derive these formulas (like $\bar{x}$ or $s^2$) in the first place?

• Method of Moments: Equates sample moments (like the sample mean or variance) to population moments to solve for unknown parameters.
• Maximum Likelihood Estimation (MLE): Finds parameter values that maximize the likelihood of the observed data under a specified model. It is widely used, but its bias, variance, and robustness depend on the model and sample.

Because a point estimate varies across samples, we construct a Confidence Interval (CI) for a population parameter. A 95% confidence procedure has 95% long-run coverage: if its assumptions hold and the same sampling-and-interval method is repeated, about 95% of the resulting intervals contain the fixed parameter. After this sample is observed, the particular interval either contains that parameter or it does not; the 95% is not a probability assigned to the fixed parameter.

The term $(\text{Critical Value} \times \text{Standard Error})$ is called the Margin of Error ($E$).

If the population standard deviation $\sigma$ is known and the sampling model supports the normal approximation (exactly for independent normal observations), use the Standard Normal distribution. The positive critical value $z_{1-\alpha/2}$ is the upper-tail cutoff leaving probability $\alpha/2$ above it.

• $\bar{x}$: Sample mean
• $z_{1-\alpha/2}$: Positive Standard Normal critical value with upper-tail area $\alpha/2$
• $\sigma$: Population standard deviation
• $n$: Sample size

When $\sigma$ is unknown, estimate it with the sample standard deviation $s$. For independent observations from a normal population, use Student's $t$ with $\nu=n-1$ degrees of freedom; for nonnormal data, the adequacy of a t-based interval depends on the sample design, sample size, and distribution.

• $\bar{x}$: Sample mean
• $t_{1-\alpha/2,\nu}$: Positive Student's $t$ critical value with upper-tail area $\alpha/2$ and $\nu=n-1$ degrees of freedom
• $s$: Sample standard deviation
• $n$: Sample size

For independent samples with known population variances, this interval compares two population means (e.g., concrete strength from two suppliers). If variances are estimated or the sample design is more complex, use an interval that reflects that model and design.

• $\bar{x}_1, \bar{x}_2$: Sample means of populations 1 and 2
• $z_{1-\alpha/2}$: Positive Standard Normal critical value with upper-tail area $\alpha/2$
• $\sigma_1^2, \sigma_2^2$: Population variances of populations 1 and 2
• $n_1, n_2$: Sample sizes of populations 1 and 2

For independent Bernoulli observations, let $\hat p=x/n$ be the observed sample proportion and $\pi$ the fixed population proportion. Use the Wilson score interval as a practical default; it avoids the impossible limits that can arise from the plain Wald approximation and behaves better near 0 or 1. For very sparse or extreme counts, a small sample, or a requirement for exact binomial coverage, use an exact binomial interval and state its conservatism and assumptions. A success/failure count rule of thumb alone does not guarantee adequate Wald coverage.

• $\hat p=x/n$: Observed sample proportion
• $\pi$: Fixed population proportion being estimated
• $z_{1-\alpha/2}$: Positive Standard Normal critical value with upper-tail area $\alpha/2$
• $n$: Sample size

For a random sample from a normal population, the sample variance gives a Chi-Square interval with $\nu=n-1$ degrees of freedom. The interval is asymmetric because the Chi-Square distribution is skewed. If population normality is not a reasonable model, this exact interval is not justified.

While a confidence interval bounds a population parameter (like the mean $\mu$), engineers often need to bound future individual measurements.

• NIST/SEMATECH e-Handbook: Confidence Limits for the Mean — repeated-sampling confidence-interval interpretation and t-based mean limits.
• NIST/SEMATECH e-Handbook: Confidence Intervals for Proportions — Wilson score and exact binomial intervals; limitations of the Wald interval.
• NIST/SEMATECH e-Handbook: Chi-Square Test for the Variance — Chi-Square variance inference under a normal model and quantile ordering.
• NIST/SEMATECH e-Handbook: Prediction — prediction intervals for future observations under a fitted process model.
• NIST/SEMATECH e-Handbook: Tolerance Intervals for a Normal Distribution — population-content and confidence interpretation under a normal model.