Estimation

Point estimation, confidence intervals for means, proportions, and variances, and prediction/tolerance intervals.
In statistical inference, we use sample data to draw conclusions about the entire population. Estimation is the process of determining the most likely value (or range of values) for an unknown population parameter, such as the true mean compressive strength of a concrete mix (μ\mu) or the true proportion of defective rivets in a bridge (π\pi).

Point Estimation

Using a single number to estimate an unknown population parameter.
A point estimate is a single value calculated from sample data. For example, the sample mean xˉ\bar{x} is a point estimate of the population mean μ\mu. The sample variance s2s^2 is a point estimate of the population variance σ2\sigma^2.

Properties of a Good Estimator

Not all estimators are created equal. A good estimator should be:
  • Unbiased: The expected value of the estimator equals the true population parameter (e.g., E[Xˉ]=μE[\bar{X}] = \mu). If we take many samples, the average of our estimates will center exactly on the true value.
  • Minimum Variance (Efficient): Among all unbiased estimators, the one with the smallest variance (tightest spread) is preferred. It consistently provides estimates closer to the true value.

Methods of Point Estimation

How do statisticians derive these formulas (like xˉ\bar{x} or s2s^2) in the first place?
  • Method of Moments: Equates sample moments (like the sample mean or variance) to population moments to solve for unknown parameters.
  • Maximum Likelihood Estimation (MLE): Finds the parameter value that makes the observed sample data the most "likely" to have occurred. It is the most robust and widely used mathematical method for deriving estimators.

Interval Estimation (Confidence Intervals)

Providing a range of plausible values for the parameter, quantifying the uncertainty.
Because a point estimate will almost never exactly equal the true parameter due to sampling error, we construct a Confidence Interval (CI). A CI provides a range of values and a level of confidence (e.g., 95%) that the true parameter lies within that range.
The general structure of a Confidence Interval is:
Point Estimate±(Critical Value×Standard Error) \text{Point Estimate} \pm (\text{Critical Value} \times \text{Standard Error})
The term (Critical Value×Standard Error)(\text{Critical Value} \times \text{Standard Error}) is called the Margin of Error (EE).

1. Confidence Interval for the Mean (μ\mu)

Case 1: Population Variance (σ2\sigma^2) Known

If we know the true standard deviation σ\sigma (rare in practice, but possible with extensive historical data), we use the Standard Normal (ZZ) distribution.
xˉ±Zα/2(σn) \bar{x} \pm Z_{\alpha/2} \left( \frac{\sigma}{\sqrt{n}} \right)

Case 2: Population Variance (σ2\sigma^2) Unknown

This is the most common scenario. We must estimate σ\sigma using the sample standard deviation ss. Because of this added uncertainty, we use the wider Student's t-distribution with n−1n-1 degrees of freedom.
xˉ±tα/2,n−1(sn) \bar{x} \pm t_{\alpha/2, n-1} \left( \frac{s}{\sqrt{n}} \right) Note: As sample size nn gets very large, the t-distribution converges to the Z-distribution, and the two formulas yield nearly identical results.

2. Confidence Interval for the Difference Between Two Means (μ1−μ2\mu_1 - \mu_2)

CI for Difference of Means

Used to compare the averages of two distinct populations (e.g., comparing the strength of concrete from two different suppliers). If variances are known:
(xˉ1−xˉ2)±Zα/2σ12n1+σ22n2 (\bar{x}_1 - \bar{x}_2) \pm Z_{\alpha/2} \sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}

3. Confidence Interval for a Proportion (π\pi or pp)

Used for categorical data (e.g., the percentage of structural beams failing an inspection).
Let pp be the sample proportion (number of successes divided by sample size). If the sample size is large enough (both np≥5np \ge 5 and n(1−p)≥5n(1-p) \ge 5), the sampling distribution of pp is approximately normal.

CI for a Single Proportion

p±Zα/2p(1−p)n p \pm Z_{\alpha/2} \sqrt{\frac{p(1-p)}{n}}

3. Confidence Interval for the Variance (σ2\sigma^2)

Estimating the variability of a process, crucial for quality control.
Because the sampling distribution of the sample variance (s2s^2) is not symmetric, we use the heavily skewed Chi-Square (χ2\chi^2) distribution with n−1n-1 degrees of freedom. The interval is not symmetric around s2s^2.

CI for a Single Variance

(n−1)s2χα/22≤σ2≤(n−1)s2χ1−α/22 \frac{(n-1)s^2}{\chi^2_{\alpha/2}} \le \sigma^2 \le \frac{(n-1)s^2}{\chi^2_{1-\alpha/2}}

Prediction and Tolerance Intervals

Estimating future individual observations rather than population averages.
While a confidence interval bounds a population parameter (like the mean μ\mu), engineers often need to bound future individual measurements.

Prediction Interval

Provides a range that is highly likely (e.g., 95% confidence) to contain a single future observation drawn from the same population. Because an individual observation is much more variable than a sample mean, a prediction interval is always significantly wider than a confidence interval.
xˉ±tα/2,n−1⋅s1+1n \bar{x} \pm t_{\alpha/2, n-1} \cdot s \sqrt{1 + \frac{1}{n}}

Tolerance Interval

Provides a range that is highly likely to contain a specified proportion of the entire population (e.g., 99% of all concrete batches produced). It captures the natural variability of the process. If a 95% tolerance interval for concrete strength is [28 MPa, 35 MPa], we are confident that 95% of all individual batches will fall in this range.
Key Takeaways
  • Point Estimate: A single value (e.g., xˉ\bar{x}, s2s^2) used to estimate a population parameter (μ\mu, σ2\sigma^2). Good estimators are unbiased and have minimum variance.
  • Confidence Interval (CI): A range of plausible values for a population parameter, incorporating a margin of error based on a chosen confidence level (e.g., 95%).
  • Mean CI (Unknown σ\sigma): The most common scenario; uses the sample standard deviation ss and the tt-distribution.
  • Proportion CI: Used for categorical (success/failure) data; relies on the normal approximation (ZZ-distribution) for large samples.
  • Variance CI: Asymmetric interval built using the Chi-Square (χ2\chi^2) distribution.
  • Prediction vs. Confidence: A CI estimates the average, while a Prediction Interval estimates a single future value, making it much wider.
PreviousSampling Distributions - Examples & Applications
Quiz Me
NextEstimation - Examples & Applications