Joint Probability Distributions

Joint Probability Distributions

Joint probability mass/density functions, marginal and conditional distributions, covariance, and correlation.

In many engineering applications, we need to understand the relationship between two or more random variables simultaneously. For example, a structural engineer might study the joint distribution of wind speed (

X

) and atmospheric pressure (

Y

) during a hurricane, or a transportation engineer might model the number of cars (

X

) and trucks (

Y

) arriving at a toll booth.

Joint Probability Mass and Density Functions

Describing the simultaneous behavior of multiple random variables.

Joint Probability Mass Function (Discrete)

For two discrete random variables

X

and

Y

, the joint probability mass function

f(x,y)

gives the probability that

X

takes the specific value

x

AND

Y

takes the specific value

y

simultaneously.

f(x, y) = P(X = x, Y = y)

It must satisfy two conditions:

$f(x, y) \ge 0$ for all $(x, y)$ .
$\sum_{x} \sum_{y} f(x, y) = 1$ .

Joint Probability Density Function (Continuous)

For two continuous random variables

X

and

Y

, the joint probability density function

f(x,y)

represents the probability that

(X, Y)

falls within a specific two-dimensional region

R

in the

xy

-plane. The probability is the volume under the surface

f(x,y)

over region

R

P((X, Y) \in R) = \iint_R f(x, y) \, dx \, dy

It must satisfy two conditions:

$f(x, y) \ge 0$ for all $(x, y)$ .
$\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1$ .

Marginal Distributions

Isolating the behavior of one variable from the joint distribution.

Sometimes we have the joint distribution of

X

and

Y

, but we only care about the distribution of

X

alone, regardless of

Y

. This is called the marginal distribution.

Marginal Probability Distributions

To find the marginal distribution of one variable, we sum (or integrate) out the other variable over its entire range.

Discrete case for $X$ :

g(x) = \sum_{y} f(x, y)

Continuous case for $X$ :

g(x) = \int_{-\infty}^{\infty} f(x, y) \, dy

Similarly,

h(y)

is the marginal distribution for

Y

found by summing or integrating out

x

Conditional Distributions and Independence

How knowledge of one variable affects the probability distribution of another.

Conditional Probability Distribution

The probability distribution of

X

, given that

Y

has taken a specific value

y

. This is analogous to basic conditional probability (

P(A|B) = P(A \cap B) / P(B)

f(x|y) = \frac{f(x, y)}{h(y)} \quad \text{provided } h(y) > 0

Similarly,

f(y|x) = \frac{f(x, y)}{g(x)}

provided

g(x) > 0

Independence of Random Variables

Two random variables

X

and

Y

are independent if and only if their joint probability distribution is the product of their marginal distributions for all possible values of

(x, y)

f(x, y) = g(x) \cdot h(y)

If this holds true, knowing the value of

X

gives no information about the value of

Y

. (e.g., The compressive strength of concrete from Plant A vs. Plant B).

Covariance and Correlation

Measuring the linear relationship between two random variables.

Covariance ( $\sigma_{xy}$ )

A measure of how much two random variables change together. A positive covariance indicates that when

X

is above its mean,

Y

tends to be above its mean (e.g., traffic volume and noise levels). A negative covariance indicates an inverse relationship (e.g., age of asphalt and its flexibility).

\sigma_{xy} = E[(X - \mu_X)(Y - \mu_Y)] = E[XY] - \mu_X\mu_Y

If $X$ and $Y$ are statistically independent, their covariance is zero ( $\sigma_{xy} = 0$ ).
However, a covariance of zero does not necessarily mean they are independent (they could have a non-linear relationship).

Correlation Coefficient ( $\rho_{xy}$ )

A standardized measure of the linear relationship between two variables. Covariance depends on the units of

X

and

Y

, making it hard to interpret the strength of the relationship. The correlation coefficient scales covariance by the standard deviations of both variables, producing a dimensionless value between -1 and 1.

\rho_{xy} = \frac{\sigma_{xy}}{\sigma_x \sigma_y}

$\rho_{xy} = 1$ : Perfect positive linear relationship.
$\rho_{xy} = -1$ : Perfect negative linear relationship.
$\rho_{xy} = 0$ : No linear relationship.

The Bivariate Normal Distribution

The foundational model for two correlated continuous variables.

Bivariate Normal Distribution

When two continuous random variables are individually normally distributed and correlated, their joint behavior is described by the bivariate normal distribution. Its PDF forms a 3-dimensional bell surface (a mound) whose orientation depends on the correlation

\rho

Key properties:

The marginal distributions $g(x)$ and $h(y)$ are both normal.
The conditional distributions $f(x|y)$ and $f(y|x)$ are both normal.
If the correlation $\rho_{xy} = 0$ for a bivariate normal distribution, then $X$ and $Y$ are strictly independent. (This is a special property; for other distributions, $\rho=0$ does not guarantee independence).

Key Takeaways

Joint Distributions ( $f(x,y)$ ): Describe the simultaneous behavior of two random variables.
Marginal Distributions ( $g(x), h(y)$ ): Isolate one variable by summing or integrating out the other.
Conditional Distributions ( $f(x|y)$ ): The behavior of $X$ given a specific value of $Y$ .
Independence: If $X$ and $Y$ are independent, $f(x,y) = g(x) \cdot h(y)$ .
Covariance and Correlation: Measure the linear relationship between variables. Correlation ( $\rho$ ) is standardized, always falling between -1 and 1.
Bivariate Normal: The standard 3D bell-shaped curve for two correlated continuous variables.

PreviousContinuous Probability Distributions - Examples & Applications

Quiz Me

NextJoint Probability Distributions - Examples & Applications

Joint Probability Mass and Density Functions

Joint Probability Mass Function (Discrete)

Joint Probability Density Function (Continuous)

Marginal Distributions

Marginal Probability Distributions

Conditional Distributions and Independence

Conditional Probability Distribution

Independence of Random Variables

Covariance and Correlation

Covariance (σxy\sigma_{xy}σxy​)

Correlation Coefficient (ρxy\rho_{xy}ρxy​)

The Bivariate Normal Distribution

Bivariate Normal Distribution

Covariance ( $\sigma_{xy}$ )

Correlation Coefficient ( $\rho_{xy}$ )