Independence and Correlation

Two events $A$ and $B$ are independent when knowing whether $A$ occurred gives you no information about whether $B$ occurred — and vice versa. The formal criterion is:

$P(A \cap B) = P(A)\,P(B).$

This is the definition, not a derived theorem. When it holds, the conditional probability $P(A \mid B) = P(A \cap B)/P(B)$ reduces to $P(A)$ , confirming that the occurrence of $B$ does not update your probability for $A$ .

Joint distributions and the product rule

When two random variables $X$ and $Y$ are independent, their joint probability factors as a product of the individual (marginal) probabilities:

$P(X = x,\; Y = y) = P(X = x)\,P(Y = y).$

As a concrete example, roll two fair dice. The outcome of die 1 is $X \in \{1,\ldots,6\}$ and the outcome of die 2 is $Y \in \{1,\ldots,6\}$ . Because the dice do not influence each other,

$P(X = 2,\; Y = 5) = \frac{1}{6} \times \frac{1}{6} = \frac{1}{36}.$

More generally, every one of the 36 pairs $(x, y)$ has probability $1/36$ . If the product rule fails — some pairs are more or less likely than $1/36$ — the variables are correlated (or anti-correlated).

Correlation: a numerical measure

The covariance of $X$ and $Y$ quantifies how they move together:

= \langle XY \rangle - \langle X \rangle \langle Y \rangle.$$ Here angle brackets denote expectation values: $\langle X \rangle = \sum_x x\,P(X=x)$. The **Pearson correlation coefficient** normalises the covariance by the standard deviations: $$\rho(X, Y) = \frac{\operatorname{Cov}(X, Y)}{\sigma_X\,\sigma_Y}, \qquad -1 \leq \rho \leq 1.$$ When $X$ and $Y$ are independent, $\langle XY \rangle = \langle X \rangle\langle Y \rangle$ (because the joint distribution factorises), so $\operatorname{Cov}(X,Y) = 0$ and $\rho = 0$. Note the logical direction carefully: **independence implies zero correlation, but zero correlation does not imply independence** in general. ## A worked example Let $X$ be the result of a fair coin ($0$ for tails, $1$ for heads) and let $Y = X$. Then $Y$ is perfectly determined by $X$, so they are maximally correlated. - $\langle X \rangle = 0 \cdot \tfrac{1}{2} + 1 \cdot \tfrac{1}{2} = \tfrac{1}{2}$. - $\langle Y \rangle = \tfrac{1}{2}$ (same). - $\langle XY \rangle = \langle X^2 \rangle = 0^2 \cdot \tfrac{1}{2} + 1^2 \cdot \tfrac{1}{2} = \tfrac{1}{2}$. - $\operatorname{Cov}(X,Y) = \tfrac{1}{2} - \tfrac{1}{2}\cdot\tfrac{1}{2} = \tfrac{1}{4}$. - $\sigma_X^2 = \langle X^2 \rangle - \langle X \rangle^2 = \tfrac{1}{2} - \tfrac{1}{4} = \tfrac{1}{4}$, so $\sigma_X = \tfrac{1}{2}$ and likewise $\sigma_Y = \tfrac{1}{2}$. - $\rho(X,Y) = (\tfrac{1}{4}) / (\tfrac{1}{2} \cdot \tfrac{1}{2}) = 1$, confirming perfect positive correlation. ## Why this matters for quantum mechanics In quantum mechanics, the outcomes of measuring two separate, non-interacting subsystems are independent — the joint probability of observing $x$ on subsystem 1 and $y$ on subsystem 2 is $$p(x, y) = p_1(x)\,p_2(y),$$ exactly the classical product rule. **Entanglement breaks this rule.** For an entangled two-qubit state the joint measurement statistics cannot be written as any product of single-qubit distributions, and there exist measurement choices for which $\operatorname{Cov}$ and $\rho$ are non-zero. (As with the classical caveat above, a particular pair of measurements can still give zero correlation even for an entangled state, so non-factorisability is the sharper signature.) Recognising when a joint distribution does or does not factorise is therefore the direct classical precursor to recognising entanglement. <Callout type="tip"> A quick check: if you can write the joint probability table as an outer product of two marginal probability vectors, the variables are independent. If any cell deviates from the product of its row and column marginals, there is correlation. </Callout>