Data Compression (Schumacher)

Shannon's source coding theorem says a classical source of entropy $H$ can be compressed to $H$ bits per symbol. Benjamin Schumacher's 1995 theorem is the quantum counterpart, and it is what gives the von Neumann entropy its deepest operational meaning: $S(\rho)$ is the number of qubits per signal needed to faithfully compress a quantum source.

The quantum source

A quantum source emits states $|\psi_x\rangle$ with probabilities $p_x$ , so a single emission is described by the density operator

\rho = \sum_x p_x |\psi_x\rangle\langle\psi_x|.

We want to store a long string of $n$ independent emissions, $\rho^{\otimes n}$ , using as few qubits as possible, and later recover the original string with fidelity approaching $1$ . The signal states may be non-orthogonal — this is the genuinely quantum difficulty, since non-orthogonal states cannot be perfectly distinguished or cloned.

The typical subspace

The key object is the typical subspace. Diagonalize $\rho = \sum_k \lambda_k |k\rangle\langle k|$ . The eigenvalues $\{\lambda_k\}$ form a probability distribution, and over $n$ copies the high-probability eigenstrings $|k_1\rangle\cdots|k_n\rangle$ are exactly the typical sequences of that distribution. By the asymptotic equipartition property, the number of typical sequences is about

2^{\,n\,H(\{\lambda_k\})} = 2^{\,n\,S(\rho)},

and they carry essentially all the weight of $\rho^{\otimes n}$ . The span of these typical eigenstrings is the typical subspace $\mathcal{T}$ , of dimension $\approx 2^{nS(\rho)}$ . Almost all of the probability of $\rho^{\otimes n}$ lives inside $\mathcal{T}$ : $\operatorname{Tr}(\Pi_{\mathcal{T}}\,\rho^{\otimes n}) \to 1$ as $n \to \infty$ .

Schumacher's theorem

Schumacher quantum source coding. A memoryless quantum source $\rho$ can be compressed to $R$ qubits per signal with fidelity approaching $1$ if and only if $R > S(\rho)$ . The compression rate $S(\rho)$ is optimal: no scheme with $R < S(\rho)$ can recover the source faithfully.

The compression works by projecting $\rho^{\otimes n}$ onto the typical subspace (which barely disturbs it, since the state is almost entirely in $\mathcal{T}$ ), then encoding the $\approx 2^{nS(\rho)}$ -dimensional typical subspace into $nS(\rho)$ qubits. Decompression is the reverse embedding. The fidelity of the recovered state with the original approaches $1$ .

Why von Neumann entropy is the rate

The bound is no accident of the construction: $S(\rho)$ is the entropy of the eigenvalue distribution, and the typical subspace dimension is fixed by that distribution's classical entropy. Quantum compression reduces to classical typicality applied to the eigenbasis of $\rho$ . The endpoints are instructive:

Pure source ( $S = 0$ ): every emission is the same known state, $0$ qubits per signal suffice — there is nothing to transmit.
Maximally mixed source ( $S = \log_2 d$ ): incompressible, requiring the full $\log_2 d$ qubits per signal.
Orthogonal signal states: $\rho$ has a flat-ish spectrum tied to $\{p_x\}$ , and the quantum rate $S(\rho)$ coincides with the classical Shannon rate $H(\{p_x\})$ — quantum compression reduces to classical compression when the alphabet is distinguishable.

Schumacher's theorem completes the parallel begun in lesson 1: Shannon entropy is the classical compression rate in bits; von Neumann entropy is the quantum compression rate in qubits. Both are the fundamental, achievable limits set by the structure of the source.