Expressivity and Trainability

Every variational quantum circuit makes an implicit design choice: how richly should it cover the space of quantum states? The answer determines both what the circuit can learn and how easily it can be trained. These two concerns — expressibility and trainability — pull in opposite directions, and understanding the tension between them is essential for building practical quantum machine learning models.

Expressibility

A parameterised circuit $U(\boldsymbol{\theta})$ acting on $n$ qubits generates a family of quantum states

$\mathcal{F} = \bigl\{U(\boldsymbol{\theta})|0\rangle^{\otimes n} : \boldsymbol{\theta} \in \Theta\bigr\}.$

Expressibility measures how uniformly $\mathcal{F}$ is distributed over the full space of $n$-qubit pure states (the complex projective space $\mathbb{CP}^{2^n - 1}$). A circuit is highly expressible when $\mathcal{F}$ looks indistinguishable from a uniformly random sample from the Haar measure on $n$-qubit unitaries.

Formally, one common measure is the frame potential distance

$\mathcal{E} = \left\|\int_{\boldsymbol{\theta}} \bigl(U(\boldsymbol{\theta})\bigr)^{\otimes t}\bigl(U(\boldsymbol{\theta})^\dagger\bigr)^{\otimes t} d\boldsymbol{\theta} - \int_{\text{Haar}} U^{\otimes t}(U^\dagger)^{\otimes t} dU \right\|^2,$

where the integral is over parameters or Haar measure respectively. A smaller $\mathcal{E}$ means the circuit is a better $t$-design — it mimics the Haar ensemble to $t$-th order.

Barren Plateaus

The central trainability problem for variational circuits is the barren plateau: for a random, highly expressible circuit, the gradient of any cost function with respect to a parameter $\theta_k$ vanishes exponentially in the number of qubits $n$.

More precisely, for a typical $n$-qubit hardware-efficient ansatz with random initialisation, the variance of the gradient satisfies

$\text{Var}\!\left[\frac{\partial C}{\partial \theta_k}\right] \in O\!\left(\frac{1}{2^n}\right),$

so the gradient magnitude shrinks as $1/\sqrt{2^n}$. At $n = 20$ this is roughly $10^{-3}$; at $n = 50$ it is around $10^{-8}$. Standard gradient-based optimisers — gradient descent, Adam — effectively stall because every direction in parameter space is nearly flat.

The barren plateau is not a numerical accident. It is a consequence of the circuit forming an approximate unitary $2$-design: the gradient concentrates exponentially around its (near-zero) mean by a measure-concentration argument, analogous to the concentration of measure on high-dimensional spheres.

What causes a barren plateau?

Three main factors are identified in the literature (Cerezo et al., 2021):

1. Circuit depth and global observables. Deep circuits applied to global (all-qubit) cost functions are the worst case. The gradient of a global observable with respect to any single gate deep in the circuit becomes exponentially small because the unitary before the gate approximates a $2$-design.

2. Entanglement. Heavy entanglement between all qubits spreads local perturbations into global correlations that wash out. Shallow or locally-connected circuits entangle only a bounded number of qubits at each layer and preserve polynomially large gradients near the beginning of training.

3. Noise. Even without the above, hardware noise creates noise-induced barren plateaus for circuits deeper than the coherence-limited depth, since mixed-state averaging also concentrates gradients.

The Expressibility–Trainability Trade-off

These two properties are in direct tension:

| Circuit property | Expressibility | Trainability | |---|---|---| | Random, deep, all-to-all entangling | High (global $2$-design) | Low (barren plateau) | | Shallow, local, structured | Low–moderate | High (polynomial gradients) | | Problem-inspired (e.g., QAOA for graphs) | Task-specific | Often preserved |

A highly expressible circuit can represent any target state but provides no gradient signal to find it. A too-shallow circuit preserves gradients but may not have enough capacity to represent the solution. The sweet spot depends on both the problem and the available hardware depth.

Strategies to Mitigate Barren Plateaus

Several techniques have been proposed to escape or avoid the flat landscape:

Layer-by-layer initialisation. Start with an identity-like circuit and add layers one at a time, pre-training each layer before adding the next. Because early layers are shallow, gradients are non-negligible; deeper layers are added only once earlier layers have found a good direction.

Local cost functions. Replace a global observable $\langle O\rangle$ (which averages over all qubits) with a sum of local two-qubit terms $\sum_i \langle O_i\rangle$. The gradient of a local term depends only on the qubits near $O_i$, giving a gradient that decays polynomially in $n$ rather than exponentially.

Problem-inspired structure. QAOA (Quantum Approximate Optimisation Algorithm) for combinatorial problems encodes the problem Hamiltonian directly into the circuit structure. The ansatz is not random; it has built-in inductive bias toward problem-relevant states and typically avoids global barren plateaus.

Correlation-aware initialisation. Rather than random parameters, initialise near known classical solutions or use tensor-network states as starting points.

Expressibility as a Design Dial

Think of expressibility as a dial with three regimes:

• Too low (under-expressive). The circuit cannot represent the solution at all, regardless of optimisation. This is a capacity failure, not a trainability failure.
• Too high (over-expressive / random). The circuit can represent everything but cannot be trained because of barren plateaus. The model is powerful in theory and useless in practice.
• Well-calibrated. The circuit has enough capacity to represent a good approximation to the solution while retaining polynomially-large gradients throughout training. This requires either structural knowledge of the problem or careful architecture search.

In classical deep learning the analogous tension is between network width/depth (expressivity) and optimisation landscape (trainability). Quantum machine learning inherits this challenge in a particularly severe form because the loss landscape is literally flat almost everywhere for random wide circuits.

Summary

Expressibility and trainability are dual concerns in variational quantum algorithms:

• Expressibility quantifies how well a parameterised circuit samples the full unitary group. It can be measured via frame potential distance from the Haar measure.
• Trainability is destroyed by barren plateaus, where gradient variances shrink as $O(2^{-n})$ for deep, expressive circuits with global cost functions.
• The remedy is structure: local cost functions, shallow architectures, problem-inspired ansatze, and careful initialisation strategies that avoid placing the optimiser in the flat region from the start.

Designing a quantum model is therefore not simply maximising parameters or depth — it is finding the smallest, most structured circuit that can solve the problem while keeping gradients informative throughout training.

Sign in on the full site to ask questions and join the discussion.