Benchmarking QML Models

A QML model is only as credible as its evaluation. The headline metric for a classifier is accuracy, but computing it correctly — and against a fair baseline — is where many quantum claims quietly fall apart. This lesson nails down the mechanics and the honest comparison.

From expectation to label

A variational classifier outputs an expectation value $f(\boldsymbol{x}) = \langle Z\rangle \in [-1,1]$. To get a discrete prediction we threshold at zero:

With labels $y\in\{-1,+1\}$, accuracy on a dataset of $N$ points is the fraction predicted correctly:

Worked numbers

Suppose a trained model produces these raw outputs on a five-point test set, with the listed true labels:

| $i$ | $f(\boldsymbol{x}_i)$ | $\hat{y}_i = \operatorname{sign} f$ | $y_i$ | correct? | | --- | --- | --- | --- | --- | | 1 | $+0.82$ | $+1$ | $+1$ | yes | | 2 | $-0.30$ | $-1$ | $-1$ | yes | | 3 | $+0.10$ | $+1$ | $-1$ | no | | 4 | $-0.65$ | $-1$ | $-1$ | yes | | 5 | $+0.45$ | $+1$ | $+1$ | yes |

Four of five are correct, so accuracy $= 4/5 = 0.8$.

Beyond a single number

Accuracy alone hides important behavior — round out a benchmark with:

• Balanced accuracy / confusion matrix when classes are imbalanced.
• A fair classical baseline: a tuned classical SVM or small neural net of comparable capacity. An advantage claim means beating this, not a strawman.
• Variance over seeds: report mean ± std across random initializations and data splits.
• Resource cost: circuit depth, shot budget, and wall-clock — a tie on accuracy at $10\times$ the cost is not a win.

Try it

Compute the test accuracy for the model outputs and labels in the table. You are given the raw outputs and the true labels; threshold at zero, count correct predictions, and return the accuracy as a number. Do not hardcode 0.8 — derive it from the arrays so the procedure is what is graded.

Sign in on the full site to ask questions and join the discussion.