Distilling Datasets into Shallow Circuits for Quantum Machine Learning

AI summary

Overview

Research area: Quantum machine learning (QML), specifically the data-loading bottleneck and dataset distillation for quantum models.

Technical level: Advanced. The paper assumes familiarity with parameterized quantum circuits, amplitude encoding, tensor-train / matrix-product-state decompositions, the parameter-shift rule, and classical dataset distillation.

Scope: The paper proposes quantum dataset distillation (QDD), a method that compresses a full classical dataset into a small set of shallow, executable "staircase" loading circuits whose parameters are optimized directly, and evaluates it on MNIST, Fashion-MNIST and three IBM quantum processors.

What This Paper Is About

In quantum machine learning, every training sample must first be converted into a quantum state by a loading circuit, and that circuit must be re-executed for every measurement shot at every training step. The cost therefore scales with both the number of samples and the per-sample preparation cost, and exact amplitude encoding of an arbitrary input generally requires exponentially many two-qubit gates in the number of qubits. The authors argue that existing fixes treat the two problems separately — either compressing individual inputs, or distilling data and then applying a separate quantum encoding — which can leave samples expensive to prepare or lose information during later compilation into shallow circuits. QDD instead distills the dataset directly into a handful of shallow circuits, so that the synthetic sample and its loading circuit are the same object.

Key Contributions

1. A joint framework (QDD). The paper introduces quantum dataset distillation, which addresses dataset size and quantum preparation cost simultaneously by distilling training data directly into executable loading circuits, with no post-hoc state-preparation synthesis.

2. A staircase circuit parameterization. Each synthetic sample is written as an L-layer staircase circuit realizing a low-rank quantized tensor-train (QTT) state. Each two-qubit block contains six trainable angles and two CNOTs, so a loader has an exact cost of 2(n−1)L CNOTs, and the staircase can be embedded on a nearest-neighbor chain without SWAP gates. An L-layer circuit produces an MPS with bond dimension at most 2^L, so depth controls both representational capacity and loading cost.

3. A two-term distillation objective optimized with exact gradients. The loss combines distribution matching of class-conditional mean features of the downstream quantum model across randomly initialized model parameters with direct matching of class-mean density matrices, weighted by α. All states and circuits are contracted exactly at the studied scale and reverse-mode automatic differentiation gives exact gradients; the downstream model is resampled rather than trained during distillation.

4. Empirical validation including real hardware. Comparisons against six selection baselines and a distill-then-compress control under matched sample and loading budgets on MNIST and Fashion-MNIST, finite-shot training-cost measurements, and inference on three IBM quantum processors (ibm_kawasaki, ibm_fez, ibm_boston).

Main Findings

• Compact distilled sets match full-data training. On MNIST and Fashion-MNIST, only 10 circuits per class (0.0017× the full dataset size) achieve accuracy comparable to full-data training. With 42-CNOT loaders, QDD reaches 81.62% ± 0.33 on MNIST and 69.67% ± 0.43 on Fashion-MNIST at IPC 10, versus 82.25% ± 0.37 and 71.01% ± 0.58 at IPC 50.

• QDD beats six selection baselines in five of six settings. Baselines are random, kernel herding, forgetting-based, coreset, k-center greedy, and k-means. The single exception is IPC 1 on Fashion-MNIST, where k-means leads by 0.38%. The largest margin over the strongest baseline is 5.71% at IPC 1 on MNIST, shrinking to roughly 1% to 2% at larger IPC.

• Saturation between IPC 10 and 20. Increasing from IPC 10 to IPC 50 yields only a slight improvement, suggesting 100 distilled circuits already capture much of the information needed to train the chosen quantum model; a finer sweep shows accuracy saturating between IPC 10 and 20.

• Joint optimization matters most under tight gate budgets. On MNIST at 14 CNOTs per sample, QDD improves over distill-then-compress by 9.25% at IPC 10 and 7.56% at IPC 50. At IPC 10 QDD remains ahead at all four budgets (14, 28, 42, 56 CNOTs), with the margin narrowing to about 1% from 28 CNOTs onward. The mean overlap between unconstrained distilled states and their compiled approximations rises from 0.76 at 14 CNOTs to 0.98 at 56 CNOTs.

• Large cumulative-shot savings in finite-shot training. On MNIST, QDD-10 is the only compact set tested that reaches 95% of best full-data accuracy at every shot setting, using 134×, 576×, and 370× fewer cumulative shots at 100, 1000, and 10000 shots per circuit respectively. Random-10, with the same sample count, reaches the 80% target but not the 95% target; Random-1 never reaches the 60% target.

• Shallow loaders are essential on real hardware. On 128 MNIST test images with 4096 shots and no error mitigation, the QDD-10 model with identical 42-CNOT staircase test loaders scores 78.91%, 82.03%, and 82.03% on ibm_kawasaki, ibm_fez, and ibm_boston, close to the full-data model (82.03%, 78.12%, 83.59%) and above Random-10 (63.28%, 71.09%, 71.09%). Exact preparation of the same rank-8 targets, with the QDD-trained model fixed, yields only 6.25% to 24.22% on hardware — roughly 55% to 73% below staircase loading — although it is slightly better in noiseless simulation (85.16% exact, 85.94% sampled).

• Noise models overestimate exact preparation. The archived device noise models substantially overestimate the accuracy of exact state preparation relative to real hardware, which the authors cite as motivation for hardware validation.

• Ablations confirm the design choices. On MNIST at IPC 10, raising the QTT inference rank from 1 to 8 lifts accuracy from 25.57% to 81.62%, with little further gain at rank 16 (81.68%). Increasing the budget from 42 to 56 CNOTs raises accuracy only 0.11%. Adding density-matrix matching to distribution matching improves MNIST accuracy by 2.14% at L = 2 and 2.91% at L = 3 within IPC 1.

Methodology in Plain English

The core idea is to stop treating "the data" and "the circuit that loads the data" as two separate things.

Each synthetic sample is defined as a specific shallow circuit: a stack of layers of two-qubit blocks arranged in a staircase pattern along the qubit chain, using only Ry rotations and CNOTs (the target amplitudes are real). Because zero angles make each block cancel to the identity, the circuit has a clean, fixed cost: 2(n−1)L CNOTs. For the 8-qubit experiments with L = 3, that is 42 CNOTs.

To decide what those circuits should be, the authors optimize their parameters on a classical computer, offline. They do not train the downstream quantum model during this stage. Instead, they repeatedly draw random initializations of the downstream model and ask: do the synthetic circuits produce, on average, the same measurement statistics as the real data for each class? That is the distribution-matching term. Because those statistics are linear in the density matrix, the same term also depends only on the class-mean input state, so they add a second, model-independent term that directly matches class-mean density matrices. Gradients are exact, computed by contracting everything and using reverse-mode automatic differentiation, so no shots are needed during distillation.

After distillation, each optimized circuit is simply exported — no compilation step from a state to a circuit is needed. During downstream training, the loading circuits stay frozen and only the quantum model's angles are updated, either with exact gradients in simulation or with parameter-shift gradients estimated from finite shots. Each parameter-shift step evaluates 2P+1 = 673 circuit settings per sample.

Experimental setup: 10-class classification on MNIST and Fashion-MNIST, images downsampled to 16×16 and mapped to normalized 8-qubit states in QTT order through a rank-8 interface. The downstream model has 16 brickwork sublayers, 112 CNOTs and P = 336 trainable angles, with readout on the four central qubits giving 16 outcomes, the first ten of which define class probabilities. A separate 10-qubit study uses a model with 432 angles and 144 CNOTs. Results are averaged over three training seeds and evaluated on the full 10k-image test set; the hardware study uses three IBM processors, 4096 shots, and no error mitigation.

Why This Matters

Impact on research. The paper reframes quantum training-set construction as a co-design problem: the data representation and the loading circuit should be optimized together rather than sequentially. The distill-then-compress comparison quantifies exactly when this matters — the gap is largest under tight gate budgets and shrinks as compilation becomes more faithful. The finite-shot analysis also connects compact training sets directly to the resource that actually dominates quantum training cost, cumulative circuit executions, rather than only to sample count. The hardware result showing exact preparation collapsing to 6.25%–24.22% while shallow loading stays near 82% is a concrete argument that approximate loading is not merely a convenience but a deployment requirement.

Real-world applications:

• Resource-constrained quantum classifiers: image or signal classification on near-term devices where gate depth and coherence time limit what can be executed.
• Quantum-enhanced medical or scientific imaging pipelines: where datasets are large but the quantum processor time available per experiment is small.
• Hybrid quantum-classical edge or cloud inference: where shallow, pre-compiled test loaders reduce per-shot overhead relative to exact amplitude encoding.
• Benchmarking and prototyping: a reusable, pre-distilled circuit training set that can be shared across downstream model architectures and training runs, amortizing its one-time classical construction cost.

Industry relevance. The work targets a practical cost driver for anyone paying for quantum execution time: shots multiplied by gate depth multiplied by samples. A frozen, pre-distilled training set that reaches 95% of full-data accuracy with more than 100× fewer cumulative shots is directly relevant to hardware providers and to groups evaluating whether quantum models are viable on current devices. The demonstration across three IBM processors, combined with the finding that device noise models substantially overestimate exact-preparation accuracy, is relevant to anyone relying on simulation for deployment decisions.

Future Directions

• Scaling to larger quantum systems. The authors name this as the key open challenge: retaining useful information within shallow circuits as the number of qubits grows. They suggest efficient tensor-network contractions and matching selected observables as promising routes.

• Beyond classification. Applying QDD to regression and generative modeling is proposed as a way to open new applications for compact, circuit-based training data.

• Better characterization of when joint optimization pays off. The benefit depends on how strongly the circuit constrains the synthetic representation, and the reported gap narrows from 9.25% at 14 CNOTs to about 1% from 28 CNOTs onward — understanding and predicting this transition is left open.

• Distillation cost and distillation at scale. The distillation stage itself is run classically with exact contraction and exact gradients, and the paper notes it requires no measurement shots; extending this to settings where exact contraction is infeasible is an implicit requirement for larger problems.

Target Audience

Quantum machine learning researchers and graduate students working on data encoding, circuit compilation, or resource-efficient training; practitioners deploying variational quantum models on near-term hardware who need to control per-shot execution cost; and classical dataset-distillation researchers interested in how the distillation objective changes when the synthetic sample must itself be a physically realizable state-preparation circuit. Readers without a background in tensor networks and parameterized circuits will find the method sections dense, though the experimental findings and the hardware comparison are accessible at an intermediate level.

Authors’ abstract

In quantum machine learning, training a quantum model requires each sample to be prepared as a quantum state by a loading circuit that must be re-executed for every shot at every training step. The total burden therefore scales with both the number of samples and the cost of preparation. Existing approaches reduce the loading cost of individual inputs or distill data and compress their representations before applying a separate quantum encoding. This separation can leave resulting samples costly to prepare or cause additional loss of the information during subsequent compilation into shallow circuits. We propose quantum dataset distillation (QDD), which distills the full dataset directly into a small set of shallow circuits. Each synthetic sample is parameterized as a staircase circuit corresponding to a low-rank tensor network, making the sample and its loading circuit the same object. The circuit parameters are optimized classically with exact gradients using distribution matching under an explicit loading budget, without post-hoc state-preparation synthesis. On MNIST and Fashion-MNIST, only 10 circuits per class ($0.0017\times$ the full dataset size) achieve accuracy comparable to full-data training and outperform selection baselines under matched sample and loading budgets. In finite-shot training, QDD reaches 95\% of the full-data accuracy with more than $100\times$ fewer cumulative shots. Additionally, we validate QDD on real quantum hardware, demonstrating its practical deployment potential.