Systematic Experiment Tracking in Quantum Software: A Case Study of Reservoir Computing with Error Mitigation

AI summary

Overview

Research area: Quantum software engineering, specifically experiment tracking and provenance for quantum software development, demonstrated through a quantum machine learning case study (quantum reservoir computing with tensor-network error mitigation).

Technical level: Intermediate. Readers need familiarity with gate-model quantum circuits, NISQ-era noise and error mitigation concepts, and basic machine learning workflow terminology, but the paper's argument is primarily about process and tooling rather than deep theory.

Scope: The paper defines experiment tracking practices for quantum software development and demonstrates, via a four-stage noisy quantum reservoir computing pipeline for chaotic time-series prediction, how the classical tool MLflow can be adapted to capture quantum-specific parameters, metrics, and artifacts.

What This Paper Is About

Quantum computing experiments produce far more recordable data than classical machine learning runs — circuit structure, transpilation and qubit-routing decisions, calibration and noise characteristics, measurement schemes, and error-mitigation configuration — yet existing experiment-tracking frameworks were designed for classical ML and lack these features. The authors ask how practitioners can be helped to collect experimental data systematically in order to make better software engineering decisions, and they answer it by defining tracking practices for quantum development and showing a working implementation in a realistic, noisy, error-mitigated case study.

Key Contributions

1. Definition of experiment tracking in the quantum software development context. The authors introduce and define what experiment tracking means for quantum software, arguing that quantum execution must be monitored, marginal gains aggregated, and error sources detected across iterative development cycles.
2. Integration of literature reproducibility practices into quantum tracking. They connect existing reproducibility and provenance work (notably the QProv quantum provenance taxonomy, which spans quantum circuit, quantum computer, compilation, and execution categories) with practical experiment tracking, and identify how tracking improves the development process.
3. Identification of quantum-specific tracking dimensions. They point to dimensions characteristic of quantum computing that have not been observed in classical ML pipelines — circuit-related parameters, measurement result handling, and noise characteristics — which impose new requirements on tracking tools.
4. A demonstrated MLflow adaptation on a realistic pipeline. They show MLflow tracking a noisy, error-mitigated quantum reservoir computing model for time-series prediction, spanning classical data preparation, noisy execution with a classical-shadows-based POVM measurement scheme, tensor-network error mitigation, and classical readout.

Main Findings

• Classical ML trackers are insufficient but adaptable: existing experiment-tracking frameworks are built for classical ML and lack quantum-specific features, while MLflow's ability to log arbitrary parameters, metrics, and artifacts was sufficient to meet the needs of the case study.
• Tracking dimension 1 — circuit metadata: the quantum provenance taxonomy covers used gates, used measurements, execution order, circuit width, circuit depth, circuit size, and applied encoding.
• Tracking dimension 2 — hardware metadata: number of qubits, decoherence times (T1 relaxation and T2 dephasing), qubit connectivity, gate set, gate fidelities, gate times, and readout fidelities; core QPU characterization metrics are T1, T2, gate error rates, and readout error rates.
• Tracking dimension 3 — compilation metadata: qubit assignments, gate mappings, optimisation goals, random seed, and compilation time.
• Tracking dimension 4 — execution metadata: input data, output data, number of shots, intermediate results, number of iterations, execution time, and applied error mitigation.
• Four-stage pipeline with logged hand-offs: classical time-series preparation, workload execution on a noisy simulator, classical error mitigation, and classical readout were run separately so that each stage's logged output becomes the next stage's input, which the authors state makes debugging, logging, and development easier.
• Concrete logged configuration for stage 1: Mackey-Glass series with τ = 17, a = 0.2, b = 0.1, x₀ = 1.2, n = 10; 180 generated data points; window size 5 matching the qubit count; evolution times from t_start = 0.1 to t_end = 3.1 with step 0.6; initial state H^⊗n|0⟩^⊗n; readout basis Z; angle encoding; seed 42; compilation/random seed 42; feature vector dimension after QRC of 75 (15 observables × 5 points); total circuit count of 875.
• Concrete logged configuration for stage 2: 15 observables, 2430 measured observables, shot repetitions of 2, 4860 total measurements, optimization level 3, maximum 10 circuits per job, 875 circuits, AerSimulator backend, depolarizing noise with p1 = 0.0002 for single-qubit gates and p2 = 0.002 for two-qubit gates, density-matrix simulation method, fully connected coupling map, no readout error, no relaxation, listed 1-qubit and 2-qubit gate sets.
• Measurement-scheme scalability limit: because classical shadows cost is governed by the shadow norm, randomized Pauli measurements are efficient for the low-weight observables used here, whereas high-weight or global observables incur a cost growing exponentially with locality; the workflow therefore scales best for readout consisting of many local observables.
• Compilation artifacts matter for error mitigation: the logical-physical qubit mapping produced by the router must be preserved, otherwise error mitigation assigns incorrect inverted noise channels to executed gates; QASM 3.0 was used because it encodes physical qubit mapping and routing automatically.
• Calibration data as a first-class artifact: the module can log the most recent calibration data, which would build the noise model used for error mitigation on real hardware; MLflow's per-experiment timestamps suit experiments spanning multiple recalibrations.
• Error mitigation has scaling limits requiring monitoring: all known error mitigation methods scale only within bounds on qubit and/or gate counts, so developers must monitor effectiveness and weigh cost against improvement.
• Standard deviation as a usable signal: error-mitigated expectation values come with standard deviations, and the authors report observing that this standard deviation can further improve results — the specific values are cut off in the available content.
• Not reported in the available content: the final quantitative prediction accuracy or error metrics of the reservoir computing model, the full contents of the insights section (Section IV), and the detailed discussion of data-management issues for larger experiments. The provided text truncates mid-sentence in Section III-D.

Methodology in Plain English

The authors followed design science research with an objective-centered approach, framed around the question of how to help quantum practitioners collect experimental data and improve decision-making for better software engineering practice.

They first reviewed how quantum software development lifecycles are described in the literature — notably Weder et al.'s three-way split into a quantum workflow lifecycle, a quantum circuit lifecycle, and a classical software lifecycle — and how existing provenance tools (QProv), reproducibility frameworks (QEF), and platforms (Aqueduct, archived with its latest release in 2024) address tracking. From this they distilled the categories of data that quantum experiments generate, based on the QProv quantum provenance taxonomy.

They then built and instrumented a real pipeline: Mackey-Glass chaotic time-series data was generated with reservoirpy, min-max scaled, and cut into sliding windows of five values, with the learning task being to predict the sixth value from five past values. Each window was encoded into a five-qubit circuit through three Hamiltonians — a Pauli-X mixer layer, data-dependent Pauli-Z linear terms, and data-dependent Pauli-Z ⊗ Pauli-Z interaction terms — with a QAOA-like structure evolved for five discrete time steps and measured in multiple bases.

Execution used a custom module built on the POVM toolbox to perform classical-shadows-based state tomography on a noisy five-qubit simulated backend, deliberately configured with relatively high depolarizing noise. The measurement data were then reconstructed as tensor networks with Quimb, combined with approximate inverse noise channels derived from the logged noise model, and contracted to produce error-mitigated expectation values — a custom implementation based on Filippov et al. rather than the premium IBM Quantum function, which is not publicly available. Those expectation values became a 75-dimensional embedding fed to a classical linear readout. Every stage logged its parameters, metrics, and artifacts to MLflow, with the stage boundaries making each hand-off explicit and inspectable.

Why This Matters

The paper's central claim is that systematic, automated experiment tracking is a prerequisite for reproducible quantum experiments as the field moves from proof-of-concept demonstrations toward practical applications, and that existing tooling does not yet cover the quantum-specific data that determine whether a result is trustworthy. Because current NISQ hardware is experimental and its calibration drifts over time, a result reported without the exact device configuration, compilation decisions, and error-mitigation settings at execution time cannot be reproduced or properly interpreted.

Real-world applications:

• Chaotic and nonlinear time-series prediction using quantum reservoirs combined with classical linear readout.
• Evaluating and comparing error mitigation strategies on real devices, tracking measured improvement against qubit and gate overhead.
• Providing publication-grade provenance for academic quantum computing results, so reviewers and other groups can reproduce experiments.
• Instrumenting hybrid classical-quantum pipelines that move between local simulation, HPC or datacenter resources, and QPU execution.
• Establishing quality-metric data collection that feeds proposed quantum software quality models, such as analyzability models for quantum programs.

Industry relevance: organizations offering quantum cloud access and proprietary SDKs, teams building quantum machine learning prototypes, and research software engineers who need audit trails will benefit from knowing which metadata categories actually matter and that a general-purpose tool such as MLflow can carry them without a bespoke platform.

Future Directions

• Extend the tools, not just the practices: the authors argue that quantum-specific dimensions require special features from tracking tools, so improving and adjusting existing trackers for quantum pipelines remains open; today's workaround is the flexibility to log arbitrary parameters and artifacts.
• Scale beyond five qubits: the paper identifies data management issues that larger experiments may cause, and its own case study deliberately uses few qubits to keep runtime reasonable. Handling state-tomography result files and tensor-network parameters such as bond dimension at larger scale is unresolved.
• Move from noisy simulation to real hardware: the execution module is designed to work with current real hardware and targets the Finnish 5- and 50-qubit quantum computers, so validating the tracking and error-mitigation loop against live calibration drift is the natural next step.
• Cover the transition toward error correction: the paper frames error mitigation as necessary because current NISQ hardware is not within the error threshold for known error-correcting codes; tracking requirements — and which metrics matter — will shift as that boundary moves.

Target Audience

Quantum software engineers and research software engineers who build and run quantum experiments; quantum machine learning researchers working with reservoir computing, classical shadows, or tensor-network error mitigation; tooling and platform developers deciding how to extend ML experiment trackers for quantum workloads; and research groups or reviewers concerned with reproducibility standards and provenance reporting in quantum computing publications.

Authors’ abstract

Quantum computers are more widely available than ever, making the field more accessible and widespread. Practitioners are coming from a wide range of domains, conducting experiments and research using quantum computing approaches across a variety of problems. The current literature suggests that developers follow certain methodologies in quantum software development, often with a matching set of tools provided. Yet with the novel paradigm, there are areas that remain unaddressed in practices and tools. In this article, we go into the details of experiment tracking in quantum software development. We explain the basic concept of experiment tracking and detail how, in essence, quantum computing sets demands on tracking practices. Given the experimental state of hardware and the constantly evolving software, quantum execution must be monitored, marginal gains aggregated for the best outcome, and error sources detected. In our case study, quantum reservoir computing for chaotic time series data prediction with error mitigation, we present a detailed quantum software development process and describe how experiments can be tracked throughout development. We then generalize this knowledge into the broader quantum development process.