Complementary Matrix-Gated QKAN Fast-Weight Programmers for Quantum Dynamics Forecasting

AI summary

Overview

Research area: Quantum machine learning and quantum-inspired sequence modeling, specifically fast-weight programmers built on quantum-inspired Kolmogorov–Arnold networks (QKANs), applied to forecasting quantum dynamics.

Technical level: Advanced. The paper assumes familiarity with recurrent/gated sequence models, fast-weight programming, and quantum circuit notation (data re-uploading unitaries, observables, density-matrix expectation values).

Scope: The paper proposes and benchmarks a new coordinate-wise gating rule, Complementary Matrix Gating (CMG), for QKAN-based fast-weight programmers, comparing it against a scalar gate and three other self-modulating rules across seven single-step forecasting benchmarks and direct multi-step forecasting of two simulated quantum systems.

What This Paper Is About

Gated fast-weight programmers store temporal context in time-varying "fast" parameters rather than a recurrent hidden state, which makes them cheaper to train than recurrent quantum models. However, their baseline gate is a single scalar that applies the same retain-versus-write balance to every fast-state coordinate, forcing all parameters to share one memory timescale. The paper asks whether QKAN-based fast-weight programmers can gain coordinate-wise memory control without losing the boundedness and the affine parallel-prefix structure that make the recurrence efficient, and answers with a low-rank matrix gate whose old-state and new-proposal weights are exact complements.

Key Contributions

1. Complementary Matrix Gating (CMG): A new self-modulating update rule in which one sigmoid matrix gate, generated from a rank-one outer product of two affine-head outputs, weights the retained old fast state while its complement weights the new proposal. The update is Θ_t = G_t ⊙ Θ_{t−1} + (1 − G_t) ⊙ Δ_t, an element-wise generalization of the scalar-gated rule.
2. Three companion self-modulating rules for comparison: Only-new, Only-old, and Full variants, following the self-modulating QFWP line of work, with the old-state branch bounded by tanh so that |M̃_old| ≤ 1 and the recurrent kernel cannot geometrically amplify stored updates.
3. Preservation of efficiency properties: The paper shows CMG keeps the bounded convex update coordinate-wise and remains expressible as an affine recurrence Θ_t = A_t ⊙ Θ_{t−1} + B_t, so the whole fast-state trajectory can still be computed by parallel prefix scan.
4. Systematic empirical comparison: Four self-modulating rules versus scalar gating, across four FWP architectures (classical/classical, classical slow with HQKAN fast, HQKAN slow with classical fast, HQKAN in both roles), seven benchmarks, five sequence lengths N ∈ {4, 8, 16, 32, 64}, and direct multi-step horizons H ∈ {4, 8, 16}.

Main Findings

• CMG is the most consistent single-step improvement: Across seven benchmarks and five sequence lengths, CMG gives the most consistent gains over scalar gating, producing the cleanest shift below the diagonal in paired scatter plots, particularly for the QKANFWP and QKAN-QKANFWP families.
• Best-arm dominance: Across 28 configurations in the cross-sequence tables, CMG variants achieve the best MSE in 24 cases.
• Best N = 16 model per dataset (Table II, MSE mean over five seeds with standard deviation): Bessel, CMG QKAN-QKANFWP, 4.94 (6.7) × 10⁻⁶; Damped SHM, CMG FWP, 1.42 (1.2) × 10⁻⁵; NARMA-5, gated QKANFWP from prior work, 5.21 (3.9) × 10⁻⁶; NARMA-10, Only-new QKANFWP, 2.55 (2.5) × 10⁻⁶; DQC, CMG QKANFWP, 6.94 (3.5) × 10⁻⁶; Jaynes–Cummings, CMG QKAN-QKANFWP, 3.18 (1.6) × 10⁻⁶; transmon–resonator, CMG QKAN-QKANFWP, 2.65 (1.9) × 10⁻⁶.
• QKAN-FWP is the exception: When only the slow programmer is QKAN-based (classical fast programmer), the scalar-gated form is already competitive and self-modulation gives only marginal or inconsistent gains. CMG's benefit is framed as stable coordinate-wise old/new balancing that matters most when the quantum-inspired module is used directly in the fast programmer.
• Direct multi-step forecasting: On Jaynes–Cummings and transmon–resonator dynamics simulated with CUDA-Q Dynamics, at N = 64 and horizons of 4, 8, and 16 steps, CMG models maintain MSE on the order of 0.001 or lower (the introduction states order 10⁻⁴ or lower) and improve on their scalar-gated counterparts by at least 91.2%.
• Parameter efficiency: CMG uses the same number of modulation heads as Only-new and Only-old and fewer than Full. Parameter ratios relative to the corresponding gated model: CMG FWP 152.6% (209 params), CMG QKANFWP 162.8% (210), CMG QKAN-QKANFWP 139.3% (234); Full reaches 218.2% (299), 232.6% (300), and 182.1% (306) respectively.
• Learning-curve area (AULC) at N = 64, averaged over the two CUDA-Q datasets and five seeds: CMG QKAN-QKANFWP 2.99 × 10⁻⁵ (lowest overall), CMG QKANFWP 4.45 × 10⁻⁵, CMG FWP 9.17 × 10⁻⁵, versus Only-old and Full values ranging from 9.48 × 10⁻⁵ to 2.86 × 10⁻⁴. CMG's advantage is not a final-epoch artifact: on Jaynes–Cummings it stays among the lowest-error rules after early training, and on transmon–resonator it reaches the low-error regime quickly.
• Longer context helps CMG more: On NARMA the shorter-sequence results are slightly heterogeneous, but CMG QKANFWP still achieves the best MSE at N = 64.

Methodology in Plain English

Fast-weight programming splits a sequence model into a "slow programmer" that reads the current input and writes new values into a memory tensor, and a "fast programmer" that makes the prediction from that memory. The baseline version of this design uses one number, between 0 and 1, to decide how much of the old memory to keep versus how much of the newly proposed memory to write in.

The authors change that single number into a whole grid of numbers, one per memory coordinate. To keep this cheap, the slow programmer produces the grid as the outer product of two small vectors, which is a rank-one matrix. They try four ways of using it: modulating only the new proposal, only the old state (passed through tanh so it cannot blow up), both, or—the new proposal, CMG—using one sigmoid matrix gate for the old state and its exact complement (1 minus the gate) for the new proposal. The complementary form matters because it keeps each coordinate's update a bounded convex combination, exactly like scalar gating but per coordinate, and it keeps the recurrence in a form that can be unrolled in parallel with a prefix scan instead of step by step.

They then plug these rules into four backbone combinations of classical and QKAN-based programmers. QKAN-based programmers use an encoder–QKAN–decoder module whose edge functions are single-qubit data re-uploading circuits. Experiments cover seven normalized univariate time-series tasks—damped simple harmonic motion, the Bessel function J₂(x), NARMA-5, NARMA-10, delayed quantum control, open Jaynes–Cummings, and dispersive transmon–resonator dynamics—with the two quantum-dynamics datasets simulated using CUDA-Q Dynamics (3000 equally spaced time steps each). Models are trained with FlashQKAN in PyTorch, accelerated with CuTe DSL, for 100 epochs using Adam at a learning rate of 10⁻³ and batch size 4, over five independent random seeds, and scored by MSE plus a relative-improvement metric against the scalar-gated baseline.

Why This Matters

Impact on research: The paper isolates a concrete weakness of gated QKAN fast-weight programmers—one shared memory timescale—and shows that fixing it does not require giving up the bounded-update guarantee or the parallel-scan structure that make fast-weight recurrences efficient. It also gives a clean negative result: the benefit depends on where the quantum-inspired module sits, appearing for architectures with a QKAN-based fast programmer but not the reverse. This gives follow-up work a specific baseline and a specific design target.

Real-world applications:

• Forecasting and characterizing the dynamics of superconducting qubit and resonator hardware, such as the transmon–resonator dispersive model studied here.
• Predicting cavity-QED and open-system observables (qubit excitation probability, quadrature expectation values) that are expensive to simulate step by step.
• General long-context multivariate time-series forecasting where recurrent step-by-step evaluation is a bottleneck.
• Serving as a compact surrogate model inside quantum control or calibration pipelines that need fast repeated rollouts.

Industry relevance: The work combines HPC and quantum-computing research centers with vendor AI technology groups, and its training stack (FlashQKAN, CuTe DSL, GPU block tiling) is explicitly engineered for GPU throughput—signaling that quantum-inspired sequence models are being positioned as practical accelerators rather than purely theoretical constructs.

Future Directions

• Resolving why QKAN-FWP does not benefit: The scalar-gated QKAN-FWP was already competitive and self-modulation gave marginal gains; a principled explanation of when coordinate-wise gating pays off would guide architecture selection.
• Scaling and generalization beyond the tested grid: The study covers univariate series, seven benchmarks, sequence lengths up to 64, and horizons up to 16; behavior on multivariate inputs, much longer contexts, larger QKAN blocks, and recursive multi-step rollouts is not established.
• Beyond rank-one gates: All rules here use rank-one outer-product modulators; whether higher-rank or otherwise structured matrix gates add capacity without breaking the affine scan form is open.
• Hardware and noise studies: The quantum dynamics were simulated with CUDA-Q Dynamics rather than executed on hardware; the effect of device noise and of running the QKAN edge functions on real quantum processors remains untested.

Target Audience

Researchers and graduate students in quantum machine learning and quantum-inspired sequence modeling who already understand gated recurrent models and fast-weight programming; practitioners building forecasting models for quantum device dynamics or other long-context time series; and engineers interested in GPU-efficient implementations of QKAN-based architectures. Readers seeking a beginner-level introduction to quantum sequence models or to Kolmogorov–Arnold networks will need background from the cited prior work.

Authors’ abstract

Sequence models must decide what to write into memory and what to retain. In quantum and quantum-inspired sequence learning, nonlinear recurrent updates often require repeated circuit evaluations and sequential backpropagation through time, making long contexts costly. Gated fast-weight programmers (FWPs) based on quantum-inspired Kolmogorov-Arnold networks (QKANs) alleviate this bottleneck by storing context in time-varying fast parameters. However, their scalar gate applies one retention-write balance to every fast-state coordinate, forcing all parameters to share a memory timescale. We introduce Self-Modulating QKAN-based FWPs, which replace this broadcast gate with low-rank-generated element-wise modulation of the new-proposal branch, a bounded old-state branch, or both. We further propose Complementary Matrix Gating (CMG), which uses one sigmoid matrix gate to retain the old state and its complement to write the new proposal. CMG provides coordinate-wise memory control while preserving the bounded convex update and affine prefix-scan structure of scalar gating, at the modulation-head cost of a single-branch rule. We compare four self-modulating rules with scalar gating across four FWP architectures combining classical and QKAN-based slow and fast programmers. Across seven single-step forecasting benchmarks and five sequence lengths, CMG gives the most consistent improvements for architectures whose fast programmer incorporates a QKAN-based module. In direct multi-step forecasting of Jaynes-Cummings and transmon-resonator dynamics simulated with CUDA-Q Dynamics, CMG models maintain mean-squared errors on the order of 0.001 or lower across forecasting horizons of 4, 8, and 16 steps, while improving on their scalar-gated counterparts by at least 91.2%. These results establish coordinate-wise complementary modulation as a stable and effective update for QKAN-based FWPs.