QuLoC: Photonic Quantum-Assisted Low-Rank LLM Compression

AI summary

Overview

Research area: Quantum physics applied to machine learning — specifically photonic (optical) quantum circuits used as an auxiliary training component for compressing large language models.

Technical level: Advanced. The paper assumes familiarity with transformer MLP blocks (SwiGLU), truncated singular value decomposition, knowledge distillation, and parameterized linear-optical circuits with photon-counting readout.

Scope: The paper introduces QuLoC, a compression algorithm that uses photon-detection probabilities from photonic quantum circuits to generate channel-wise gating coefficients for SVD-initialized low-rank weight factors, then folds those coefficients back into the factors so that inference runs entirely on classical hardware.

What This Paper Is About

Low-rank compression replaces a large weight matrix with two smaller matrices, but at aggressive compression ratios the model loses accuracy on downstream tasks because truncated SVD only minimizes weight reconstruction error, not the error on real inputs. QuLoC asks whether a small photonic quantum circuit can supply an extra, trainable per-channel scaling signal that lets the compressed model recover more of its original accuracy at the same parameter budget, without requiring any quantum hardware at deployment time.

Key Contributions

1. A photonic quantum-assisted low-rank compression algorithm. The low-rank factors reduce the parameter count of the original projection matrices, while a gating vector derived from photonic quantum circuit outputs adjusts the contribution of each rank channel during training — without ever restoring the original full-size weight matrices.

2. A two-stage training strategy. Stage I performs local functional reconstruction of each compressed MLP against its uncompressed counterpart; Stage II performs end-to-end knowledge distillation to recover whole-model language modeling capability and account for interactions between compressed layers.

3. Gain folding for classical deployment. The optimized gating vector is absorbed into the low-rank factors after training (the gate is input-independent because the circuits receive fixed input states), so no photonic quantum circuit is evaluated during inference.

4. Empirical evaluation across two model families and two circuit sizes. Results on Qwen3.5-4B, on LLaMA-7B at 80% and 60% parameter retention, and with four-mode versus sixteen-mode photonic circuits.

Main Findings

• Qwen3.5-4B accuracy gain: Using a sixteen-mode photonic quantum circuit, QuLoC achieves a 9.73% relative improvement in average accuracy across six downstream tasks over the classical baseline Swift-SVD, while reducing the language-model parameter count by 20.05%.

• Compression target and scale: The 32 SwiGLU MLP blocks in Qwen3.5-4B contain approximately 2.265 billion parameters; only these projections are compressed, leaving attention modules and other components unchanged.

• LLaMA-7B at two retention ratios: At model parameter retention ratios of 80% and 60%, mean downstream accuracies are 52% and 44% respectively. Compared with Swift-SVD*, QuLoC is one percentage point higher at 80% retention and ties at 60% retention at the reported precision.

• Smaller circuits lose little: With photonic quantum circuits using four modes and a squeezed vacuum state of light, average downstream accuracy is only 1.48% lower than the sixteen-mode implementation, while still remaining higher than the baselines.

• Gate projection reconstruction was already accurate: The gate-projection reconstruction distance L_gate remained small throughout most of Stage I training, so it was monitored only as a diagnostic metric and not added as a separate weighted term in the local objective.

• Rank allocation is asymmetric: Gate and up projections share a rank r, while the down projection receives 2r, on the heuristic that the down projection maps intermediate features back to the residual stream and directly determines the MLP output.

Methodology in Plain English

Where the compression happens. The authors target the SwiGLU MLP blocks because they hold large dense projection matrices (gate, up, and down projections) and account for a large share of model parameters.

Step 1 — Start from SVD. Each of the three projection matrices is factored into two smaller matrices P_k and B_k using truncated singular value decomposition, which gives the best possible approximation of the original weights for a given rank, but not necessarily the best preserved behavior.

Step 2 — Insert a quantum-controlled dial. Each projection gets its own parameterized photonic circuit fed with a fixed input state. The circuit's photon detection probabilities are passed through a small trainable classical matrix and a tanh nonlinearity to produce a gain vector with entries bounded in (0, 2) when the scaling hyperparameter is set to 1. These gains multiply the individual channels of the low-rank bottleneck — for all three projections, unlike standard SwiGLU where only the gate branch is gated.

Step 3 — Repair each layer locally (Stage I). Each compressed MLP is trained independently to match the outputs of the corresponding uncompressed MLP, which receives the same layer input taken from the frozen teacher. The objective mixes a normalized squared error with a cosine-distance term, weighting the full MLP output at 0.8 and the up-projection and down-projection terms at 0.1 each. Classical factors, mapping matrices, and circuit parameters are updated jointly with one shared objective; the SVD-initialized factors get a smaller learning rate so they stay close to their initialization.

Step 4 — Repair the whole model (Stage II). The reconstructed modules are assembled into a student model, all non-compressed parameters are frozen, and the student is trained end-to-end against the teacher. The loss combines next-token cross entropy with a Top-K distillation loss (K = 64, temperature 2, equal weighting of the two terms), computed over a per-position candidate set formed from the teacher's top-K predictions plus the ground-truth token.

Step 5 — Throw the quantum part away. Because the circuits always see the same fixed input, each final gain vector is a constant. It can be multiplied exactly into the B matrix, the circuits and mapping matrices are deleted, and the deployed model is a plain pair of low-rank matrices per projection.

Why This Matters

Impact on research. The work sits at a specific seam between quantum machine learning and practical model compression: rather than replacing the model or the student with a quantum circuit (which would force quantum resources into inference), it uses the circuit purely as a structured, low-dimensional source of trainable parameters during training. It also extends the idea of quantum-assisted distillation from the CNN setting to transformer MLP factorization, and offers a concrete demonstration that a photonic circuit's measurement statistics can serve as a useful inductive bias in a classical optimization loop.

Real-world applications (implied by the compression setting):

• Deploying LLMs on memory-constrained edge devices and consumer hardware where the full parameter footprint does not fit.
• Reducing serving memory cost and hardware requirements in datacenter inference fleets.
• Enabling multiple task-specialized compressed variants of one base model to be stored and swapped cheaply.
• Compressing models for settings where inference must remain purely classical, such as environments without any quantum or photonic hardware.

Industry relevance. The deployment story is deliberately classical: no cryogenics, no photonic hardware, and no quantum simulator at inference. That makes the method compatible with existing model-serving stacks and standard low-rank kernel support, which is the main practical barrier for quantum-adjacent compression techniques.

Future Directions

• Scaling to larger models. The authors explicitly motivate further exploration of photonic quantum-assisted compression for larger models, beyond the 4B and 7B scales tested here.
• More complex agentic tasks. The paper names agentic tasks as a target for follow-up evaluation, since downstream accuracy on standard benchmarks may not capture multi-step reasoning degradation.
• Where the quantum advantage actually comes from. With only a four-mode circuit costing 1.48% accuracy relative to sixteen modes, the question of how circuit size, input state (including squeezed vacuum), and measurement statistics trade off against accuracy and training cost remains open.
• Combining with other compression axes. Since QuLoC only touches MLP projections and leaves attention untouched, whether these gains compose with attention-side compression, quantization, or pruning is unresolved.

Target Audience

Researchers and engineers working on efficient LLM deployment who already understand low-rank decomposition and distillation; quantum machine learning researchers interested in "quantum-assisted" classical pipelines where the quantum device never appears at inference; and photonic hardware groups looking for concrete classical-machine-learning workloads that stress parameterized circuits with measurement-based readout. Readers without a quantum optics background will need to consult the appendices for the circuit preliminaries, which are referenced but not included in the retrieved text.

Authors’ abstract

As LLMs grow in size, compression becomes increasingly important for efficient deployment. SVD-based low-rank compression reduces parameter counts but can degrade downstream performance. To improve performance after compression, we introduce QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training. Model performance is recovered through local functional reconstruction followed by end-to-end knowledge distillation. After training, the gating coefficients are absorbed into the low-rank factors, allowing the compressed model to run on classical hardware without executing quantum circuits during inference. Experiments on Qwen3.5-4B show that QuLoC achieves a 9.73\% relative improvement in average accuracy over state-of-the-art baselines across multiple downstream tasks, demonstrating its effectiveness. We further evaluate QuLoC on LLaMA-7B at different parameter compression ratios. It achieves comparable or higher average downstream accuracy than state-of-the-art baselines, supporting its applicability to a larger model across different compression settings. These results motivate further exploration of photonic quantum-assisted compression for larger models and more complex agentic tasks.