Turning Quantum Compiler Configuration Selection Upside Down: A Top-Down Paradigm

AI summary

Overview

Research area: Quantum computing software — specifically quantum compiler configuration selection (device choice plus Qiskit compiler options) and the use of machine learning and LLMs to automate it.

Technical level: Advanced. The paper assumes familiarity with quantum circuit compilation (layout, routing, translation, optimization levels), noise-aware fidelity estimation, and LLM/ML-based prediction.

Scope (one sentence): The paper introduces an oracle-guided "top-down" alternative to iterative search for choosing a quantum backend and Qiskit compiler options, built on a new exhaustively characterized 268-circuit benchmark (QuOptBench) and a rule-guided ML selector (QCompass).

What This Paper Is About

When a quantum circuit is compiled for real hardware, someone must pick both a target device and a set of compiler options (optimization level, layout, routing, translation method). These two decisions interact, and no single setting — not even the heaviest optimization level, O₃ — is best for every circuit. Existing state-of-the-art tooling (e.g., MQT Predictor) makes the device decision first, then tunes options with reinforcement learning on that device, so a poor initial device choice can cap the achievable fidelity no matter how well options are later tuned. The authors flip this around: because the practical Qiskit configuration space is small enough to enumerate offline (960 configurations per circuit), they build an exhaustive oracle benchmark, mine reusable selection rules from those oracles, and use them to pick configurations directly for unseen circuits without any online search.

Key Contributions

1. QuOptBench, described as the first exhaustive oracle benchmark for Qiskit compiler configuration selection, covering 268 circuits from QASMBench and VeriQBench with empirical oracle configurations found by enumerating all 960 device-option combinations per circuit.
2. An empirical study of ten LLMs as a natural "direct selection" baseline, including a quantitative aggressiveness metric showing that vanilla LLMs systematically favor more complex settings than the oracle.
3. An oracle-guided rule-distillation workflow that contrasts the default Device-Aware O₃ baseline against exhaustive-search oracles, explains the gaps using circuit and device features, and consolidates patterns into 13 transferable device- and option-selection rules (distilled with Claude-Opus-5 from 68 sampled circuits).
4. QCompass, a compact rule-guided ML framework that applies the distilled rules plus lightweight predictors trained on the fully enumerated configuration data of the sampled circuits, enabling fast configuration selection without iterative online search.

Main Findings

• Device choice can be a hard ceiling: Under the paper's settings, 213 of 268 cases (79.5%) had at least one capacity-feasible device whose best fidelity over all option combinations was still lower than the default O₃ configuration fidelity on another feasible device — evidence that a bad device pick cannot be rescued by later option tuning.
• No configuration dominates: FakeAuckland and FakeBrisbane account for 47% and 37% of oracle device choices; O₂ and O₃ account for 31% and 37% of oracle optimization levels, and O₀ accounts for 16%. SABRE is used in more than half of oracle cases for both layout and routing, while translation choices are more diverse.
• Motivating example: For the ising_n10 circuit, O₁ reached about 0.20 Hellinger fidelity versus only 0.12 for O₃ on a fixed device, and fidelity also varied substantially across devices under fixed O₃.
• Vanilla LLMs underperform the simple baseline: All ten LLMs were generally worse than the Device-Aware O₃ baseline on the headroom-normalized gain metric at all three levels (all paired differences statistically significant, p < 0.05). Worst case was Seed-2.0-Lite at −43.89% (Level 1) and −42.90% (Level 2); Qwen3.5-397B-A17B reached −12.88% on Level 1.
• Only one LLM configuration beat the baseline: GPT-5.5 was the only entry with a positive gain on any level, at 0.04% on Level 3 (p = 0.032), and it was also the best-performing LLM on average.
• A clear over-aggressiveness bias: The Δ_Aggressiveness analysis (LLM score minus oracle score) was dominated by positive values across all ten LLMs, meaning LLMs pick more aggressive device/option combinations than the oracle. GPT-5.5 selected FakeAuckland about 20% more often and FakeBrisbane about 20% less often than the oracle distribution.
• Oracle headroom grows with difficulty level: Oracle improvement over the Device-Aware O₃ baseline was 0.03% on Level 1 (104 circuits, <1% headroom), 5.83% on Level 2 (88 circuits, 1–10%), and 31.68% on Level 3 (76 circuits, >10%).
• Oracle configurations are much cheaper to compile: Oracle compilation time was 1.50 s versus 9.89 s for the Device-Aware O₃ baseline on Level 1, and 1.06 s versus 8.61 s on Level 3 — the fidelity-optimal configuration is often also faster.
• Distilled rules help LLMs; QCompass is the best balance: The 13 rules consistently strengthened LLM-based selection, and QCompass provided the strongest overall trade-off between solution quality and selection latency among the compared methods (bottom-up search, nearest-neighbor retrieval, and direct LLM selection).
• Over-aggressive compilation is a major error source: In the 68 sampled circuits, 64.71% had the Device-Aware O₃ baseline more aggressive than the oracle; 52.94% fell in the "Fidelity-loss / O₃^DA more aggressive" category and 27.94% were near-optimal.

Methodology in Plain English

1. Build a fixed playground. The authors took 268 open-source circuits (39 small, 24 medium, and 40 large from QASMBench; 47 variational and 118 combinational from VeriQBench, all in OpenQASM 2.0) and fixed the Qiskit version (V2.4.1) plus one fake backend per qubit-size class with the lowest average gate error: FakeGuadalupeV2 (16 qubits), FakeAuckland (27), FakeBrooklynV2 (65), FakeBrisbane (127).
2. Fix the knobs. Options are optimization level (0–3), layout (dense, sabre, trivial), routing (basic, lookahead, sabre, none), and translation (synthesis, translator, ibm_backend, ibm_dynamic_circuits, ibm_fractional), giving 4 × 3 × 4 × 5 × 4 = 960 configurations per circuit. approximation_degree was fixed to 1.0 to avoid changing circuit semantics, scheduling_method to None, and seed_transpiler to 42.
3. Score everything cheaply. Instead of simulating each compiled circuit (prohibitive in time and memory), they use estimated fidelity — the product of gate and readout fidelities from device calibration data — normalized by the number of non-barrier input operations to produce effective estimated fidelity, so large circuits are not penalized purely by size. This normalization preserves the ranking within a circuit, so the oracle choice is unchanged.
4. Brute-force the oracle. Every circuit was compiled under all 960 configurations with a 600-second per-compilation timeout; only five of the 268 circuits exceeded it under the default O₃. The highest effective estimated fidelity wins, with shorter compilation time as tie-breaker. Total cost: 8,345.40 CPU-hours, reduced to roughly 618 hours wall-clock with 20 workers.
5. Define a fair baseline and difficulty levels. "Device-Aware O₃" picks the feasible device with the lowest average gate error and applies the default O₃ settings (sabre layout and routing, translator translation). Circuits are split into three levels by how much the oracle improves on this baseline.
6. Test direct LLM selection. Ten models were prompted with circuit features, device features, and the candidate option space, and asked to maximize effective estimated fidelity with compilation time as tie-breaker, returning JSON (temperature 0.0, seed 42, max 1,024 output tokens, via OpenRouter).
7. Distill rules from the oracles. They classified cases by fidelity gap, compilation-time ratio, and aggressiveness gap (thresholds 0.01, 1.5, and 2 respectively), sampled 25% of each non-empty category (68 cases, with the remaining 200 held out for evaluation), and had Claude-Opus-5 write one rule per case and then merge them into a compact set of 13 rules, which the authors manually inspected for redundancy and consistency.
8. Deploy the rules. QCompass combines those rules with lightweight predictors trained on the fully enumerated configuration data of the sampled circuits, so a new circuit gets a configuration in one shot.

Why This Matters

Impact on research. The paper reframes quantum compiler tuning: instead of assuming the search space demands iterative optimization, it shows the space (960 configurations here) is small enough that expensive one-time offline characterization can be amortized into reusable knowledge. QuOptBench gives the community a fixed reference with per-circuit oracles, so future methods can report both improvement over a standard baseline and the remaining gap to the best possible choice — something most prior compiler-tuning work could not measure. The aggressiveness metric is also a reusable diagnostic for why learned or prompted selectors fail.

Real-world applications (bullets):

• Cloud quantum job submission, where a service must pick a backend and transpilation settings for each user circuit without running an expensive search per job.
• Variational algorithm workflows (QAOA, EfficientSU2, VQE, QCNN are all in the benchmark), where many related circuits are compiled repeatedly and consistent, fast configuration choices matter.
• Compiler and SDK defaults, where vendors could replace fixed "O₃ everywhere" defaults with circuit-aware recommendation.
• Benchmarking and procurement decisions, where hardware and compiler combinations are compared on fidelity per compilation cost rather than raw gate counts.

Industry relevance. The evaluation is entirely on Qiskit, the most widely used open-source quantum SDK, and on device profiles matching real IBM hardware sizes (16, 27, 65, 127 qubits), so the findings translate directly into tooling that quantum cloud providers and SDK maintainers could ship. The 8,345.40 CPU-hour oracle cost is a one-time expense; per-circuit selection by QCompass avoids repeating it.

Future Directions

• Generalizing beyond the four selected backends: the paper's evaluation includes generalization to additional circuits and devices, but the oracle benchmark itself is tied to Qiskit V2.4.1 and four fake backends; extending the oracle characterization to new devices and compiler versions is an open engineering question.
• How much does the distilling model matter? The authors note an ablation with two alternative LLMs (Section 6.6) to test the sensitivity of the rule set to the model that wrote it, and the robustness of the 13 rules beyond the 68 sampled circuits remains a question worth probing.
• Closing the oracle gap: QCompass offers the best quality/latency balance among compared methods, but the paper's framing implies a remaining gap to the oracle; quantifying and shrinking that gap with richer rule conditions or better predictors is a natural next step.
• Real-hardware validation: all fidelity numbers here are estimated fidelity computed from calibration data rather than measured execution fidelity on physical devices, so confirming that top-down selections hold up on real hardware is an important follow-up.

Target Audience

Quantum software and compiler researchers, quantum cloud platform engineers, and ML-for-systems researchers who work on autotuning or configuration selection. It is also useful for practitioners running workloads on Qiskit who want evidence that "just use O₃" is often the wrong default. Readers should already be comfortable with transpilation concepts (layout, routing, translation) and with fidelity proxies based on device error rates; the LLM and rule-distillation sections are accessible to ML readers, while the benchmark construction and fidelity metric sections are aimed at quantum compilation specialists.

Authors’ abstract

Selecting an effective quantum compiler configuration requires coordinated decisions over both hardware targets and compiler settings, with the ultimate goal of improving circuit execution fidelity. Existing SOTA approaches typically decompose this problem into two sequential stages: a device is chosen first, after which compiler options are tuned on that device through RL-based search. This bottom-up strategy can be fundamentally limited by its initial device decision, since a poorly matched backend may constrain the best fidelity achievable regardless of how effectively later compiler options are optimized. In contrast to classical compiler tuning, where the search space is often prohibitively large, the configuration space of quantum compilers remains manageable enough to support exhaustive search. Consequently, rather than repeatedly searching from candidate configurations, one can derive reusable configuration knowledge from offline oracle results and apply it directly to new circuits. In this paper, we introduce a top-down configuration-selection paradigm and build QuOPTBench, an exhaustively searched benchmark containing oracle configurations for 268 quantum circuits. We first study whether LLMs can directly infer suitable configurations and evaluate ten representative LLMs. The results reveal that vanilla LLMs remain far from the oracle. We then employ an LLM to extract transferable rules from sampled circuits governing configuration selection. Using these rules together with predictors trained on the enumerated configuration data of the sampled circuits, we design QuTOP, a rule-guided ML framework that performs fast configuration selection without iterative online search. Across our evaluation, the distilled rules consistently strengthen LLM-based selection, while QuTOP provides the strongest overall balance between solution quality and selection latency among the baselines.