Enabling Hybrid HPCQC Workflows with a Heterogeneous Software Stack

AI summary

Overview

Research area: Quantum computing systems engineering, specifically the integration of quantum processors into high-performance computing (HPC) environments (HPCQC).

Technical level: Intermediate. Readers should understand basic HPC scheduling concepts (Slurm, batch jobs, resource allocation) and basic variational quantum algorithms (VQE, QAOA).

Scope: The paper describes the architecture, Slurm integration plugins, and measured latency of a production hybrid HPCQC deployment linking the SuperMUC-NG supercomputer, a 20-qubit IQM superconducting processor, and the Munich Quantum Software Stack.

What This Paper Is About

Quantum computers are scarce, unusual devices, and supercomputing centers cannot easily change their scheduling software to accommodate them. The authors show how to plug a real quantum processor into an existing Slurm-managed supercomputer so that users request a quantum processing unit (QPU) in their job script much like they request a GPU, without modifying Slurm's core code. The goal is to run "hybrid" applications — classical code on the supercomputer calling quantum circuits on the QPU — with negligible overhead compared to ordinary HPC jobs.

Key Contributions

1. A two-level scheduling architecture. Slurm remains the authoritative system-level scheduler and exposes QPUs as Generic RESources (GRES), requested via --gres=qpu:1. A second level, the MQSS Quantum Resource Manager & Compiler Infrastructure (QRM&CI), handles queuing, just-in-time compilation, hardware-aware optimization, and dispatch of quantum circuits to the device.
2. The MQSS SLURM Plugin Suite. An open-source, production-grade integration built from Slurm's native extension mechanisms — Prolog/Epilog scripts and SPANK plugins — so the suite can be deployed, updated, or removed as an external layer with no changes to the Slurm source code.
3. An operational case study at LRZ. A deployment linking SuperMUC-NG with IQM's Q-Exa 20-qubit superconducting system, including the use of the Quantum Device Management Interface (QDMI) to expose device topology, gate fidelities, calibration status, and availability to the scheduler.
4. A portable integration model. The authors argue the same stack applies directly to next-generation HPE Cray platforms, including LRZ's forthcoming "Blue Lion" system.

Main Findings

• VQE benchmark scale: The VQE experiment used a precomputed Hamiltonian for the H2 molecule at a bond length of 0.7414 Angstroms. After Jordan-Wigner mapping the circuit used 4 qubits, with 17 iterations, 16 circuits per iteration, corresponding to 14 Pauli terms.
• VQE did not converge fully: The 17 iterations ran in about 35.67 minutes and reached a minimum energy of only -0.3345; the authors state this does not reach acceptable convergence, which would require hundreds of iterations for this H2 problem, and that their interest was the workflow and latency breakdown.
• VQE latency breakdown: Average submit latency was 54.95 ms, compared with 465.6 ms average classical runtime and 4068 ms average QPU runtime per circuit.
• QAOA benchmark design: The QAOA experiment targeted MaxCut on a 4-node, 4-edge graph using 2 QAOA layers, run for about 50 iterations to reach convergence. The final sampled bitstrings showed a clear peak for 0101 and 1010, the optimal solutions for the selected graph.
• QAOA latency breakdown: Average submit latency was 56.37 ms, MQSS environment load took 4036 ms, and environment unload took 9 ms. The paper does not report a numeric average QPU runtime for QAOA, noting only that it depends on how circuits are submitted.
• Submit latency is negligible: In both benchmarks, the offload latency is small relative to QPU and classical execution times, supporting the claim that hybrid jobs run without significant latency overhead versus conventional workloads.
• Environment load is one-time: Loading the MQSS software environment is required only once per Slurm job launch, and unloading cleans up the connection and releases the QPU.

Methodology in Plain English

The team treated the quantum processor as an accelerator that the batch scheduler owns. In the Slurm configuration, they declared a new resource type alongside GPUs (GresTypes = gpu, qpu) and attached one QPU to compute nodes in slurm.conf and gres.conf. When a job starts, hooks fire at the job-allocation phase: a quantum daemon starts, the QPU is marked "occupied," and an MQSS Client on the compute node opens an authenticated connection to the quantum side. During execution, classical code runs on SuperMUC-NG while quantum circuits are sent through QRM&CI, compiled for the specific hardware, and dispatched over QDMI; results return the same way. When the job ends, the hooks stop the daemon, close the connection, and free the QPU. The authors also implemented a SPANK variant in C for finer-grained control of job steps. To keep utilization high, MQSS computes a "quantum availability" metric from QDMI status queries so Slurm can backfill other work instead of starting a job that would immediately stall. Performance was measured by timestamps at each pipeline stage — user environment load, MQSS offload setup, circuit submission, scheduling, and QPU completion — across VQE and QAOA workloads.

Why This Matters

Impact on research: The work shows that quantum accelerators can be added to a petascale production supercomputer using existing operational policies rather than bespoke infrastructure, giving researchers a realistic path to test hybrid algorithms at scale. The two-level split — global allocation in Slurm, device-level compilation and dispatch in MQSS — is a reusable design pattern for other centers.

Real-world applications:

• Quantum chemistry, illustrated by the H2 VQE simulation used in the benchmark.
• Graph optimization and combinatorial problems, illustrated by the MaxCut QAOA benchmark.
• General hybrid workloads where classical pre- and post-processing surrounds short quantum kernels, run as a single Slurm job.
• Portable deployment on future HPE Cray systems such as Blue Lion, and adoption by other Cray User Group centers.

Industry relevance: The integration is a collaboration between a supercomputing center (LRZ), a quantum hardware vendor (IQM), a national laboratory (Argonne), and a regional quantum initiative (Munich Quantum Valley). It demonstrates vendor-supplied QPU control stacks and device interfaces (QDMI) coexisting with mainstream HPC operations, which matters for vendors and centers planning quantum offerings with familiar user experiences comparable to GPUs or FPGAs.

Future Directions

• Coordinated scheduling across both sides. The current model prioritizes classical allocation, so a reserved classical slot can idle while waiting on the QPU, or vice versa; the authors call for studying coordinated or multilevel scheduling algorithms across the two schedulers.
• Finer-grained QPU sharing. Only a single QPU is exposed as one GRES unit in the demonstrated configuration and is serialized across jobs; concurrent sharing of one QPU among multiple Slurm jobs is described as an open item.
• Broader workload coverage. VQE and QAOA exchange data once per iteration over many short-lived circuits; a systematic study across the full spectrum of coupling granularities is left as future work, and the reported overhead should be read as a conservative upper bound.
• Characterizing QPU utilization at scale. Beyond the per-circuit latency breakdown reported here, the authors list more detailed utilization characterization and validation on Blue Lion as ongoing work.

Target Audience

HPC center operators and system administrators who manage Slurm-based production systems, quantum software and runtime developers, and researchers building hybrid quantum-classical applications. It is also relevant to hardware vendors integrating quantum devices into HPC environments, and to readers of the Cray User Group community planning quantum-accelerated deployments.

Authors’ abstract

In this work, we demonstrate hybrid High Performance Computing-Quantum Computing (HPCQC) workflows on a production petascale system. The demonstration combines three components: the SuperMUC-NG supercomputer at the Leibniz Supercomputing Centre (LRZ), a 20-qubit superconducting quantum processor provided by IQM Quantum Computers (IQM), and Munich Quantum Valley (MQV)'s Munich Quantum Software Stack (MQSS). Integrating quantum processors into High Performance Computing (HPC) systems requires a heterogeneous software stack capable of orchestrating classical and quantum resources within established supercomputing workflows. MQSS treats Quantum Processing Units (QPUs) as scheduler-managed accelerators and it performs resource coordination following a two-level scheduling scheme. Slurm performs system-level allocation by exposing QPUs as Generic RESources (GRES), while the MQSS Quantum Resource Manager & Compiler Infrastructure (QRM&CI) performs just-in-time compilation and subsequent dispatch of quantum circuits. To integrate with existing HPC operations without modifying the scheduler core, MQSS introduces an open-source SLURM Plugin Suite based on Prolog/Epilog scripts and SPANK modules. Experimental results show that hybrid HPCQC workflows can be executed without significant latency overhead compared to conventional workloads. The presented architecture provides a portable integration model for quantum accelerators on large-scale HPC systems and is directly applicable to next-generation Hewlett Packard Enterprise (HPE) Cray platforms, including LRZ's upcoming 'Blue Lion' supercomputer.