Benchmarking the computational power of quantum computers

AI summary

Overview

Research area: Quantum computing hardware benchmarking and fault-tolerant quantum computing.
Technical level: Intermediate.
Scope: This paper introduces the QUOPS benchmark, measures it experimentally on four leading physical-qubit processors and a small fault-tolerant logical processor, and compares measured capability with resource estimates for utility-scale applications.

What This Paper Is About

Most existing quantum-computer benchmarks measure gate or qubit error rates rather than the computational capability of the whole machine, and they are difficult to compare across different qubit technologies. This paper asks how to measure, directly on real hardware, the largest computationally relevant quantum circuits a machine can execute successfully and how quickly it can execute them, in a way that can be compared with the requirements of recognized challenge problems. It then uses that benchmark to quantify the gap to quantum utility and to track a fault-tolerant logical processor.

Key Contributions

1. Defines the QUOPS benchmark and metrics. QUOPS consists of randomized circuits built from alternating layers of single-qubit R_P(θ) rotations and CNOT gates, with independently variable width and size. It defines a capability region, a single-number QUOPS score, and a throughput metric called the QUOPS rate.
2. Performs a cross-platform experimental comparison. The authors measure QUOPS on Google’s Willow, IBM’s ibm_boston, and Quantinuum’s H2-1 and Helios-1, computing directly on physical qubits. They describe this as the first cross-platform benchmarking experiment to directly compare physical- and logical-qubit quantum computations on an identical footing.
3. Translates challenge-problem resource estimates into QUOPS targets. State-of-the-art resource estimates for factoring RSA-2048 and for estimating an energy eigenvalue of FeMoco are converted into approximate QUOPS score and rate requirements, showing that current capability must grow by roughly five orders of magnitude.
4. Benchmarks a fault-tolerant logical processor and projects future architectures. A [[7,1,3]] Steane-code processor implemented on Helios-1 is measured with up to eight logical qubits, and QUOPS capability regions are projected for two families of fault-tolerant architectures.

Main Findings

• Physical-qubit QUOPS scores and rates. Willow (105 transmon qubits, degree-4 connectivity) achieved 216 QUOPS at 2.0 × 10^7 QUOPS/s at width 6. ibm_boston (156 transmon qubits, degree-3 connectivity) achieved 204 QUOPS at 3.1 × 10^5 QUOPS/s at width 6. H2-1 (56 trapped-ion qubits) achieved 1320 QUOPS at 353 QUOPS/s at width 12. Helios-1 (98 trapped-ion qubits) achieved 1504 QUOPS at 303 QUOPS/s at width 16.
• Leakage postselection improves Quantinuum results. On Helios-1, leakage detection increased throughput at 1504 QUOPS to 344 QUOPS/s and enabled 1824 QUOPS at 247 QUOPS/s. On H2-1, it increased throughput at 1320 QUOPS to 360 QUOPS/s and enabled 1392 QUOPS at 231 QUOPS/s.
• Error mitigation trades speed for size. At a 1% polarization threshold, corresponding to a sampling overhead of at most 10^4, Willow achieved 651 QUOPS at 1.2 × 10^4 QUOPS/s; ibm_boston achieved 899 QUOPS at 91 QUOPS/s; H2-1 achieved 7796 QUOPS at 0.09 QUOPS/s; and Helios-1 reached 20442 QUOPS at 0.29 QUOPS/s. These 3–11× increases in QUOPS score came at the cost of 3–4 orders of magnitude reduction in QUOPS rate.
• Connectivity strongly affects capability. The two-dimensional, locally connected transmons show maximum circuit size decreasing as circuit width increases, because non-adjacent CNOT gates are costly. The QCCD trapped-ion architecture has much less of a connectivity penalty, although both ion systems show a small dependence of achievable circuit size on width due to ion transport phase errors.
• Historical growth is steady but insufficient. From quantum volume data going back to 2018, QUOPS scores doubled approximately every 1.4 years for Quantinuum and every 2.1 years for IBM. Continuing those trends would not reach the capability required for useful challenge problems until 2050–2070. Physical-qubit architectures are directly limited by the physical-qubit error rate ε to approximately 1/(2ε) QUOPS, or less with connectivity constraints.
• The utility gap is about five orders of magnitude in score, but not in rate. The analyzed algorithms require width-1399 circuits with 7 × 10^8 Toffoli gates for RSA-2048 and width-1459 circuits with 1 × 10^9 Toffoli gates for FeMoco. Converting these into QUOPS targets gives 2.5 × 10^8 QUOPS for RSA-2048 and 3.4 × 10^8 QUOPS for FeMoco, with rates of 5.7 × 10^3 QUOPS/s and 8.0 × 10^2 QUOPS/s respectively to finish in five days. Today’s physical-qubit processors have or nearly have the necessary rate, but their scores are short by roughly five orders of magnitude.
• The fault-tolerant Steane-code processor is currently smaller than the physical-qubit machines. Implemented on Helios-1 with up to eight logical qubits, it achieved 40 QUOPS at width 4 with a rate of 4.9 QUOPS/s. The s = 40 circuits were compiled into 20 or fewer T gates, with an expected value of 10, and embedded into mirror circuits with double the size. An initial version achieved 24 QUOPS; a second used higher-fidelity magic states and per-shot randomization to reach 40; a third enforced composable fault tolerance and improved parallelization, raising the rate from 1.3 QUOPS/s to 4.9 QUOPS/s.
• Projected fault-tolerant architectures improve capability substantially but still fall short. A FLASQ surface-code architecture with 5 × 10^3–5 × 10^4 physical qubits and a uniform physical gate error rate of ε = 10^-3 achieves QUOPS scores from 2 × 10^3 to 2.7 × 10^5, with capability regions cutting off around

Authors’ abstract

Quantum computing hardware is advancing rapidly toward utility-scale machines that will enable scientific breakthroughs. Many teams are pursuing distinct and difficult-to-compare routes to this goal, using different qubit technologies and logical architectures. Tracking progress toward quantum utility therefore requires rigorous benchmarks that measure computational capability relative to utility-scale challenge problems and enable fair comparison across disparate platforms. Here we demonstrate direct, cross-platform measurement of quantum computational capability using a new benchmark that quantifies the size of the largest computationally relevant quantum circuits that a machine can execute successfully and the speed at which it can execute them. We apply this quantum universal operation performance system (QUOPS) experimentally to leading processors from Quantinuum, Google, and IBM, computing directly on physical qubits. Translating state-of-the-art resource requirements for recognized challenge problems that represent useful quantum computation into effective QUOPS circuit sizes shows that computational capability must grow by 5 orders of magnitude, motivating fault-tolerant approaches. We use the same benchmark to assess the performance of a simple fault-tolerant logical-qubit processor implemented on up to eight [[7,1,3]]-encoded logical qubits using Quantinuum Helios-1, and project the growth of capability across successive generations of fault-tolerant quantum computers to show how QUOPS can track progress toward quantum scientific utility.