What we learned from 45 experiments on IBM Heron
The most useful part of the QANTIS hardware campaign was not one impressive number. It was learning when a result changed and why.
The first encouraging result is the one you want to explain. The awkward result is the one that asks whether your explanation was good enough. Running the QANTIS experiments across three IBM Heron devices brought that question into focus: when the result changes, are we seeing the method, the hardware, or the conditions under which they met?
The foundational QANTIS paper is 31 pages long, with 4 figures and 12 tables. Behind them are 45 experiments across three IBM Heron backends. It is a lot of material for one paper. Much of its value came from the runs that stopped the first encouraging result from becoming our whole explanation.
The experiments followed two kinds of calculation: updating a probability distribution as observations arrive, and associating observations with multiple targets. Those jobs led to the Grover and BIQAE, QUBO and FPC-QAOA methods described in the paper. Looking across the runs also let us examine error mitigation: whether a change intended to improve the result actually did so in that setting.
Similar hardware did not behave identically
The three Heron processors were not interchangeable in practice. A difference that looked like a method effect on one backend could turn out to be related to calibration on another. Recording the actual backend and calibration window became essential to explaining a run.
Error mitigation had its own surprises. Techniques that helped separately did not always help in combination. Trying several configurations mattered because an apparently sensible improvement could make the final result worse.
The good number and the harder runs
In the reported rare-observation case, probability rose from 0.179 to 0.907 and the Hellinger distance from exact Bayes was 0.0015. It is a useful result under the stated conditions. The harder runs helped us understand the range around it: where calibration or configuration began to change the picture.
The foundational work also included an initial closed-loop Tiger example. The sequential paper takes repeated belief updates as its main question and compares amplification choices along matched trajectories. Its 8-step and 12-step campaigns, with 20-step and 32-step controls, examine that operating range in more detail.
Plan to learn even when it does not work
If I were starting again, I would keep the experiments that can teach us something when the method fails. They use queue time too, but they stop a campaign from becoming a search only for the result we wanted. They also give the next researcher a clearer place to begin.
Both QANTIS papers are public arXiv preprints and have not been peer-reviewed. Neither claims a wall-clock advantage over strong classical methods. What this campaign gives us is a more detailed account of a mechanism and the conditions that shape it.
- The hardware and calibration
- The method configuration
- The result and where it changes