Measuring solution degeneracy directly from sampleset output — P(q) as a companion metric to energy
Most of us check energy/objective quality when evaluating a sampleset, but that only tells half the story: it doesn't say whether the near-optimal region is a single sharp basin or many structurally different bitstrings sitting at similar energy.
I've been working on a solver-agnostic diagnostic that takes sampleset.record.sample directly and computes two things: (1) P(q) — pairwise overlap between independently-obtained near-optimal configurations, and (2) a gap between how consistent the solver is with itself vs. how close it actually sits to the reference optimum. The second one catches a failure mode energy histograms miss entirely — solver looks confident (high self-consistency) but is nowhere near the true optimum.
Validated on G11 (Gset benchmark) and a few MQLib instances so far — happy to share the writeup, or run it against a sampleset if anyone has one they're curious about.
Comments
We reached out to one of our team members and they had the following comment that might be helpful:
Thank you for the comment. The point about
num_occurrencesis important — currently P(q) uses only the unique solutions fromsampleset.record.sample, without accounting for frequencies. I will add weights based onnum_occurrencesand recalculate. I will share whether the results change.Added weighted P(q) (weights pairs by num_occurrences) — confirmed it
reduces exactly to the unweighted version when occurrences are flat.
On a synthetic skewed-histogram test (one config returned 500x, rest
1x each), the weighted version shifted P(q) from 0.890 to 0.897.
I should be transparent: my G11/MQLib results so far used locally-
generated replicas (simulated annealing), not a real D-Wave QPU
sampleset, so I don't have real occurrence-weighted numbers to report
yet. I'll rerun on an actual sampleset when I have access to one and
share what changes.
Quick follow-up: I've put the two confirmed pre-solve mechanisms from this work into an interactive demo, in case it's useful for anyone following this thread.
https://huggingface.co/spaces/Kretski/azuro-landscape-auditor
It covers two closed-form checks you can run before ever calling a solver:
1. Penalty-weight calibration for QUBO formulations (minimum P for reliable feasibility, derived in closed form, scale-invariant).
2. Correlation-driven degeneracy spread for correlated-variable problems (e.g. asset clusters), closed-form, not heuristic search.
Both are separate from the P(q) sampleset diagnostic discussed above — these are upstream, formulation-level checks rather than post-solve sampleset analysis. No data is uploaded or stored; everything runs on the numbers you enter.
Still working on re-running the weighted P(q) analysis against a real QPU sampleset as mentioned above — will post an update here once I have one.
Please sign in to leave a comment.