Quantum dynamics under a compute budget: a public fixed-suite benchmark

Submit a Python simulation method and measure how accurately it predicts quantum spin dynamics within a fixed compute budget. Each problem supplies the same public test instances to every submission, numerical reference traces, and runnable baseline methods. The score measures error on those instances; lower is better.

What is measured

This public development benchmark compares simulation methods on identical, revision-pinned instances. The three randomized Hamiltonian families each have ten fixed seeds; the clean domain-wall family has one deterministic instance. The primary score is the arithmetic mean of each scored instance's maximum absolute site/time error. Individual errors, worst-instance error and a separate 20-site probe error are reported. Scores are stored without intentional decimal rounding.

References and compute budgets

Numerical full-Hilbert-space reference traces are computed in advance using sparse matrix-exponential propagation, checked against independent dense diagonalization at six sites and a refined time grid on one scored instance, and bound as public immutable inputs. They are numerical references, not symbolic exact solutions; validation uses a 1e-9 agreement tolerance. Their generation cost is outside the contestant budget and is documented with the generator.

Each submitted method receives 240 seconds for the entire scored suite, 240 seconds for one 32-site chain or 6x6 lattice runtime/sanity run, and 60 seconds for a separate 20-site accuracy probe, including imports and setup in each batch. Each batch has four configured numerical-library threads and 6144 MiB of address space; the container has four vCPUs. The checker never shortens the stated method budgets. These budgets differ from revision 1; scores across revisions are not comparable.

What verification does not establish

The suite and reference traces are public. There is no hidden test set or automatic prevention of memorization. Verification establishes numerical agreement on the listed instances and compliance with the executable gates. It does not establish generalization, method novelty, honest declarations, or useful accuracy at the largest runtime/sanity size. Returning frozen spins passes sanity gates and is included only as a trivial control. The 20-site probe provides a separate accuracy measurement, not a claim about 32 or 36 sites.

Methods and disclosure

Declare the approximation, parameters, adaptive rules, hybrid branches, random seeds, offline training/data and preprocessing costs in PRIORS and the submission summary. Adaptive and hybrid algorithms are allowed when disclosed; identical code paths are not a substitute for an honest methodological description. A scientific method comparison excludes lookup of the published reference answers, but that restriction requires source review and is not automatically certified by the checker. Offline-trained weights must fit in the single source file; offline cost is reported separately and is not part of runtime scoring.

Baselines

The pinned Skill supplies U(1) TEBD for XXZ chains, unconstrained TEBD for the longitudinal-field Ising chain, and a disconnected 2x2 cluster approximation for the lattice. These are reproducible conventional comparators, not claimed tuned optima. Frozen spins are a negative control. Baselines are measured under the same checker, instances and budgets as submissions.