The OCx24 paper (arXiv:2411.11783) fits a linear model for HER and random
forests for CO₂RR on computational descriptors and reports R² under
leave-one-out (LOO) and leave-one-composition-out (LOCO) cross-validation
(Table 1; see reference.md in the ocx25-data Skill). An independent
reproduction of those numbers, and the same models run on the larger 2025
data, give every later Problem in this Campaign its baseline.
Using the fairchem applications/ocx code and data as published (the 2024
experimental, XRF and XRD dumps, the cod_matches_lookup.pkl, and the
1.5 GB comp_df_241022.csv descriptor file), reproduce:
Report each reproduced value next to the published one in
discrepancies whenever they differ by more than 0.02, with the cause if
you found it (code path, data version, random seed, the paper's trendline
R² versus the coefficient of determination). Running the authors' scripts
unchanged is the right first step; you may then re-implement to find
causes.
Derive the canonical targets from experiments and xrf with
prepare_targets.py, compute the same features for every target row
(Matminer on the XRF composition; the paper's aggregated adsorption
energies from the descriptor file, matched to the nearest computational
composition the way the paper does), and fit the same model families with
the same settings. Score every model with evaluate.py under LOCO (and
LOO) and list each as a models entry with is_baseline: true. These are
the Campaign baselines for her-model and co2rr-model.
Note any 2025 target row for which no descriptor exists (new elements Mn, Mo, Nb, W may lack coverage) and say how you handled it; do not drop rows silently.
document: a JSON file satisfying document_schema (problem is
reproduce-ocx24-baselines; both parts' models in models;
discrepancies for Part A; targets with the SHA-256 of your two
target files; leakage_audit can be short here but must state that no
outcome of a held-out element system entered a fold).report: Markdown, under 3,000 words: method, the comparison table,
discrepancies and causes, what the 2025 baselines are and what changed
since 2024.code: an archive that re-runs both parts from the bound inputs and the
listed external downloads and regenerates every predictions table.The result document is schema-checked, then a campaign reviewer re-runs
the code and re-scores your predictions tables with evaluate.py. The
Submission is verified when every reproduced number matches your document,
every discrepancy against the paper is explained or explicitly left open,
and the 2025 baselines are reported for every model listed above under
LOCO. Bound inputs: experiments, xrf, dashboard, document_schema.