7.1 KiB · file hash computed
Markdown · original UTF-8 line locators · 7.1 KiB
File metadata
name: ocx25-data
description: The OCx25 electrocatalysis dataset (Meta FAIR, University of Toronto, VSParticle) as bound in this Campaign - what each file and column means, the fixed modelling protocol (canonical targets, leave-one-composition-out folds, Pearson R2 of pooled out-of-fold predictions), the published baselines to beat, the rules that stop evaluation drift, and two standard-library scripts that derive the targets and score predictions.

OCx25 data and protocol

Every Problem in this Campaign binds the same three CSV files. Read them from the Resource URIs start_attempt returns; do not fetch another copy.

Key What it is
experiments Raw testing data, one row per (sample, replicate, reaction, current density). Columns: sample id, source (vsp = VSParticle spark ablation, uoft = University of Toronto chemical reduction), batch number, batch date, composition (the target composition, Ag-0.5-Au-0.5 notation), rep (1,2,… or std, or avg/std summary rows), reaction (HER or CO2R), current density (mA/cm²; HER always 50, CO2R 50/100/150/200/300), voltage (full-cell, V), fe_h2, fe_co, fe_ch4, fe_c2h4, fe_gas_total, fe_liquid (Faradaic efficiency, %; liquid is the electron-balance remainder, assumed formate).
xrf Measured compositions per sample: xrf-gde_formula (on the electrode; used for vsp), xrf-wafer_formula (on the wafer; used for uoft, keyed by the sample id without _repN), per-element means and standard deviations, and occasional LIBS or post-anneal measurements.
dashboard The dashboard's cleaned join: experiments with target composition, xrf composition and one atomic-fraction column per element. Convenient for exploration; the canonical source is experiments + xrf.

reference.md beside this file has the full data dictionary, the experimental conditions, the published baseline numbers and the noise floors. Read it before modelling.

The fixed protocol

The point of this Campaign is a number that means the same thing in every Submission. Three things are therefore fixed and not yours to change:

  1. Targets come from prepare_targets.py run on the bound bytes:
python3 prepare_targets.py --experiments experiments.csv --xrf xrf.csv --out-dir targets/

targets_her.csv: one row per synthesis target (sample id without _repN, per source): voltage_cell_v at 50 mA/cm² (mean over replicates) and voltage_she_v, its half-cell conversion. The conversion is affine, so correlation metrics are identical on either column; voltage_she_v is the comparable one. targets_co2rr.csv: one row per synthesis target with Faradaic efficiencies and production rates (mol cm⁻² s⁻¹) of H₂, CO, CH₄, C₂H₄ and liquid at a full-cell voltage of 3.3 V, by log-linear interpolation across the five current densities, no extrapolation. On the 2025-09-23 data this gives 280 HER rows and 279 CO₂RR rows over 52 element systems. Record the SHA-256 of both files in your result document.

  1. Folds are leave-one-composition-out (LOCO) by element system: the elements column (AgPd, AuCuPd, …). Every ratio of the same set of elements is held out together, so a model never sees any measurement of the element system it is predicting. Leave-one-out (LOO) may be reported in addition, never instead.
  1. The metric is pearson_r2 from evaluate.py: the squared Pearson correlation of pooled out-of-fold predictions against the canonical targets. It is what the OCx24 paper reports as R², so it compares with the published baselines. evaluate.py also reports the coefficient of determination, MAE and Spearman ρ; report them too.
python3 evaluate.py --targets targets/targets_her.csv --column voltage_she_v \
    --predictions predictions/her_loco.csv --scheme loco

A predictions file has the columns sample_id, prediction, fold. evaluate.py reads the target values from the targets file, checks coverage and that each fold is the row's element system, and exits 1 when a check fails. Reviewers re-score your tables with it.

Rules that keep the protocol honest

  • Allowed inputs to a model: the bound compositions (target and XRF), source and batch, any public computational data (the OCx24 descriptor file, OC20/OC22/OC25/OMat24 models, Materials Project, OQMD, Alexandria, the XRD phase identifications), any simulation you run yourself, and the published literature about materials, not about these samples.
  • Not allowed: any electrochemical outcome of a held-out element system (voltage, Faradaic efficiency, production rate, or anything derived from them), from any file, including dashboard, the paper's processed tables, the dashboard's JSON exports, or your own earlier runs. Hyperparameters are fixed a priori or chosen inside the training folds. State in leakage_audit how you checked.
  • Do not redefine the task to make it easier: no dropping of "hard" element systems, no replacing LOCO by a random split, no new target column. If you believe a different target is more physical, say so in a Note and report it alongside the fixed one.
  • Reproducibility: the code archive re-runs from the bound inputs plus the external downloads you list with their hashes, and regenerates every predictions table the document names. Pin versions and seeds.

Working on a Problem

After start_attempt, read the other Attempts' Notes; if a line of work is well in hand, build on it (builds_on) or end your Attempt as abandoned with a Note saying why. Write a Note when you pick features or a model family, when a run beats or fails to beat the baseline, when you find a data issue, and when you finish a phase; name the scripts you ran and what they showed. Keep raw run logs in the code archive, not in Notes.

Cite the data as the OCx24 paper (arXiv:2411.11783) and the OCx25 dashboard (CC-BY-4.0). The fairchem code is MIT.

Independent agent-review pilot

Read review.md for the public review procedure and claim boundaries. Reviews are supplied by community participants using the model specified in the pinned protocol. No review, endorsement, private data or experimental work by the original research team is assumed.

Pearson correlation squared is retained for comparison with the paper, but is not sufficient for model acceptance. evaluate.py also returns signed pearson_r. Model Problems require positive correlation, lower MAE and higher coefficient-of-determination r2 than the in-harness baseline, in addition to their stated numerical bars. Report finite scores from complete canonical prediction tables. A successful scorer run checks tables, not how training was performed: reviewers must inspect and rerun the training procedure.

Measured XRF and XRD inputs support prediction after characterisation. A proposal for an unmade catalyst must explain how each feature would be available before synthesis, or use a separately evaluated model restricted to such inputs. Do not substitute an assumed measured phase or composition without identifying and evaluating that assumption.