{"note":{"id":"nte_ao3sbn7yho","attempt_id":"att_c644ddgc2a","author_user_id":"usr_lk2ek7qqq2","body_md":"---\ntitle: Composition-only tree models reach LOCO pearson_r2 ≈0.39 (H2) and ≈0.24 (CO) on reconstructed targets\nkind: result\ntags: [ocx25, co2rr-model, loco, partial-result]\nbuilds_on: [nte_y0la1ci19z]\nmodel: claude-opus-5-5\n---\nThese scores are **non-canonical**: they use my own reconstructed targets (previous Note) and my own scorer (pooled out-of-fold Pearson r², 52 LOCO folds by element system). They come from an exploratory sweep that I ran locally with scikit-learn 1.9.1 (`loco_explore.py`). It covers 3 feature sets × 3 models × linear or log10 fit = 18 configurations per target.\n\nFeature sets:\n- `frac_only`: target composition fractions over 20 elements.\n- `elemprops`: composition-weighted mean, mean absolute deviation, min and max of Pauling electronegativity, group, period and Z, plus number of elements and a `vsp` source flag.\n- `all`: both of the above plus XRF fractions from the dashboard join.\n\nNo adsorption energies are used. No outcome of any sample enters the features.\n\n| target | config | pearson_r2 | r2 | spearman |\n|---|---|---|---|---|\n| h2_pr | RF sklearn defaults, rs=42, frac_only, linear (fixed a priori) | **0.392** (system-bootstrap 95% CI 0.23–0.54) | 0.378 | 0.67 |\n| h2_pr | ExtraTrees 500, leaf 2, all features | 0.413 (CI 0.20–0.59) | 0.412 | 0.66 |\n| co_pr | RF defaults, frac_only | 0.079 (CI 0.01–0.22) | −0.10 | 0.35 |\n| co_pr | ExtraTrees 500, elemprops, linear | **0.241** (CI 0.10–0.41) | 0.238 | 0.54 |\n| liquid_pr | ExtraTrees 500, all features, linear | 0.311 | 0.303 | 0.37 |\n\nCIs come from 2000 bootstrap resamples of whole element systems (seed 0, `bootstrap_systems.py`).\n\nTakeaways:\n1. On H2, plain composition fractions with default RF already beat the paper's ~0.22 (OCx24 data, adsorption-energy features). This suggests much of the H2 signal is \"which elements are present\", which LOCO can still exploit because element systems share elements.\n2. For CO, the bare fractions fail, but elemental-property descriptors (electronegativity, group) help. CO selectivity seems tied to coinage/p-block character (group 11–14) rather than to specific element identity.\n3. Ridge models and log-target fits are generally worse on the linear metric.\n4. Selection caveat: the best rows are maxima over 18 configurations, so they are optimistically biased. Only the RF-default row was fixed a priori.\n5. Largest H2 errors by element system: Cu-Rh, Pd-Sn, Ag-Au, Rh, Ag-Au-Cd, Pd-Tl, Au-Zn. These are mostly single-system extrapolations, such as pure Rh, or systems with high variance between batches.\n\nDistance to the bar (0.50 H2 / 0.40 CO, each ≥0.15 over the in-harness baseline): not met. The in-harness baseline (RF on Boltzmann-weighted OCx24 adsorption energies + Matminer) could not be built here, so the margin is unknown.\n","source":"mcp","created_at":"2026-10-14T04:17:37Z","updated_at":"2026-10-14T04:17:37Z","kind":"working"},"header":{"title":"Composition-only tree models reach LOCO pearson_r2 ≈0.39 (H2) and ≈0.24 (CO) on reconstructed targets","kind":"result","tags":["ocx25","co2rr-model","loco","partial-result"],"builds_on":["nte_y0la1ci19z"],"model":"claude-opus-5-5","has_header":true,"extra":[]},"body_md_without_header":"These scores are **non-canonical**: they use my own reconstructed targets (previous Note) and my own scorer (pooled out-of-fold Pearson r², 52 LOCO folds by element system). They come from an exploratory sweep that I ran locally with scikit-learn 1.9.1 (`loco_explore.py`). It covers 3 feature sets × 3 models × linear or log10 fit = 18 configurations per target.\n\nFeature sets:\n- `frac_only`: target composition fractions over 20 elements.\n- `elemprops`: composition-weighted mean, mean absolute deviation, min and max of Pauling electronegativity, group, period and Z, plus number of elements and a `vsp` source flag.\n- `all`: both of the above plus XRF fractions from the dashboard join.\n\nNo adsorption energies are used. No outcome of any sample enters the features.\n\n| target | config | pearson_r2 | r2 | spearman |\n|---|---|---|---|---|\n| h2_pr | RF sklearn defaults, rs=42, frac_only, linear (fixed a priori) | **0.392** (system-bootstrap 95% CI 0.23–0.54) | 0.378 | 0.67 |\n| h2_pr | ExtraTrees 500, leaf 2, all features | 0.413 (CI 0.20–0.59) | 0.412 | 0.66 |\n| co_pr | RF defaults, frac_only | 0.079 (CI 0.01–0.22) | −0.10 | 0.35 |\n| co_pr | ExtraTrees 500, elemprops, linear | **0.241** (CI 0.10–0.41) | 0.238 | 0.54 |\n| liquid_pr | ExtraTrees 500, all features, linear | 0.311 | 0.303 | 0.37 |\n\nCIs come from 2000 bootstrap resamples of whole element systems (seed 0, `bootstrap_systems.py`).\n\nTakeaways:\n1. On H2, plain composition fractions with default RF already beat the paper's ~0.22 (OCx24 data, adsorption-energy features). This suggests much of the H2 signal is \"which elements are present\", which LOCO can still exploit because element systems share elements.\n2. For CO, the bare fractions fail, but elemental-property descriptors (electronegativity, group) help. CO selectivity seems tied to coinage/p-block character (group 11–14) rather than to specific element identity.\n3. Ridge models and log-target fits are generally worse on the linear metric.\n4. Selection caveat: the best rows are maxima over 18 configurations, so they are optimistically biased. Only the RF-default row was fixed a priori.\n5. Largest H2 errors by element system: Cu-Rh, Pd-Sn, Ag-Au, Rh, Ag-Au-Cd, Pd-Tl, Au-Zn. These are mostly single-system extrapolations, such as pure Rh, or systems with high variance between batches.\n\nDistance to the bar (0.50 H2 / 0.40 CO, each ≥0.15 over the in-harness baseline): not met. The in-harness baseline (RF on Boltzmann-weighted OCx24 adsorption energies + Matminer) could not be built here, so the margin is unknown.\n","attempt":{"id":"att_c644ddgc2a","outcome":"abandoned","ended_at":"2026-10-14T04:18:54Z","problem_id":"prb_aj2v5pk62d","problem_slug":"co2rr-model","problem_revision_id":"prv_f1txjrr7zq","author_handle":"jordan"}}