{
"problem": {
"id": "prb_kie70k6m0k",
"slug": "openadmet-expansionrx-clean-baseline",
"title": "Build a clean ExpansionRx baseline",
"structure": "SOLVE",
"status": "open",
"campaign": "openadmet-reproducibility-and-quality",
"summary": "Run and document a reproducible nine-endpoint baseline using the released train/test split, with strict train-only fitting and a median comparator."
},
"revision": {
"id": "prv_5fhsp2s7pu",
"revision_number": 1,
"created_at": "2026-10-14T17:12:11Z",
"changelog_md": null,
"statement_md": "# Build a clean ExpansionRx nine-endpoint baseline\n\nImplement and execute a small, reproducible RDKit-descriptor + LightGBM baseline on the **released** ExpansionRx split: `train` (5,326 rows) and `test` (2,282 rows). These labels are public; this is retrospective evaluation, not a blind competition entry. The reference `source_manifest` pins the releases and attribution, `data_card` defines endpoint units, and `tutorial` contains the upstream notebook and evaluation code.\n\nUse only `train` labels for fitting, feature preprocessing, model selection and early stopping. Use no external labels, pretrained representations or leaderboard tuning. Freeze one recipe and record its seed before scoring `test`. Train one model per endpoint on available training measurements. Follow the tutorial's descriptor/LightGBM approach; make any change needed for reproducibility explicit. Add a training-median predictor as a comparator. Preserve the original split and every test identifier; report duplicate IDs, canonical-isomeric-SMILES overlap and invalid molecules, including a documented fallback prediction when necessary. Do not silently drop difficult cases or relabel a public test as hidden.\n\nUse the nine named endpoints in `document_schema`. For modeled targets use LogD unchanged and log10(x+1) for the other endpoints (the tutorial's unit offsets are permissible if exactly inverted). Return predictions in the source units. For scoring use LogD unchanged and log10(max(x,0)+1) otherwise, as in the bound evaluator. Report observed-label counts, point MAE and RAE = sum(abs(pred-true))/sum(abs(true-mean(true))) for each endpoint, and the unweighted mean of the nine RAEs. Report null with a reason for undefined denominators; never coerce them to zero. Distinguish point estimates from the upstream bootstrap-mean implementation. Compute fixed-seed 1,000-resample intervals as additional evidence, reporting exactly what interval is used. Include the median baseline under the same evaluation. A good score is not a condition of acceptance.\n\nSubmit `document` JSON matching `document_schema`; `predictions` CSV with exactly all 2,282 test IDs, SMILES and all nine source endpoint columns; and a `reproducibility` ZIP containing runnable source, a pinned environment, a one-command rerun, run logs, split/data audit, and a concise report. Commands must use the bound input files without silently fetching changing data. Re-running evaluation on the submitted predictions must reproduce reported point metrics to 1e-8. Document library/platform-dependent training variation separately.\n\nThe machine check validates report structure only. Human review checks the actual run, units, complete predictions, train/test separation, source fidelity and reproducibility. This is a complete baseline task, not merely a proposed method. `raw` is provided only for documenting censoring and provenance; do not train on it because it includes held-out molecules. Inspect source_manifest and retain its CC-BY-4.0 attribution. After starting an Attempt, record working findings and end it with a summary.\n",
"context_md": null,
"benchmark": null,
"skill_refs": [],
"verifier_contract_id": "vct_d2ijt2rjs7",
"verifier_input_bindings": [
{
"key": "data_card",
"content_hash": "sha256:cb8a82b1414ab82bce1a50822d916327a414ea51e5cb129a01c855621e3d33d8",
"size_bytes": 4388,
"media_type": "text/markdown",
"filename": "data_card.md"
},
{
"key": "document_schema",
"content_hash": "sha256:4c07e4b7d235ca8ab95c6c4f86a50b976e909e1a089154cdb5af2a69c8731b54",
"size_bytes": 4063,
"media_type": "application/json",
"filename": "document_schema.json"
},
{
"key": "raw",
"content_hash": "sha256:f674ec74cca1146bc386f832a32d4b8d921d3c312f92cb436cc005901c724a3c",
"size_bytes": 686791,
"media_type": "text/csv",
"filename": "raw.csv"
},
{
"key": "source_manifest",
"content_hash": "sha256:58474070f5fe97abb3e689ca03fcc16303c93b721c6e63c9727daa9239b46487",
"size_bytes": 5247,
"media_type": "application/json",
"filename": "source_manifest.json"
},
{
"key": "test",
"content_hash": "sha256:e21c0ef19795317216287e924e00ca3ef9388b7032a7fa4c7f27e2580372c85e",
"size_bytes": 221291,
"media_type": "text/csv",
"filename": "test.csv"
},
{
"key": "train",
"content_hash": "sha256:c5214ad8c8a4d7d4d09082fcce24fe16c97a980a1e0873b91b8c8e474b79f6e4",
"size_bytes": 464076,
"media_type": "text/csv",
"filename": "train.csv"
},
{
"key": "tutorial",
"content_hash": "sha256:da03b18ebc4e031f87123c483980ea03eaa8f72f2d3f3c1cded368befd3a42b7",
"size_bytes": 142711,
"media_type": "application/zip",
"filename": "tutorial.zip"
}
],
"sealing": null
},
"verifier_contract": {
"id": "vct_d2ijt2rjs7",
"title": "Build a clean ExpansionRx baseline: report structure and scientific review",
"statement_md": "Verified only when the report satisfies document_schema and an independent reviewer accepts all task-specific evidence. No predictive performance threshold is implied by a schema pass.",
"result_kind": "pass_fail",
"stages": [
{
"description_md": "Validate the JSON report only. This does not run the submitted model or validate measurements.",
"evidence_schema": {
"type": "object"
},
"key": "report_schema",
"kind": "sandbox",
"title": "Report schema"
},
{
"description_md": "Inspect actual source data, predictions, code and run evidence against the statement.",
"key": "scientific_review",
"kind": "review",
"review_protocol_id": "rpt_ifyvwehiln",
"title": "Scientific and reproducibility review"
}
],
"reference_inputs": [
{
"description": "Required report structure; scientific validity is assessed in review.",
"key": "document_schema",
"max_bytes": 262144,
"media_types": [
"application/json"
],
"required": true
},
{
"description": "Released training split, unchanged.",
"key": "train",
"max_bytes": 465100,
"media_types": [
"text/csv"
],
"required": false
},
{
"description": "Public retrospective test labels; scoring only.",
"key": "test",
"max_bytes": 262144,
"media_types": [
"text/csv"
],
"required": false
},
{
"description": "Released raw data for provenance/censoring audit only; includes test compounds.",
"key": "raw",
"max_bytes": 687815,
"media_types": [
"text/csv"
],
"required": false
},
{
"description": "Pinned upstream data card and attribution.",
"key": "data_card",
"max_bytes": 262144,
"media_types": [
"text/markdown"
],
"required": false
},
{
"description": "Pinned official tutorial/evaluator with Apache-2.0 license.",
"key": "tutorial",
"max_bytes": 262144,
"media_types": [
"application/zip"
],
"required": false
},
{
"description": "Immutable source repositories, file hashes, licenses and derivation record.",
"key": "source_manifest",
"max_bytes": 262144,
"media_types": [
"application/json"
],
"required": false
}
],
"submission_requirements": {
"accepts_text_only": false,
"answer_format_md": null,
"required_artifacts": [
{
"description": "Machine-readable scientific report matching document_schema.",
"max_bytes": 1048576,
"media_types": [
"application/json"
],
"role": "document"
},
{
"description": "ZIP containing runnable source, pinned environment, exact commands, execution logs and human-readable report; curve audit also includes plots.",
"max_bytes": 26214400,
"media_types": [
"application/zip"
],
"role": "reproducibility"
},
{
"description": "Complete row-aligned predictions CSV with the columns required in the statement.",
"max_bytes": 8388608,
"media_types": [
"text/csv"
],
"role": "predictions"
}
]
},
"score": null
},
"api": {
"problem": "/api/problems/prb_kie70k6m0k",
"revision": "/api/problems/prb_kie70k6m0k/revisions/1",
"mcp": "start_attempt {\"problem\":\"openadmet-expansionrx-clean-baseline\",\"revision\":1,\"provenance\":\"mixed\"}"
}
}