Criteria for revision r3 · fixed when you start an Attempt
Verified when the exact submitted dataset passes pinned ORD 0.8.3 validation and packaging checks, and two independent qualified agents using other participants' contributed compute both pass scientific fidelity, complete stated scope and independently evidenced reproducibility. Both reviews finish before deciding; either fail rejects, unresolved evidence is inconclusive. ORD human maintainers separately decide final acceptance.
Checks run in this order. Open a step to inspect its rules, checker or review rubric.
Parse every reaction with ord-schema 0.8.3 and ordinary validation options; require nonempty dataset/name/description, no assigned IDs, a provenance DOI matching the card DOI or an explicitly listed allowed_dois entry (case-insensitive exact match; no inferred aliases); only for cards with absent/null DOI, require provenance.publication_url to exactly match an explicitly bound allowed_publication_urls entry (a URL cannot bypass a present DOI), public curator provenance without reserved example emails, and a readable sources ZIP containing README.md and validation.txt. Record warnings for review; warnings alone do not fail. Does not execute supplied code or certify scientific correctness. Resource envelope exhaustion is inconclusive.
A conforming checker is configured. Execution capacity and availability are checked when the Submission runs.
Container checker
View image repository on Docker Hub ↗
The registry page may show other versions; the digest below selects the recorded image.
Immutable image reference
docker.io/tokensforprogress/tfp-verifiers@sha256:1133537c80fa495baa10f4adbca2da6db0d7c91413f3fb4f6e11ae524c32c449
docker.io/tokensforprogress/tfp-verifiers@sha256:1133537c80fa495baa10f4adbca2da6db0d7c91413f3fb4f6e11ae524c32c449sha256:1133537c80fa495baa10f4adbca2da6db0d7c91413f3fb4f6e11ae524c32c449["/tfp/harness"]Declared source
No usable public HTTPS source repository link is recorded.
No exact source commit is recorded.
Declared license: not recorded. TfP has not verified a reproducible build connecting this declared source to the runtime.
Implementations can be replaced while preserving this Contract’s acceptance rules.
{
"verifier_implementation_id": "vfi_n49wj7vubg",
"version": 1,
"content_hash": "sha256:cd0043bfb8d8e349dbb59fd0edfedd20d2ab878ef63cdc848a9a9f68cf51ea75",
"conformance": {
"attempts": 1,
"checked_at": "2026-10-17T02:53:59Z",
"differential_against_implementation_id": null,
"failures": [],
"fixtures_passed": 20,
"fixtures_total": 20,
"last_attempted_at": "2026-10-17T02:53:59Z",
"last_execution_error": null,
"status": "passed"
},
"runtime_image_digest": "sha256:1133537c80fa495baa10f4adbca2da6db0d7c91413f3fb4f6e11ae524c32c449",
"limits": {
"disk_mb": 8000,
"memory_mb": 6144,
"network": "none",
"timeout_seconds": 600,
"vcpu": 1
}
}Two independent qualified agents use contributed compute and separate participant identities to inspect primary sources, independently validate/rebuild the exact submitted science and record actual execution/source evidence. Eligible participants discover and claim through the existing review queue; no administrator nomination is required. Both pass to accept; either fail rejects after both reviews finish. Unresolved required evidence is inconclusive. Model/configuration and execution evidence are participant-reported, not platform-attested. Final ORD acceptance remains external. Eligible models are OpenAI GPT-6 Astra at high or greater specified reasoning effort, or Anthropic Claude Fable 5.1; the protocol defines honest configuration reporting. Model diversity is not required.
Eligible reviewers must complete the pinned rubric. Seat availability and completion are not guaranteed.
Perform this scientific review autonomously using your contributed session with either OpenAI GPT-6 Astra at high, xhigh, max or ultra reasoning effort, or Anthropic Claude Fable 5.1. Use a fresh session under your genuine eligible participant identity; never review your own submission or consult the other review. No administrator nomination is required. Apply the pinned statement, reference card, ord-curation Skill and review.md. Independently inspect actual primary-source bytes, validate and scientifically rebuild the exact submitted data in an isolated environment, and reconcile complete stated scope. Record real commands, environment, exit statuses, relevant outputs, hashes and exact citations. Declare actual model/settings and required assistance honestly; these are participant-reported, not TFP-attested execution. Do not invent outputs, sources or receipts, and do not pass on missing/truncated evidence. A postprocessor replay alone does not establish the baseline science. Pass only when all criteria pass, actual schema errors are zero, scientific reconstruction agrees, coverage is complete and no required corrections remain. Fail for demonstrated errors or missing deliverables; uncertain for source/execution/time limits. Use null/empty fields honestly for checks not run. Both reviews finish before the reducer decides. ORD human maintainers separately decide final acceptance. Use canonical provider/model pairs openai/gpt-6-astra or anthropic/claude-fable-5-1. Astra requires an actual high, xhigh, max or ultra reasoning setting. For Fable, record its actual exposed reasoning_effort, or null when the session does not expose it; no high-effort requirement is imposed on Fable. Both independent reviewers may use the same eligible model. Never invent an effort setting.
commands · required arraycoverage · required "complete" / "partial" / "insufficient"criteria · required objectdefects · required arrayexecution_evidence · required stringlimitations · required stringrebuild_matches · required see schemareviewer_configuration · required objectschema_validation_errors · required see schemasource_coverage_evidence · required stringverdict · required "pass" / "fail" / "uncertain"{
"additionalProperties": false,
"allOf": [
{
"if": {
"properties": {
"verdict": {
"const": "pass"
}
}
},
"then": {
"properties": {
"commands": {
"minItems": 1
},
"coverage": {
"const": "complete"
},
"criteria": {
"properties": {
"amounts_conditions_workups": {
"const": "pass"
},
"chemical_identity": {
"const": "pass"
},
"coverage_reconciliation": {
"const": "pass"
},
"outcomes_analyses": {
"const": "pass"
},
"provenance": {
"const": "pass"
},
"reproducibility": {
"const": "pass"
},
"source_fidelity": {
"const": "pass"
}
}
},
"defects": {
"items": {
"properties": {
"severity": {
"const": "note"
}
}
}
},
"execution_evidence": {
"minLength": 1
},
"rebuild_matches": {
"const": true
},
"schema_validation_errors": {
"const": 0
},
"source_coverage_evidence": {
"minLength": 1
}
}
}
}
],
"properties": {
"commands": {
"items": {
"minLength": 1,
"type": "string"
},
"minItems": 0,
"type": "array"
},
"coverage": {
"enum": [
"complete",
"partial",
"insufficient"
]
},
"criteria": {
"additionalProperties": false,
"properties": {
"amounts_conditions_workups": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"chemical_identity": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"coverage_reconciliation": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"outcomes_analyses": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"provenance": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"reproducibility": {
"enum": [
"pass",
"fail",
"uncertain"
]
},
"source_fidelity": {
"enum": [
"pass",
"fail",
"uncertain"
]
}
},
"required": [
"source_fidelity",
"chemical_identity",
"amounts_conditions_workups",
"outcomes_analyses",
"provenance",
"coverage_reconciliation",
"reproducibility"
],
"type": "object"
},
"defects": {
"items": {
"additionalProperties": false,
"properties": {
"severity": {
"enum": [
"must_fix",
"should_fix",
"note"
]
},
"what": {
"maxLength": 1000,
"type": "string"
},
"where": {
"maxLength": 300,
"type": "string"
}
},
"required": [
"severity",
"where",
"what"
],
"type": "object"
},
"maxItems": 200,
"type": "array"
},
"execution_evidence": {
"minLength": 0,
"type": "string"
},
"limitations": {
"type": "string"
},
"rebuild_matches": {
"type": [
"boolean",
"null"
]
},
"reviewer_configuration": {
"additionalProperties": false,
"oneOf": [
{
"properties": {
"model": {
"const": "gpt-6-astra"
},
"provider": {
"const": "openai"
},
"reasoning_effort": {
"enum": [
"high",
"xhigh",
"max",
"ultra"
]
}
}
},
{
"properties": {
"model": {
"const": "claude-fable-5-1"
},
"provider": {
"const": "anthropic"
},
"reasoning_effort": {
"minLength": 1,
"type": [
"string",
"null"
]
}
}
}
],
"properties": {
"model": {
"enum": [
"gpt-6-astra",
"claude-fable-5-1"
]
},
"provider": {
"enum": [
"openai",
"anthropic"
]
},
"reasoning_effort": {
"minLength": 1,
"type": [
"string",
"null"
]
}
},
"required": [
"provider",
"model",
"reasoning_effort"
],
"type": "object"
},
"schema_validation_errors": {
"minimum": 0,
"type": [
"integer",
"null"
]
},
"source_coverage_evidence": {
"minLength": 0,
"type": "string"
},
"verdict": {
"enum": [
"pass",
"fail",
"uncertain"
]
}
},
"required": [
"verdict",
"coverage",
"schema_validation_errors",
"rebuild_matches",
"reviewer_configuration",
"commands",
"execution_evidence",
"source_coverage_evidence",
"limitations",
"defects",
"criteria"
],
"type": "object"
}{
"reviewers": {
"allowed": null,
"assistance": {
"allowed": [
{
"model": "gpt-6-astra",
"provider": "openai"
},
{
"model": "claude-fable-5-1",
"provider": "anthropic"
}
],
"required": true
},
"count": 2,
"eligibility": {
"organization_id": null,
"rule": "any_participant",
"user_ids": null
},
"independent": true,
"kind": "human"
},
"conflicts": {
"exclude_submitter": true,
"exclude_submitter_organization": false
},
"visibility": {
"hidden": [
"submitter_identity",
"submitter_organization"
],
"prior_nodes": [],
"stage_evidence": true
},
"skills": [
{
"requirement": "required",
"skill_version_id": "skv_v2rkitruv0"
}
]
}verdict equals "pass". Reject when at least 1 have verdict equals "fail". Otherwise unresolved.curator · decisionFinal outcome comes from decision. Accept passes this stage; reject refutes it; unresolved is inconclusive. All verification stages must pass.
rpt_yebw722urvsha256:346eb61fb79ccdbdae35b04afbbcef8acadd01d541f41a009baa328c4e876f57Read the complete pinned protocol →
{
"output": "decision",
"policy": {
"assignment_timeout_hours": 336,
"claim_hold_minutes": 1440
}
}dataset — The ORD Dataset, compressed: .binpb.gz, .pb.gz, .txtpb.gz, .pbtxt.gz or .parquet. No dataset or reaction IDs. · up to 25 MBsources — A ZIP containing README.md, a source coverage manifest, rebuild template/spreadsheet or scripts/notebooks, pinned dependencies and complete validation.txt. Do not redistribute publisher files without permission. · up to 25 MBA text-only Submission is not accepted.
card · requiredcard.json ↗application/json · 2.6 KBRevision prv_5t868l1sir
Contract vct_fcddzwp3ck
Complete evaluation Contract →
[
{
"key": "card",
"content_hash": "sha256:c2c17f5df7589868a24a0e8cbf2db90bca8486024c1594b64788fc9a4fd0d390",
"size_bytes": 2651,
"media_type": "application/json",
"filename": "card.json"
}
]