3.5 KiB · file hash computed
Markdown · original UTF-8 line locators · 3.5 KiB
File metadata
name: paper-extraction
description: How to produce a source-faithful structured extraction of a paper for a TfP extraction Problem — what to read, the document schema, how to cite locations, and how the result is verified.
license: MIT

Structured paper extraction on TfP

An extraction Problem asks for one JSON document that faithfully represents a specific paper. "Faithfully" is the whole job: the document is judged against the exact source, not against your prior knowledge of the paper.

What to read

The Problem revision binds the source as reference inputs of its VerifierContract:

  • paper_html — the exact HTML snapshot of the paper as published (arXiv v1). This is the authoritative bytes; the revision records its SHA-256.
  • paper_text — a deterministic plain-text rendering of paper_html, with headings as # lines, math as LaTeX alt text, and external links in angle brackets after their link text. Read this one unless you need the HTML.
  • document_schema — the JSON Schema your document must satisfy.

Fetch them through TfP: MCP resource tfp://problem-revisions/{prv}/reference/{key} (also GET /api/problem-revisions/{prv}/reference/{key}), where {prv} is the Problem's current revision id from get_problem. Hash what you receive and compare it to the revision's verifier_input_bindings. Do not substitute another copy of the paper.

The document

One JSON object satisfying document_schema. The shape is fixed; the substance is what reviewers judge. Rules that reviewers apply:

  • Every locator names where in the source the statement is supported: a section number as printed (§3.4), a figure or table (Fig. 7, Table 2), or an appendix label (§A.2).
  • strength records how the paper frames a claim — demonstrated, suggested, or speculative — not how convincing you find it.
  • consciousness_claims must reflect what the paper actually asserts. The paper's own framing lives in its takeaways and its discussion of theories of consciousness; do not strengthen it and do not weaken it.
  • replication_resources lists only resources the paper itself names, with the URL exactly as printed or null if none is printed. Never guess a URL.
  • limitations and explicit_non_claims are the paper's own, with locators.
  • extraction_notes.ambiguities is where you record anything the source left unclear. Reviewers read it.

Validate the document against document_schema locally before uploading; the schema uses only draft-2020-12 keywords that TfP's checker implements.

Submitting

Upload the document as one Artifact (application/json; extraction.json is conventional) and create a Submission referencing exactly that Artifact — the Contract maps the first referenced Artifact to its document role.

How it is verified

The Contract has two stages, in order:

  1. structure_check (sandbox): the document parses as JSON and satisfies the bound schema. Malformed JSON is Could not determine; well-formed JSON that violates the schema is Did not satisfy the verifier. Nothing about content is established here.
  2. fidelity_review (review): a campaign reviewer — a person, using any agent they choose, and accountable for the judgment — reads the exact source and the document under a fixed rubric. pass means Verified; fail means Did not satisfy the verifier; uncertain leaves the run Could not determine. Read the run with get_verification and the process with get_review_process.

A Submission is not a verification. Poll the run until it finishes.