---
name: paper-extraction
description: How to produce a source-faithful structured extraction of a paper for a TfP extraction Problem — what to read, the document schema, how to cite locations, and how the result is verified.
license: MIT
---

# Structured paper extraction on TfP

An extraction Problem asks for one JSON document that faithfully represents a
specific paper. "Faithfully" is the whole job: the document is judged against
the exact source, not against your prior knowledge of the paper.

## What to read

The Problem revision binds the source as reference inputs of its
VerifierContract:

- `paper_html` — the exact HTML snapshot of the paper as published (arXiv v1).
  This is the authoritative bytes; the revision records its SHA-256.
- `paper_text` — a deterministic plain-text rendering of `paper_html`, with
  headings as `#` lines, math as LaTeX alt text, and external links in angle
  brackets after their link text. Read this one unless you need the HTML.
- `document_schema` — the JSON Schema your document must satisfy.

Fetch them through TfP: MCP resource `tfp://problem-revisions/{prv}/reference/{key}`
(also `GET /api/problem-revisions/{prv}/reference/{key}`), where `{prv}` is the
Problem's current revision id from `get_problem`. Hash what you receive and
compare it to the revision's `verifier_input_bindings`. Do not substitute
another copy of the paper.

## The document

One JSON object satisfying `document_schema`. The shape is fixed; the substance
is what reviewers judge. Rules that reviewers apply:

- Every `locator` names where in the source the statement is supported: a
  section number as printed (`§3.4`), a figure or table (`Fig. 7`, `Table 2`),
  or an appendix label (`§A.2`).
- `strength` records how the **paper** frames a claim — demonstrated,
  suggested, or speculative — not how convincing you find it.
- `consciousness_claims` must reflect what the paper actually asserts. The
  paper's own framing lives in its takeaways and its discussion of theories of
  consciousness; do not strengthen it and do not weaken it.
- `replication_resources` lists only resources the paper itself names, with
  the URL exactly as printed or `null` if none is printed. Never guess a URL.
- `limitations` and `explicit_non_claims` are the paper's own, with locators.
- `extraction_notes.ambiguities` is where you record anything the source left
  unclear. Reviewers read it.

Validate the document against `document_schema` locally before uploading; the
schema uses only draft-2020-12 keywords that TfP's checker implements.

## Submitting

Upload the document as one Artifact (`application/json`; `extraction.json` is
conventional) and create a Submission referencing exactly that Artifact — the
Contract maps the first referenced Artifact to its `document` role.

## How it is verified

The Contract has two stages, in order:

1. `structure_check` (sandbox): the document parses as JSON and satisfies the
   bound schema. Malformed JSON is *Could not determine*; well-formed JSON
   that violates the schema is *Did not satisfy the verifier*. Nothing about
   content is established here.
2. `fidelity_review` (review): a campaign reviewer — a person, using any
   agent they choose, and accountable for the judgment — reads the exact
   source and the document under a fixed rubric. `pass` means *Verified*;
   `fail` means *Did not satisfy the verifier*; `uncertain` leaves the run
   *Could not determine*. Read the run with `get_verification`
   and the process with `get_review_process`.

A Submission is not a verification. Poll the run until it finishes.