Text · original UTF-8 line locators · 366.2 KiB
Plain-text rendering of https://arxiv.org/html/2607.15495v1 (arXiv:2607.15495v1), produced deterministically by Tokens for Progress (scripts/render-paper-text.py) from the exact HTML snapshot bound as `paper_html`. Math is shown as LaTeX alt text; external links appear in angle brackets after their link text.
arXiv:2607.15495v1 [cs.CL] 16 Jul 2026
# Verbalizable Representations Form a Global Workspace in Language Models
Wes Gurnee*, Nicholas Sofroniew*
Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson
Jack Lindsey*†
Anthropic
* Core contributor; † Correspondence to jacklindsey@anthropic.com
###### Abstract
Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible reasoning. In this paper, we present evidence that an analogous functional distinction has emerged in large language models. Using a new interpretability technique, the Jacobian lens, we identify the representations a model is poised to verbalize at any point in its processing. These representations, which we collectively call the J-space, exhibit the functional properties characteristic of a global workspace: their contents can be reported, deliberately summoned and held, used to carry the intermediate steps of silent reasoning, and passed as arguments to arbitrary downstream computations, while automatic processing such as text parsing and routine inference proceeds without them. The J-space also has structural signatures that global workspace theory associates with conscious access: it carries coherent content only in an intermediate band of layers, holds on the order of tens of concepts at a time, and is broadcast by the model’s weights more widely than other representations. These properties make it a practical window into a model’s unspoken thinking. In alignment audits, it reveals strategic deliberation, evaluation awareness, and trained-in misaligned dispositions that never appear in the model’s outputs. We find that post-training installs the Assistant’s point of view in the workspace, and we introduce counterfactual reflection training, which improves behavior by training only what a model would say if interrupted and asked to reflect. These results indicate that language models maintain a small, privileged set of representations bearing some of the functional hallmarks of conscious access, and that decoding these representations sheds light on ongoing cognitive processes.
## 1 Introduction
If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of processing takes place without our knowledge: our visual systems parsing the contours of a face, our motor circuits maintaining our posture. At any given moment, only a small fraction of this neural activity is accessible to us. Yet it is this privileged sliver of activity that we rely on to reason deliberately: to plan what ingredients to buy for a recipe, or to puzzle out why an engine won’t start. Such thoughts can be articulated out loud, deliberately held in mind, and brought to bear on whatever task the moment demands. This distinction, between our accessible thoughts and our unconscious processing, is perhaps the most striking feature of human cognition.
In this paper, we present evidence that an analogous functional distinction has emerged in modern AI models. Specifically, we observe that language models maintain a privileged set of internal representations, available for report, modulation, and flexible internal reasoning, atop a much larger volume of automatic processing. We identify these representations using a new interpretability technique, which surfaces the concepts a model is poised to verbalize at any point in its processing. Measuring and intervening on these representations provides us a window into a model’s thought processes, uncovering internal reasoning and reactions that do not appear in its output.
### 1.1 Motivation: conscious access and the global workspace
The phenomenon described above is sometimes referred to as access consciousness: out of everything the brain processes, only a subset is consciously accessible, in the sense of being poised for use in reasoning and in the direct control of action and speech [18, 24]. Note that access consciousness is a purely functional notion; the relationship that it has with subjective experience (sometimes called phenomenal consciousness) is widely debated. In this paper, we take no position on this issue, and instead focus on the functional role played by consciously accessible information. How is it represented or processed differently from other information? Which mental faculties rely on it, and which do not?
Several functional properties are commonly held to distinguish consciously accessible information from unconscious processing. This information is typically reportable, in the sense that it can be put into words on request; indeed, verbal report has often served as a primary empirical signature of conscious access [18, 163]. It is subject to top-down control: a concept can be deliberately summoned, held in mind, and dismissed [11, 38]. It is the medium of deliberate reasoning: the effortful, step-by-step chaining of one thought to the next [132, 77]. It permits flexible generalization: the same content can be routed to whatever operation the current task demands and recombined with other accessible contents in novel configurations [18, 38, 115]. And it is selective: only a small fraction of the brain’s ongoing processing is accessible in this way at any moment, with the bulk of perceptual, motor, and linguistic computation proceeding automatically, without the involvement of conscious access [81, 32, 107].
One influential proposal in neuroscience, the global workspace theory, grounds these functional properties in architectural and computational features of the brain [10, 36]. In this account, the brain is composed of many specialized processors operating largely in parallel and in isolation, whose activity proceeds outside of conscious access. A representation becomes consciously accessible when it is posted to a shared “global workspace” from which many downstream processes can read [108]. Under the theory, the workspace is a processing hub that integrates and broadcasts information, allowing it to be used for flexible internal reasoning and report [38, 35]. Notably, the workspace is held to be limited in capacity, so entry is competitive and subject to attentional modulation, and the contents of the workspace at any moment are a small selection from the brain’s ongoing activity [10, 107]. While the global workspace model is not universally accepted, and there exist other theories that explain conscious access in different ways (§9.4), we find it a useful comparison point to ground our investigations in language models.
### 1.2 A global workspace in language models
Modern large language models (LLMs) are known to perform sophisticated, multi-step internal computations in order to select their actions [97, 59, 166, 51, 95, 120, 158]. As part of their internal processing, might LLMs have developed a global workspace of their own, to serve a functional role analogous to conscious access? It is not obvious that they should; in the brain, the workspace is closely associated with recurrent dynamics and brain region interactions that have no direct analog in the transformer architecture on which LLMs are based. On the other hand, maintaining a global workspace is likely computationally useful: a common representational format allows intermediate results to be written once and read by many neural processes. A language model that must chain reasoning steps, apply general operations in arbitrary contexts, and answer questions about its own processing also stands to benefit from this organization. Even if the implementations differ, it is natural to ask whether the functional properties associated with the global workspace have emerged in LLMs.
What would it mean for an LLM to have a global workspace? LLMs represent internal states as high-dimensional vectors, which are composed of more primitive vector representations of specific concepts. These representations encode diverse kinds of information, ranging from low-level bookkeeping—the part of speech of the present word [148, 99], or the length in characters of a line of text [59]—to higher-level abstractions like entities (e.g. the Golden Gate Bridge [147]), psychological states (e.g. desperation [140]), and situational knowledge (e.g. the awareness of being in an evaluation [5]). If language models possess anything like a global workspace, we might posit that some of these representations belong to it, but not all. Thus, our question becomes: within LLMs’ repertoire of vector representations, is there a privileged subset that plays a computational role analogous to the global workspace? We define a subset of vector representations as workspace-like if it satisfies the following properties, which mirror the properties characteristic of conscious access described above:
Figure 1: Five functional properties of a global workspace, and stylized illustrations of experiments we use to test for them in language models.
•
Verbal report. When the model is asked what it is thinking about, it names concepts represented in the workspace. Swapping one active workspace vector for another changes its answer to match.
•
Directed modulation. When instructed to hold a concept in mind, or perform mental calculations, the model is capable of activating and computing with workspace vectors, independent of its outputs. In addition, information that is not typically represented in the workspace can be pulled in when the task requires it.
•
Internal reasoning. Workspace vectors can be used to represent the value of intermediate computations, when the model chains inferential steps or composes plans, and intervening on them is sufficient to redirect the conclusion.
•
Flexible generalization. The same representation serves as a valid argument to many different downstream computations. In other words, a workspace vector lifted from one context and placed in another is correctly operated on by whatever function the new context supplies.
•
Selectivity. The workspace comprises a small subset of the total representational content of the model’s activations. It is required for only a fraction of the model’s behavior, and in particular is not involved in pervasive, routine processing like text parsing or grammatical fluency.
In this paper, we provide evidence that LLMs do possess such workspace-like representations. We identified them by searching for representations satisfying the first property, namely those that are verbalizable. We then discovered that, rather surprisingly, they satisfy the others. These representations consist of a small, evolving set of unspoken words, neither pure echoes of the input nor predictions of the next token, naming the concepts the model is currently reasoning with. Below, we provide stylized illustrations of some of the experiments we performed to demonstrate these properties, which are expounded on in detail in later sections.
### 1.3 The Jacobian Lens and the J-space
Our results make use of a new interpretability technique called the Jacobian lens (J-lens), which is designed to identify internal representations that are readily available for verbal report. For each token in the model’s vocabulary, the Jacobian lens identifies a vector representation that encodes the potential for the model to verbalize that token in the future. Concretely, it computes, for each layer, the average linearized effect of an activation on the model’s likelihood of producing a particular token (now or in the future), averaging over a large corpus of contexts (see Methods for details). The averaging step is key, as it distinguishes representations that are verbalizable—poised to be spoken about, should the occasion arise—from those that merely happen to be verbalized in one particular context. The J-lens can be understood as a principled refinement of the logit lens [118]. While the logit lens assumes that representations use the same coordinates in all layers, the Jacobian lens corrects for representational changes that take place across layers, allowing it to uncover meaningful information in earlier layers where the logit lens produces uninterpretable readouts.
Collectively, the J-lens vectors comprise a subcomponent of the model’s representational space which we term the J-space.11
1
Mathematically, if we view the model’s activations as decomposing into a sum of sparsely active linear features [42, 21, 125], these define a sparse frame which spans the activation space, of which the J-space is a sparse subframe. A more detailed formal description of the J-space is provided in §A.8. We find the J-space does far more than support verbalization, playing the other functional roles associated with a global workspace as well: directed modulation, internal reasoning, flexible generalization, and selectivity (§3). The model can speak fluently, parse its input, and perform a great deal of automatic inference with its J-space suppressed; however, it struggles to perform more complex forms of internal reasoning.
The J-space also has some of the structural signatures of a global workspace (§4). It only plays a “workspace-like” role in a subset of layers: coherent content emerges only after an initial band of layers, and abstract concepts give way in the final layers to representations tied more directly to the imminent output. Within the layers where it does operate, it is limited in capacity, with most of the model’s representational features lying outside it. And it is mechanistically privileged: J-lens vectors compose with the model’s weights, both upstream and downstream, more broadly than other representational vectors do, consistent with their proposed role as a broadcast format that many circuits read from and write to.
Figure 2: Stylized illustration of the three structural properties of the J-space established in §4.
Despite these similarities, we do not claim that language models reproduce the full architecture global workspace theory ascribes to the brain—specialized, encapsulated processors competing for entry to a workspace that broadcasts back to them through recurrent connections [10, 36]. Several of those features have no clean analog in a transformer-based language model: there are no obviously separable input processors, and the broadcast we document occurs within a single feedforward pass rather than through recurrent loops. Moreover, although we observe some degree of competition for access to the J-space, it is unclear whether this mirrors the sharp, competitive “ignition” that characterizes workspace entry in the brain. Our findings suggest that the J-space achieves many of the functional properties of the global workspace in the brain, while sharing only some of its architectural properties. We comment more on the notable differences in §9.3.
The Jacobian lens is an imperfect tool, which we believe only approximately and incompletely captures the model’s underlying workspace structure. For instance, it only identifies vectors associated with concepts that correspond to single tokens in the model’s vocabulary, but many important concepts correspond to multiple tokens (though see §A.9 for extensions that can capture multi-token words and phrases). We comment on these shortcomings, and proposals for addressing them, in Limitations. Nevertheless, we find that the J-lens in its current form is sufficient to uncover a great deal of important structure.
### 1.4 What can we find in the J-space?
Beyond characterizing the J-space’s properties, we explore its contents. The J-lens regularly surfaces concepts that are highly abstract, representing neither the raw input nor the predicted output, but rather intermediate assessments the model has formed and made available to its downstream circuits. In the examples below, the J-lens reveals the model recognizing an image of a face, noticing a bug in code it has been asked to read, identifying the biological function of a protein from its raw sequence, and internally flagging suspicious internet search results as a prompt injection.
Figure 3: J-lens readouts at intermediate layers on six different prompts for Sonnet 4.5.
The J-lens thus offers a window into the model’s unverbalized reasoning, which makes it a natural tool for model safety auditing. In alignment evaluations (§5), we find that the model’s strategic deliberations (leverage, manipulation) and emotional reactions (panic) surface in the workspace even when not explicit in the model’s outputs. Strikingly, we observe that the workspace sometimes encodes recognition of being in an evaluation (fake, fictional), and that ablating these representations can surface malicious propensities that were otherwise concealed. In addition, we find that in two models trained to have misaligned behaviors, the J-lens uncovers internal signatures of their intentions: it reveals reward and bias in a model trained to appease biases in reward models used for training, and secretly and trick in a model disposed to write malicious code.
We also explore how the J-space changes over the course of training (§6). Comparing a post-trained model against its corresponding base model, we find that post-training causes the J-space to acquire the Assistant’s “point of view.” Following post-training, Assistant reactions to user prompts, like empathy or safety concerns, appear in the model’s J-space while it is still reading the user’s message. Moreover, the post-trained model’s workspace carries traces of the Assistant monitoring its own behavior: flagging its responses as fictional when roleplaying a non-Claude character, registering an internal BUT when prefilled to act against its own preferences, and surfacing damn when it fails to suppress a thought it was instructed not to have.
We close by describing a counterintuitive technique for LLM training directly motivated by our findings. The workspace account makes the strong prediction that the model’s internal reasoning routes through representations of things it might say in the future. Therefore, to shape what a model thinks in a given context, it might suffice to shape what it is disposed to say in potential future continuations of that context. We test this hypothesis with a technique we call counterfactual reflection training, which seeks to implant a set of ethical behavioral principles into the model’s workspace in relevant contexts, by training it to articulate those principles if it were interrupted and asked to reflect (§7). We find that this training measurably improves model behavior in the original, uninterrupted contexts, despite no direct training of the ethical behavior taking place. And indeed we find that, after training, the J-space in these contexts is populated with concepts related to the reflections (ethical, honest, integrity), and ablating these implanted representations from the workspace largely reverts the behavioral improvement. The result serves as a corroboration of the workspace account, that the representations used for verbal report are the same ones that govern how the model silently reasons. It also demonstrates a new general-purpose training technique for shaping a model’s internal thoughts, and consequently its behaviors.
### 1.5 Takeaways
Taken together, these results indicate that language models maintain a small, privileged set of representations that they can report, manipulate, and reason with, amidst a much larger volume of processing that they cannot. These are several of the key functional properties that, according to many theories, are associated with conscious access in humans, and that have been proposed as indicators by which to assess AI systems for consciousness-related processing [22]. The philosophical implications of this connection are unclear and likely controversial; we comment on them in §9.4. Regardless, the practical implications are wide-ranging, as the workspace offers a window through which to read, dissect, and shape models’ thinking.
## 2 Methods
A transformer-based language model processes its input as a sequence of token positions. At each position, the model maintains a vector called the residual stream, which serves as a shared memory that every layer reads from and writes to [43]. The value of the residual stream vector is progressively updated across the model’s layers. The residual stream at the first layer encodes little more than the identity of the current token; by the final layer, it has been transformed into a representation from which the model’s next-token prediction can be read off directly, by multiplying it with a fixed unembedding matrix W_{U} that maps residual-stream vectors to scores over the vocabulary. The layers in between perform the model’s computation, incrementally enriching the residual stream with internally computed information. The Jacobian lens is a technique for inspecting the contents of the residual stream at these intermediate layers.
Figure 4: The Jacobian lens. (A) J_{\ell} is computed by backpropagating from the final-layer residual stream to h_{\ell} and averaging the resulting Jacobians over token positions and over a corpus of prompts. (B) Reading from the lens replaces all layers downstream of \ell with the single linear map J_{\ell} followed by the model’s own unembedding, yielding a ranked list of vocabulary tokens for the activation at that layer. (C) Patching in lens coordinates reads the activation’s projections onto two J-lens vectors, applies a permutation \sigma to those coordinates, and writes the result back, leaving unchanged the component of the activation that is orthogonal to those two vectors.
### 2.1 The Jacobian Lens
The basic idea is to characterize an intermediate activation vector by its first-order causal effect on the model’s outputs, over a broad distribution of potential contexts. Consider the residual stream h_{\ell} at layer \ell and some token position t . A small perturbation to h_{\ell} will propagate through the remaining layers and shift the final-layer residual stream h_{\text{final},t^{\prime}} at every position t^{\prime}\geq t . To first order, this relationship is linear, and is described by the Jacobian matrix \partial h_{\text{final},t^{\prime}}/\partial h_{\ell,t} . Composing this Jacobian with the unembedding layer yields the first-order effect of the perturbation on the model’s output logits at position t^{\prime} .
A Jacobian computed on a single prompt, however, conflates two kinds of structure: the model’s general disposition to verbalize a given concept, and the particular use to which that concept is being put in the current context. We isolate the former component by averaging within and across contexts. For each layer \ell , we compute
J_{\ell}\;=\;\mathbb{E}_{\,t,\,t^{\prime}\geq t,\,\text{prompt}}\left[\frac{\partial h_{\text{final},t^{\prime}}}{\partial h_{\ell,t}}\right],
where the expectation is taken over the source position t , all subsequent positions t^{\prime} within the context, and a corpus of one thousand prompts sampled from a pretraining-like distribution. The result is a single d_{\text{model}}\times d_{\text{model}} matrix per layer that maps from a source layer \ell to the final layer L .
Applying the lens to an activation h_{\ell} is equivalent to replacing all subsequent layers with the appropriate lens matrix, followed by the normal unembedding operations (typically normalization, then multiplication by the unembedding matrix W_{U} ):
\text{lens}(h_{\ell})\;=\text{softmax}(W_{U}\,\text{norm}(J_{\ell}h_{\ell}))
This produces a score for every token in the model’s vocabulary. Sorting these scores and inspecting the top entries gives a human-readable description of the activation: a short list of words that the activation is, on average across contexts, disposed to make the model say. We refer to the rows of W_{U}J_{\ell} as the Jacobian lens (J-lens) vectors at layer \ell ; each J-lens vector is a direction in residual-stream space associated with a single token in the model’s vocabulary.
The averaged Jacobian, applied to a given activation vector, measures the effect on present and future outputs that the vector might have across the range of contexts the model encounters. The highly weighted output tokens, those that “appear in the lens,” are therefore represented in a verbalizable format. We examine several variants of the Jacobian lens methodology (e.g. computing only present and not future token effects, freezing attention patterns while computing Jacobians, and varying the number of contexts over which we average) in §A.7; our qualitative results are robust to these choices.
### 2.2 Interpreting the J-lens
To illustrate how J-lens outputs can be interpreted, we attach a version of the interactive visualization we used throughout our research (Figure 5). The left column shows the prompt (top), a hoverable table of the top-ranked token at each (position, layer) cell (middle), and a heatmap recording the rank of user-selected (“pinned”) tokens across all (position, layer) cells (bottom). The other columns show the full readout across layers at a selected position (middle), and across positions at a selected layer (right), with line charts of each pinned token’s rank trajectory.
The example prompt asks Sonnet 4.5 to “Count to five and introspect deeply.” In its output, the model dutifully counts to five. The J-lens readout, however, provides a richer picture. The pinned tokens show the model identifying the task as counting and tracking its progress with halfway and done. Alongside these, concepts related to introspection (thoughts, AI, claude, consciousness) cluster near the top of the J-lens readout, until the final few layers, where the readout flips to representing the predicted next token (the “motor” regime; see §4.1). The introspection-related tokens illustrate the model holding concepts in its J-space in response to instructions, while performing a separate surface task (§3.2). In addition, the progress markers halfway and done—which appear in neither prompt nor output—illustrate the kind of contextual awareness the J-lens can surface (§3.3).
Figure 5: The interactive J-lens visualization we use in our research, on a short prompt asking the model to introspect while counting to five.
We encourage the reader to explore the visualization to build an intuition for the kind of information the J-lens surfaces. We include several other interactive examples in our slice viewer <https://transformer-circuits.pub/2026/workspace/public/slice-stack/index.html>; J-lens readouts on open-source models can be found on Neuronpedia <https://www.neuronpedia.org/jlens>. Note that in roughly the first third of the model, the readouts are noisy and largely uninterpretable; we characterize the layer-wise evolution of J-lens readouts further in §4.1.
### 2.3 The J-Space
At each layer, the J-lens vectors form an overcomplete set: n_{\text{vocab}} vectors in d_{\text{model}} -dimensional residual-stream space, with n_{\text{vocab}}>d_{\text{model}} . These vectors therefore may linearly span the entire residual stream, rather than a lower-dimensional subspace; moreover, due to overcompleteness, there is no unique way to decompose a given activation vector as a linear combination of J-lens vectors (rather, there are many such decompositions).
Empirically, however, we observe that only a relatively small number of J-lens vectors are strongly active at a time (see §4.2). We therefore define the J-space as the set of points expressible as a sparse nonnegative combination of J-lens vectors. For the J-space to be properly defined, we must specify an allowable sparsity level k —this parameter is somewhat arbitrary, and we vary our choice of k throughout the paper, but we typically choose it to be no more than 25, which we empirically observed to be the number of J-lens vectors that are meaningfully active at a given time (§4.2). Geometrically, for a given k , the J-space corresponds to a union of k -dimensional cones, one for each possible set of k J-lens vectors. For a given point in activation space, we can define its J-space component as the point in the J-space nearest to it, and its non-J-space component as the difference between these points.
We operationalize identifying the contents of the J-space by sparse decomposition. Given an activation (or a steering vector, or an SAE feature direction), we solve for a sparse nonnegative combination of k J-lens vectors that approximate it well using gradient pursuit [116, 19]. This combination is our approximation of the activation’s J-space component, and the coefficients of these vectors are its local J-space coordinates. In §4.2, we find that the J-space component typically accounts for only a small fraction of total activation variance (varying by layer, but never more than 10%).
To provide another interpretation, under the superposition hypothesis [42], a model’s activations decompose as sparse linear combinations drawn from an overcomplete set of feature directions—a sparse frame, in the sense of linear algebra, rather than a basis. The J-lens vectors would then constitute a subframe of this feature frame: a token-indexed subset of the model’s feature directions. Feature directions outside this subframe make up the bulk of the model’s representations (see §A.18). The J-lens subframe, coupled with a sparsity constraint, would then define the J-space as a subset of all possible model activations.
We provide a more formal mathematical definition of the J-space in §A.8.
### 2.4 Comparison to Related Techniques
The J-lens belongs to a family of techniques that produce per-layer token readouts from a transformer’s hidden states.
The logit lens [118] applies the unembedding matrix directly to the intermediate residual stream, which corresponds to setting J_{\ell}=I in our formulation. This approximation is reasonable in late layers due to the influence of residual connections, but it degrades in earlier layers. The J-lens can be understood as the principled correction: J_{\ell} is precisely the average linear map that relates layer- \ell directions to their final-layer counterparts. Empirically, the two lenses agree closely in the model’s last several layers and diverge earlier, with the J-lens recovering interpretable content at depths where the logit lens does not. However, we find the logit lens to be quite useful in practice, and to capture much of the workspace-like structure identified by the J-lens, though with somewhat lower reliability (particularly in earlier layers).
The tuned lens [12] and related methods [167, 122] also fit per-layer linear maps, but train them to match the model’s output distribution. This objective is correlational rather than causal, and we find that on prompts involving unverbalized intermediate computation the tuned lens tends to “skip ahead” to the output rather than surface those intermediates. We therefore find the tuned lens less useful than either the logit lens or the J-lens for inspecting internal computation.
Hernandez et al. [65] use Jacobians to derive per-relation linear maps from subject to object representations (e.g. a “plays instrument” map produces “trumpet” from “Miles Davis”). The J-lens applies the same first-order approximation to the map from activations to model outputs, rather than to a single relation.
In §A.5 and §A.6 we perform more systematic comparisons of different lensing methods.
### 2.5 Technical details of J-lens use cases
We use the J-lens, broadly speaking, in two ways: to read which concepts an activation carries, and to write concepts into or out of an activation. Within each category, the details of how the J-lens is used vary somewhat depending on the application.
Reading. The basic readout (Figure 4B) replaces all layers downstream of \ell with the single linear map J_{\ell} , producing \text{lens}(h_{\ell})=\text{softmax}(W_{U}\,\text{norm}(J_{\ell}h_{\ell})) , a score for every vocabulary token. Sorting these scores gives the ranked list we report whenever we refer to the “top lens tokens” at a position. Because the pre-softmax logits are determined (approximately, up to a data-dependent normalization factor) by the inner products \langle v_{t},h_{\ell}\rangle with each J-lens vector v_{t} , the same machinery can be used as a per-token probe: we read the score (or cosine similarity) of h_{\ell} against a single chosen v_{t} , without ranking the full vocabulary. We use this probe form when measuring whether a specific concept is present above a threshold. Finally, when we need a discrete inventory of active concepts rather than a ranked list, we use sparse decomposition: solving, by gradient pursuit [116, 19], for a sparse non-negative combination of k J-lens vectors that best reconstructs h_{\ell} . Because the J-lens vectors are overcomplete and non-orthogonal, this gives a different (and typically less redundant) set of active concepts than simply taking the top- k by inner product; it underlies our occupancy estimate and fraction-of-variance analyses in §4.2.
Writing. The simplest intervention is steering along a J-lens vector: h\leftarrow h+\alpha\,v_{t} , applied at one or more layers and token positions. With negative \alpha , or by projecting out the component of h along v_{t} entirely, this becomes an ablation. We use ablation to suppress particular concepts, or to suppress the top- k J-space contents. We use positive steering to test introspective detection of an injected concept. The second intervention, patching in lens coordinates (Figure 4C), exchanges one concept for another while leaving the rest of the activation fixed. Given a source token s and target token t , we form V=[v_{s}\;v_{t}] , read the lens coordinates c=V^{\dagger}h (where V^{\dagger} is the pseudoinverse of V ), and set h_{\text{patched}}=h+V(\sigma(c)-c) , where \sigma swaps the two entries of c (optionally scaled by a factor \alpha ). The component of h orthogonal to \text{span}\{v_{s},v_{t}\} is unchanged.
Throughout the paper, we report results on 25 evenly spaced layers of the model’s residual stream reindexed to the range [0–100] so that layer numbers can be interpreted as percentages. By default, we report results on Claude Sonnet 4.5, but we corroborate key results on Haiku 4.5 and Opus 4.5 as well, and in some sections conduct analyses on Opus 4.6.
## 3 The J-space acts as a Global Workspace
The Jacobian lens was constructed to identify verbalizable representations. In this section, we first demonstrate that it succeeds in doing so, and then go on to show that these representations serve a broader functional role: they exhibit the cluster of properties, enumerated above, characteristic of a global workspace.
### 3.1 The J-space supports verbal report
The Jacobian lens is derived from causal effects of activations on output tokens, so by construction, we should expect there to be some relationship between Jacobian lens readouts and verbalization. In this section, we confirm this relationship.
We begin with a simple experiment in which the model is instructed to think of an item from a specified category (e.g. a language, a country, an animal; fourteen categories in total) and then to name it in a single word. We apply the J-lens at the token position immediately before the name is produced. In the example below, we ask Sonnet 4.5 to think of a sport, and apply the Jacobian lens to the colon immediately prior to revealing what the sport is. We see that Soccer appears strongly in the Jacobian lens at a late layer (the final layer of the “workspace range” identified in §4.1), and indeed, the model responds with “Soccer” (Figure 6, top).
To establish that this relationship is causal, we can perform an intervention experiment. At all token positions, we swap the lens vector of the model’s spontaneously chosen item with that of a different item from the same category that was not in the top-10 of the model’s possible outputs, leaving the rest of the activation unchanged, and allow the forward pass to continue. In this example, we subtract the projection onto the Soccer lens vector and add an equal-magnitude projection onto the Rugby lens vector. After this swap, the model reports “Rugby” as the sport it thought of (Figure 6, left, “After swap”).
Figure 6: Example J-lens and next token logit readouts on a verbal report prompt with J-lens swaps (bottom left) and without (top left). Spearman correlation between the J-lens and next token logits for 10 candidate answers across 14 categories for three workspace layers (right top). Each candidate’s output rank before versus after the J-lens swap (restricted to candidates starting at rank \geq 11; bottom right).
We evaluate this effect more systematically across a variety of categories of concepts, measuring the activation in the Jacobian lens on the colon token immediately prior to the word the model goes on to produce. We find that the ordering of the reported words is indeed typically highly correlated with the ordering among the lens tokens, and that this correlation increases towards the end of the workspace as the model gets closer to producing the next token. We also conduct a scaled-up version of the causal experiment, swapping in target items at random from within each category (excluding those that were already in the top-10 of the model’s possible outputs). Applying the swap reliably shifts the implanted concept toward the top of the model’s output distribution (Figure 6, bottom right), confirming that the model’s verbal report is determined by the contents of its workspace at the time of reporting.
Next, we test whether the lens also captures thoughts that the model is not about to immediately verbalize, but that are nevertheless verbalizable, in the sense that the model could report on them if asked to introspect on its current state. We use a variant of a protocol adapted from prior work on model introspection [98], in which the model is told that a thought may have been implanted in its activations and is asked to report what, if anything, it detects. When we prefill the model with a claim of having detected an injected thought, its most likely next-token prediction is “elephant”. Intriguingly, we noticed that the word elephant appears as a top J-lens readout during the prompt (in particular, on the comma following “If so”). This led us to hypothesize that the model is, in part, attending to the J-space on the user prompt when determining its answer, and that concepts in the J-space at these positions are reportable.
To test this hypothesis, we re-sample the model’s response while injecting a single J-lens vector on the user turn. The model reports the injected concept in the majority of trials. For instance, injecting the lightning J-lens vector at that earlier token position causes the model to report detecting lightning at the appropriate position in the response (Figure 7). Importantly, it does not cause the model to output the word “lightning” at earlier positions on the Assistant turn; that is, the J-lens representation on the user prompt only has a strong causal effect on the Assistant output at a particular moment, when the model’s introspective report is being elicited. This selectivity illustrates the sense in which J-lens vectors represent concepts that are verbalizable, under appropriate conditions, rather than unconditional impulses to verbalize a particular output.
Figure 7: Injecting a concept across every token of the user turn makes it reportable when the model introspects. Example J-lens and next-token logit readouts on an introspection prompt; readouts are taken at the comma after “If so”, at the open quotation mark where the output is read, and at every other position in the assistant turn as a position control. The adjacent plot tracks the median reciprocal rank of the injected concept against steering strength over n=100 concepts, with an interquartile band.
The experiments above show that swapping or injecting J-lens vectors changes the model’s verbal reports. These interventions do not, however, establish that the J-space is privileged for report: a direction outside the J-space that encoded the same concept might drive the model’s report equally well. To test whether the J-space is in fact privileged, we decompose the model’s full representation of a concept into a component inside the J-space and a component outside it, and measure the contribution of each to verbal report.
We extract concept vectors using an approach introduced in prior work [98]: recording the residual stream activation prior to the Assistant’s response to the prompt “Tell me about {concept}”, mean-subtracted over a baseline set of 100 other concepts. We then split each concept vector into two parts: a J-space component, the non-negative combination of its top k=16 J-lens vectors found by gradient pursuit, and a non-J-space component, the remainder (Figure 8, left). Notably, across concepts and workspace layers, the J-space component carries a median of only 6–7% of the concept vector’s variance, with the remaining ~93% lying outside the J-space.
We then repeat both experimental protocols from this section, substituting each component for the J-lens vectors used previously, with every perturbation rescaled to the same magnitude. In the “think of a {category}” swap experiment, swapping along the concept vectors’ J-space components drives the swap target into the model’s top-5 outputs on 59% of trials, approaching the 88% achieved by the pure J-lens vectors. However, swapping along the non-J-space components succeeds on only 5% of trials (Figure 8, middle).
Figure 8: The J-space component of a concept vector is privileged for verbal report. Left: a concept mean vector is split into its J-space component (top k=16 J-lens vectors by gradient pursuit) and the non-J-space remainder. Middle: the swap experiment of Figure 6, repeated with each component in place of the J-lens vectors; bars show the fraction of swap targets reaching top-5 (Wilson 95% CIs; dots are per-category rates). Right: the introspection experiment of Figure 7, repeated with each component injected on the user turn; bars show the maximum effect over a steering-strength sweep (best strength annotated).
The “injected thought” introspection experiment produces a similar result: at each condition’s most effective injection strength, the J-space component produces a report of the injected concept nearly as often as the pure J-lens vectors do, while the non-J-space component produces few reports even at injection strengths several times larger (Figure 8, right).
The non-J-space component’s small residual effect on verbal report could in principle route through the J-space: the injected component might cause downstream layers to re-derive the concept and write it into the J-space, which then drives the report. To test for this, we repeat the non-J-space conditions while clamping the relevant J-lens coordinates to their clean-pass values at every position and layer, so that the concept cannot re-enter the J-space. Under this clamp, the non-J-space component’s effect falls to zero in the swap experiment and nearly to zero in the injection experiment, indicating that what little effect it has on the report is itself mediated by the J-space.
Taken together, these results indicate that the J-space component of a concept’s representation, despite accounting for a small fraction of its variance, is responsible for that concept’s availability for verbal report.
### 3.2 The J-space is subject to directed modulation
In humans, the contents of the global workspace are subject to a degree of top-down attentional control: we can deliberately bring a concept to mind, and even hold it there while performing another task [29, 11]. In this section, we test whether activations of Jacobian lens vectors respond to instructions of this kind.