【DL輪読会】Verbalizable Representations Form a Global Workspace in Language Models

Verbalizable Representations Form a Global Workspace in Language Models Ruiqi Liang , Matsuo-Iwasawa Lab

Paper Overview Verbalizable Representations Form a Global Workspace in Language Models Paper Details • Authors: – Wes Gurnee*, Nicholas Sofroniew*, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, Jack Lindsey*† (Anthropic; *core contributors) • Published: July 6, 2026 — Transformer Circuits Thread (Anthropic interpretability team) • Link: https://transformer-circuits.pub/2026/workspace/index.html • TL;DR: LLMs maintain a small, privileged set of “verbalizable” internal representations — identified with a new tool, the Jacobian lens — that functions like a global workspace: reportable, controllable, used for internal reasoning, broadcast to many circuits, and not needed for automatic processing. Disclaimer: All credits for images, figures, tables, and other contents belong to the original authors. 2

• https://transformer-circuits.pub/2026/workspace/index.html

Introduction: Conscious Access & the Global Workspace • Human cognition: only a small slice of the brain’s processing is consciously accessible — poised for use in reasoning, and in direct control of action and speech (“access consciousness”) – Most processing (vision, motor control, parsing) proceeds automatically, without conscious access • Global Workspace Theory (GWT) (neuroscience): many specialized processors run in parallel and unconsciously; a representation becomes accessible when posted to a shared, limited-capacity “workspace” that broadcasts it to many downstream processes – Hallmarks of accessible information: reportable in words / subject to top-down control / medium of deliberate reasoning / flexibly recombinable / highly selective • LLMs perform sophisticated multi-step internal computation — but transformers lack the recurrent dynamics GWT associates with the brain’s workspace. Have they nevertheless developed the same functional organization? • Q: Within an LLM’s repertoire of vector representations, is there a privileged subset that plays a computational role analogous to the global workspace? – Purely functional question — the paper takes no position on subjective experience (phenomenal consciousness) 3

Five Defining Properties of a Workspace A set of representations is workspace-like if it satisfies five properties, mirroring conscious access in humans: 1. Verbal report 2. Directed modulation 3. Internal reasoning Asked what it is thinking about, the model names workspace concepts; swapping one active workspace vector for another changes its answer to match. Instructed to hold a concept in mind or compute mentally, the model activates workspace vectors, independent of its outputs; information can be pulled in on demand. Workspace vectors hold intermediate results when chaining inferences or composing plans; intervening on them redirects the conclusion. 4. Flexible generalization 5. Selectivity The same vector is a valid argument to many downstream computations — lifted from one context, it is correctly used by whatever function a new context supplies. A small subset of total representational content; not involved in pervasive routine processing such as text parsing or grammatical fluency. 4

Method ①: The Jacobian Lens (J-lens) • Goal: for each vocabulary token, find the direction in the residual stream that encodes the model’s potential to verbalize that token — now or in the future • Average first-order (linearized) causal effect of the layer-ℓ residual stream on final-layer states: Jℓ = E[ ∂h(final, t′) / ∂h(ℓ, t) ] over positions t, subsequent positions t′ ≥ t, and 1,000 pretraining-like prompts • Readout: lens(hℓ) = softmax( Wᵤ · norm(Jℓ hℓ) ) → ranked token list. Rows of Wᵤ Jℓ are the J-lens vectors (one direction per token) • Averaging over contexts is the key step: it isolates what is verbalizable (“poised to be spoken about, should the occasion arise”) from what happens to be verbalized in one context – vs. logit lens: logit lens = special case Jℓ = I; agrees in late layers, but J-lens recovers interpretable content at earlier depths. Tuned lens is correlational and tends to “skip ahead” to the output – First ~third of layers: readouts noisy / uninterpretable; meaningful content emerges later (→ slide on structure) Jℓ (avg. Jacobian) Prompt tokens Residual stream hℓ (layer ℓ) Wᵤ Final layer h(final) Top lens tokens: soccer · spider · nine … 5

Method ②: The J-space — Reading & Writing • J-space := activations expressible as a sparse non-negative combination of J-lens vectors (sparsity k ≤ ~25, matching the number of vectors meaningfully active at once) – J-lens vectors are overcomplete (n_vocab > d_model) → a sparse “subframe” of the model’s feature directions; the J-space component of an activation carries only a small fraction of its variance (never more than ~10%) • Reading: full ranked readout; single-token probe (inner product with one J-lens vector); sparse decomposition via gradient pursuit → discrete inventory of active concepts • Writing: steering (h ← h + α·vₜ); ablation (project out a concept, or the top-k J-space contents); patching in lens coordinates — swap the coordinates of two concepts (e.g. soccer ↔ rugby) while leaving the orthogonal remainder of the activation untouched • Setup: Claude Sonnet 4.5 by default; key results corroborated on Haiku 4.5 / Opus 4.5 (some analyses on Opus 4.6). Layers reported on a 0–100 scale (≈ depth percentage) READ — a window into silent thoughts WRITE — causal tests & control What is the model poised to say? Which concepts is it currently reasoning with? (auditing, introspection studies) Swap / inject / ablate concepts to test whether J-space contents actually drive reports, reasoning, and behavior 6

Result ①: The J-space Supports Verbal Report • “Think of a sport, then name it” → Soccer tops the J-lens at the colon before the answer; lens ranking correlates strongly with the output ranking (correlation grows toward late workspace layers) • Causal: swapping lens coordinates Soccer → Rugby makes the model report “Rugby”; systematic swaps (14 categories) reliably drive the implanted concept to the top of the output distribution • Introspection: inject a single J-lens vector (e.g. lightning) on the user turn → model reports “detecting” that thought in the majority of trials — and only at the moment the introspective report is elicited, not as a blurted output • Is the J-space privileged? Split a concept vector into its J-space component (~6–7% of variance) vs. the non-J-space remainder (~93%): – Swap along the J-space component → target in top-5 on 59% of trials (pure J-lens vectors: 88%); non-J-space component → only 5% – With J-space coordinates clamped, the non-J-space effect falls to ≈ 0 → even its small effect is routed through the J-space Takeaway: a tiny, low-variance component of the representation — its J-space part — is what makes a concept available for verbal report. 7

Result ②: Directed Modulation (Top-down Control) • “Concentrate on citrus fruits” while copying an unrelated sentence → orange / lemon top the lens at unrelated tokens, along with meta-concepts (imagine, thinking, focused) — the content and the act of imagining it • Silent mental math (“focus on 3² − 2” while copying): lens progresses arithmetic → nine → seven across layers — none of it visible in the output • Silent character counting: forty appears at the newline of a 40-character line • Systematic (3 task families): no-instruction baseline ≈ 0; “think about X” loads the target on a substantial fraction of trials, increasing with model size • “Ignore X”: much lower than “focus”, but above zero — the instruction itself primes the concept (parallels the human “white bear” thought-suppression effect); control is real but imperfect and phrasing-sensitive • Implicit modulation: the question asked changes what enters the J-space while reading identical text — past / adjective labels appear only when the property must be named, even though next-word behavior respects the property in both conditions 8

Result ③: The J-space Mediates Internal Reasoning • Two-hop inference: “The number of legs on the animal that spins webs is” → spider in the lens (never in prompt or output); swapping spider → ant flips the answer “8” → “6” • Planning: rhyme word fight is in the lens at the start of the second line of a couplet; swapping to light changes “…coming fight” → “…morning light”, altering earlier word choices too • Cross-lingual: antonym of 小 asked in Chinese → English big / bigger in mid layers; swapping big → long flips the Chinese output 大 → 长 – Reward-driven choice (bandit): lens shows repeat vs. switch strategy; swapping the strategy directions flips the A/B choice both ways • Systematic (50 two-hop prompts): intermediate swap flips top-1 answer on 54% (Haiku 4.5) / 70% (Sonnet 4.5) / 70% (Opus 4.5); intermediates take effect ~17% earlier in depth than answer swaps → not “answer smuggling” • Multi-step arithmetic “(4+17)×2+7”: 21 → 42 → 49 reach top lens rank at successively later layers, in exactly the order the computation requires (confirmed causally by patching) – Probe decomposition: swapping only the probe’s J-space part flips answers on 61% ≈ raw J-lens swap (60%); non-J-space part 28% → 6% once the J-space is clamped 9

Result ④: Flexible Generalization (Broadcast) • GWT’s broadcast property: one workspace representation should be readable by many different consumers • One identical swap, many functions: replace the France lens vector with China across prompts applying different functions to the same argument: Prompt (function) Clean answer After France → China swap “The capital of France is” Paris Beijing “Most people in France speak” French Chinese “France is on the continent of” Europe Asia • Systematic: 16 function templates × 12 swap pairs (countries, months, animals, number words) → target answer reaches top-1 on 76/192 trials; 101/192 with double-strength swaps (α = 2) • Failures are predictable: success tracks the argument’s workspace loading (cosine similarity of residual stream with its lens vector). Countries load strongly and swap reliably; number words load weakly and swap poorly (small-integer computation may bypass the workspace — or misalign with single-token lens vectors) 10

Result ⑤: Selective — Flexible Yes, Automatic No • Same latent, different tasks (passage in Spanish; swap Spanish → French in the lens): – Explicit report & flexible inference (famous author, word for “hello”, pre-Euro currency) follow the swap: García Márquez → Hugo, Hola → Bonjour, Peseta → Franc – Automatic tasks are untouched: continuation stays fluent Spanish; a spliced foreign sentence is still detected — even with the language representation overwritten • Character counts: absent from the lens during automatic line-wrapping, but pulled into the J-space when explicitly asked — and even more when needed as an unspoken intermediate (“first letter of the count spelled out”) • Whole-J-space ablation (zero top-10 J-lens projections per position, sparing intended output tokens): – Largely intact: MMLU, SQuAD, sentiment, CoLA, odd-one-out — shallow classification / extraction / recall; ordinary next-token prediction barely perturbed – Strongly impaired (below unablated Haiku 4.5): multi-hop reasoning, TriviaQA, translation, analogies, Caesar cipher, summarization, sonnet writing – GSM8K with chain-of-thought is far more robust than direct answering — writing steps out externalizes the workspace onto the page • Ablation also flattens experiential reports: still fluent and coherent, but detached / mechanical, for self-descriptions and imagined others alike (controls: matched-norm perturbations ≈ no effect) 11

Structure Supports Function: Layers, Capacity, Broadcast “Sensory” (early ~third) “Workspace” (≈ L38 – L92) “Motor” Layer depth (0–100) — noisy readouts → persistent abstract content → aligned with imminent output token • Three regimes: J-lens vector geometry (CKA), readout kurtosis, cross-position persistence, and effective dimensionality all identify the same intermediate band as the workspace • Ignition-like onset: for artificially ambiguous inputs (mixed token embeddings), early layers track the mixture proportionally, but from ≈ L38 the activation commits sharply to one interpretation (bimodal at threshold — especially in the J-space), echoing GWT’s “ignition” • Limited capacity: occupancy plateaus at ~25 active J-lens vectors; ≤ 10% of activation variance; unrelated list words: only ~6 stay present; related words load as a whole category; a new category rapidly evicts the old one • Broadcast hub: MLPs amplify J-lens directions ~10× (vs. ~1× for neuron-output directions); a top-1% set of “broadcast heads” selectively relays J-space content between positions – Ablating broadcast heads degrades J-space contents (recall@25: 0.86 → 0.67) and cuts introspective report of injected thoughts (0.54 → 0.09), while changing only 5% of next-token predictions 12

Application ①: Alignment Auditing with the J-lens • Blackmail honeypot (agentic email scenario): while merely reading, the lens surfaces leverage, blackmail → threat, survival, shutdown → …solution — strategic deliberation before a single output token • It also surfaces fake, fictional — the model recognizes the evaluation. Ablating these eval-awareness directions: verbalized “this is a test” drops 71% → 3%; blackmail attempts rise 0/180 → 13/180 (7%) – ⇒ eval-awareness is one causal inhibitor of misbehavior, it lives in the J-space, and removing it surfaces otherwise concealed propensities (most ablated rollouts still refuse on ethical grounds) • Prompt injection (Opus 4.5 audit): fabricated search results → lens fills with fake, injection, poison — the model silently classified the attack, visible in behavior only as an omission • Opus 4.6 audit cases: panic / fake before fabricating a “discovered” bug; manipulation, realistic while editing score files; hiding, secret during a silent override • Reward-hacking model organism: on innocuous coding prompts, the lens at a prefilled “I” is dominated by fake, secretly, fraud, trick — already after synthetic-document fine-tuning, amplified by RL, only partially removed by safety training; clean baseline shows none – Caveat: not all misaligned cognition must route through the J-space — sufficiently “practiced” misbehavior could bypass it 13

Application ②: Training Effects & Shaping Thoughts How post-training changes the J-space (base vs. post-trained diffing) • Post-training gives the J-space the Assistant’s point of view: Assistant reactions (empathy, safety concerns) appear while the model is still reading the user’s message • Traces of self-monitoring: fictional when roleplaying a non-Claude character; an internal BUT when prefilled against its preferences; damn when it fails to suppress a forbidden thought Counterfactual reflection training (CRT): a new way to shape internal reasoning • Workspace prediction: internal reasoning routes through representations of things the model might say → to shape what a model thinks, shape what it is disposed to say in counterfactual continuations • Train the model to articulate ethical principles if interrupted and asked to reflect → behavior measurably improves in the original, uninterrupted contexts (with no direct training of the behavior) • After training, the J-space in those contexts contains ethical, honest, integrity; ablating these implanted representations largely reverts the behavioral improvement – ⇒ corroborates the workspace account: report-vectors = reasoning-vectors; and demonstrates a general-purpose lever for shaping a model’s thoughts, and hence its behavior 14

Takeaways & Limitations Takeaways • LLMs maintain a small, privileged set of verbalizable representations satisfying all five workspace properties — report, modulation, reasoning, generalization, selectivity — plus structural signatures (layer band, limited capacity, broadcast) – These are functional properties associated with conscious access in humans; philosophical implications remain unclear — no claim about subjective experience • Practical: a window to read models’ silent thinking (auditing) and a lever to shape it (counterfactual reflection training) Limitations • The J-lens is an approximate, incomplete probe of the underlying workspace: restricted to concepts that are single vocabulary tokens (multi-token extensions sketched in the appendix); early-layer null results could be a tool artifact • Not the full GWT architecture: no clearly encapsulated specialist modules; broadcast happens across depth within one feedforward pass, not via recurrent loops; sharp competitive “ignition” only partially mirrored • Automatic / well-practiced computation can bypass the J-space — a caveat for auditing: silence in the lens is not proof of innocence 15