Probing Transformer Representations
Layer by Layer, Module by Module
Abstract
Which representation should be probed in a transformer-based foundation model? Recent work shows that the standard practice of probing the final layer is often suboptimal, as intermediate layers can provide better features. Yet, representations are still largely studied along depth alone, treating each transformer block as an atomic unit. In this work, we analyze transformer representations layer by layer, module by module. Through more than 144,000 linear probing evaluations of vision foundation models (VFMs) and large language models (LLMs), we show that layer and module choices should be considered jointly: the best layer-module pair yields substantially larger gains over the final-layer residual stream than the best layer with the residual stream fixed. Moreover, module performance can vary substantially across layers, showing that useful representations cannot be characterized by depth alone. Building on these findings, we evaluate simple fixed probing strategies: the final-layer feedforward activation (Act@100) for vision and the second normalization layer at 60% depth (LN2@60) for language. These strategies improve over the probing baseline in a majority of evaluations and recover, on average, 42% and 86% of the gains from probing the best layer-module pair for vision and language, respectively. For VFMs, these gains increase with the transfer difficulty between pretraining and downstream data.
What does the probing baseline read?
- LN1
- first normalization
- MHA
- multi-head attention
- RC1
- residual stream after attention
- LN2
- second normalization
- FC1
- first linear layer of the FFN
- Act
- FFN activation
- FC2
- output of the FFN
- RC2
- residual stream at the block output
A transformer block exposes eight intermediate representations. Probing usually reads one of them, RC2, at the last layer (sometimes after an additional normalization layer). We take it as the probing baseline, RC2@100. We probe all eight at every layer, in a single forward pass, with linear probing on 18 image datasets for vision foundation models (VFMs) and on the 24 classification tasks of MTEB(Multilingual) for LLMs.
Each location is compared to the probing baseline. The oracle gain is the accuracy of the best layer-module pair minus that of RC2@100, in percentage points (pp). It is an upper bound: finding that pair needs an exhaustive search and test labels.
How much does the probing baseline leave on the table?
- 260 / 270
- VFM model-dataset pairs where another location beats the probing baseline
- +2.2 pp
- average oracle gain in VFMs, up to +10.5 pp
- +2.9 pp
- average oracle gain in LLMs, from +2.0 to +4.1 pp across the 9 models
The gain comes from choosing the layer and the module together. Searching over both beats searching over the layers of the residual stream or over the modules of the last layer, in both modalities. In LLMs, the residual stream is never part of the best pair.
Where are the better representations?
For vision, mostly in the FFN activation of the last layers: Act is in the best pair for 64.8% of the model-dataset pairs, against 5.9% for the residual stream. For language, in the second normalization at intermediate depth: LN2 is in the best pair for 8 of the 9 LLMs, at 48% to 78% of the depth.
The heatmaps of the animation above show where the best pair lies across all models. You can explore each model and dataset at the end of this page.
Can we get there without search?
Yes, with one fixed location per modality. Act@100, the FFN activation of the last layer, improves over the probing baseline in 63.7% of the 270 VFM evaluations. LN2@60, the second normalization at 60% depth, improves on 21 of the 24 MTEB tasks and on all 9 LLMs.
On average, Act@100 recovers 42% of the oracle gain in VFMs and LN2@60 recovers 86% in LLMs, more than the best layer of the residual stream in both cases. Probing the best layer of the same module recovers 78.1% and 99.8%.
When does it matter most?
The benefits of optimal probing locations increase with the distribution shift between pretraining and downstream data. In VFMs, the oracle gain grows with transfer difficulty, measured by the probing error of the baseline (Spearman ρ = 0.87). On the distribution shifts of WILDS and DomainNet, Act@100 improves 25 of the 28 evaluations of the 307M variants, by +1.8 pp on average.
Explore the results yourself
Pick a model, then click any cell, or use the module and layer controls, to compare that location with the probing baseline (○), the fixed rule (◇) and the best location (★).
Discussion
Transformer blocks are not monolithic: where a representation is read inside the block matters as much as how deep it sits. When transformers are used as frozen feature extractors, the module is a choice worth making alongside the layer, and the fixed rules give a cheap default, one extra hook in the same forward pass.
Our analysis focuses on linear probing for classification. Natural next steps are other downstream tasks and other modalities, such as tabular and time series foundation models. Knowing which internal representations are useful could also guide the design of aggregation mechanisms that combine intermediate representations, rather than treating all layers and modules uniformly.
BibTeX
@misc{odonnat2026probing,
title={Probing Transformer Representations Layer by Layer, Module by Module},
author={Ambroise Odonnat and Theo Gnassounou and Laetitia Chapel and Romain Tavenard and Ievgen Redko},
year={2026},
eprint={2611.11531},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2611.11531},
}