Reasoning fine-tuning amplifies a familiar activation pattern
After training a chat model on worked-out solutions, we found very large internal activations at conversation-formatting tokens. Most of the measured change just before the model answers is a common shift in a few dimensions—the same high-magnitude dimensions already observed in ordinary Qwen models.
What this suggests: the largest numerical changes partly reflect an existing model phenomenon becoming stronger and appearing at additional positions. Their size alone does not tell us where the model learned to reason.
What we compared—and what the terms mean
| Before | Qwen2.5-7B-Instruct, a general-purpose chat model. |
|---|---|
| After | OpenThinker3-7B, made from that model by training on examples of step-by-step solutions. |
At each piece of text (a token), the model carries an internal list of 3,584 numbers. These are its activations; each numbered entry is a dimension. We compare those numbers at the same processing layer and text position.
A chat template is the formatting around the actual question: role names such as user and assistant, special markers, and line breaks. “Ready to answer” below means the final template token, immediately before the model generates its reply.
This is supervised fine-tuning (SFT): learning from example solutions. It is not reinforcement learning with verifiable rewards (RLVR). The model card confirms this training method. “Ordinary model” here means before this reasoning-specific fine-tuning, not a model incapable of reasoning.
01The peaks appear at conversation boundaries
Before fine-tuning, a large activation peak is already present at the line break after system. After fine-tuning, that peak grows and additional peaks appear, including at the line break after assistant—the point where the model is ready to answer.
The chart follows one prompt from left to right. Higher means a larger internal activation vector, not greater importance or better reasoning. Change the prompt or layer, or select a token to inspect it.
Each step up the vertical axis is a 10× increase. Values below 1 are shown at 1. Shaded tokens belong to the conversation template; ↵ means a line break.
How this was measured
We passed eight fixed math questions through both models, using exactly the same token sequences. We saved activations after each of the 28 processing blocks (numbered 0–27). This is a small exploratory sample. The chart shows the full vector’s Euclidean norm, or the largest absolute value among dimensions 458, 2570, and 2718.
Layer 19, shown by default, matches the large-sample analysis below. The prompt-ending check in section 4 instead uses block 18: the model API calls its output hidden_states[19] because it also counts the initial embedding state.
These are activation magnitudes, not a measurement of what fraction of the full before/after difference lies at each token. We did not measure attention weights or run causal interventions. Saved measurements and scan code.
02Three dimensions carry 92.65% of the average shift
At the ready-to-answer position, we compared activations before and after fine-tuning on 30,193 questions. The change is almost the same for every question: one shared offset explains 99.974% of the total squared change.
Think of the offset as moving every point by approximately the same arrow. Of that arrow’s squared length, 92.65% sits in dimensions 458, 2570, and 2718. These are numbered entries in the activation vector, not three identified reasoning skills.
Each “Top” bar includes the dimensions in the bars above it. Shares are of the squared average shift, not all information in the model. “Remaining” covers all 3,579 dimensions outside the top five.
Exact dimension values and calculation
What changes when we measure generated answers instead?
The ready-to-answer shift is about 236× the original activation-vector length. For the average state over the generated answer, the shift is 0.58× the original length (medians over questions). Answer states change in more question-dependent ways.
The answer comparison mixes changes to model weights and changes to the words each model generates. The “before → after thinking” option is a different comparison within OpenThinker: from the prompt ending to its </think> marker. It does not measure the effect of training. Full analysis.
03These are the same massive dimensions seen without reasoning fine-tuning
Sun and colleagues’ Massive Activations in Large Language Models documents unusually large values in a few fixed dimensions of ordinary language models, often at starting tokens and separators such as line breaks. Their values change little across inputs. This establishes that the broader phenomenon does not require reasoning-specific fine-tuning.
We found a match in the actual dimension numbers, not just a similar-looking pattern. Our two strongest dimensions, 458 and 2570, are exactly the massive dimensions listed for Qwen2.5-7B in Raman and colleagues’ Table 1. Their study analyzes ordinary text from WikiText and C4, rather than a before/after reasoning-training comparison.
| Our dimension | Share of average shift | Earlier observation |
|---|---|---|
| 458 | 59.85% | Qwen2.5-7B massive dimension |
| 2570 | 26.90% | Qwen2.5-7B massive dimension |
| 2718 | 5.90% | Also reported for Qwen2.5-7B-Instruct in an independent engineering analysis |
The third match comes from a March 2026 engineering report, which lists spike dimensions [458, 2570, 2718, 2730] for the same Qwen2.5-7B-Instruct parent. It is an author-reported experiment in a GitHub issue, not a peer-reviewed paper. The Raman preprint supports the first two matches; it does not establish the third.
Our interpretation: reasoning fine-tuning amplifies and changes where an existing massive-activation pattern appears. We have evidence for reuse of the same coordinates; we have not shown that these coordinates cause improved reasoning. Dimension numbers are model-specific—the original Sun paper, for example, reports different numbers for LLaMA.
Related reading and why it matters
- Massive Activations in Large Language Models — Sun et al., COLM 2024. The foundational result: a few enormous, largely input-independent activations can shape attention. This motivates checking whether our large shifts reflect the same phenomenon.
- Rethinking the Outlier Distribution in Large Language Models: An In-depth Study — Raman et al., 2025 preprint. Table 1 supplies the exact Qwen2.5 dimension match, 458 and 2570.
- See What You Are Told: Visual Attention Sink in Large Multimodal Models — Kang et al., 2025. A related extension outside reasoning training: Qwen2-VL-7B also uses dimensions 458 and 2570 for the massive activations associated with visual attention sinks.
An “attention sink” is a token that attracts substantial attention even when it carries little relevant content. Prior work connects sinks to massive activations. Our measurements establish large activation values, but do not directly establish attention-sink behavior in OpenThinker.
04The large value depends on which token we measure
We kept each question fixed but appended <think>, the marker that starts OpenThinker’s written reasoning. At the new final token, its activation magnitude was about 1,869–2,006, compared with 23,320 at the original ending. The original model remained near 74–86. This measures a different position; it does not show that the earlier peak disappeared.
| End of the input | Before fine-tuning | After fine-tuning |
|---|
Median activation-vector magnitude over eight questions, layer 18. Each row measures its own final token; ↵ means a line break. Saved control measurements.
What the results do—and do not—tell us
- Supported: very large activations appear at template boundaries, and a few known massive dimensions dominate the shared change at the prompt ending.
- Unresolved: whether these changes help reasoning, are a by-product of training, or mainly support formatting and attention behavior. Big activation values are not a measure of causal importance.
- Scope: the detailed position scan has eight prompts; the dimension result has 30,193 questions. This is one supervised-fine-tuning lineage, not yet a general finding about RLVR or all reasoning models.
Technical consequence: comparing models can be misleading without scale controls
We also tested whether a linear predictor trained to map the original model’s prompt-ending state to its answer state still worked after fine-tuning. On MATH questions, direct transfer gave R² ≈ −49,329: much worse than predicting an average answer state. Allowing an added constant offset improved this to −13.36; an offset plus one scale factor reached approximately zero, meaning no useful predictive gain over the average.
Separately standardizing each model’s input and output coordinates gave R² = −0.174. Because this also changes the output weighting and R² denominator, it is a different metric, not a pure input-scale correction. These checks show why raw transfer failure needs careful interpretation; they do not restore successful transfer. Numerical audit and saved scores.
05The next question is what these dimensions actually do
- Test their causal role. Change the identified dimensions at template boundaries and compare with changes at question-text positions. Measure answer accuracy and generation behavior.
- Separate formatting from reasoning. Vary role names, line breaks, system instructions, and reasoning markers across many prompts.
- Check other training methods. Repeat the same measurements on a matched model pair before and after RLVR, keeping the input text and measurement positions fixed.
Methods and definitions for reproducing the analysis
The large analysis uses 30,193 deduplicated questions across seven datasets spanning math, factual knowledge, and commonsense: MATH, GSM8K train, ContextHub, MMLU, ARC-Challenge, CommonsenseQA, and PIQA. Each model generated one deterministic reply per question, choosing the most likely next token. We measured states after zero-based decoder block 19.
For each question, the training shift is the after-model vector minus the before-model vector. We estimate a shared offset and one common scale factor using training folds, and measure their contribution on held-out questions, reusing five data splits. The 99.974% figure is the share of total squared shift explained by the shared offset. The 92.65% figure is a separate denominator: the squared length of the average shift, restricted to its three largest coordinates.
Source field names: context means the final input-token state; end of thought means the state at </think>; answer mean averages states over the answer following the reasoning trace. A dimension is a raw activation coordinate, not a feature learned by a separate interpretability model. Source files identify this analysis as experiment 2546.
The all-token scan uses eight fixed math prompts with identical token IDs, bfloat16 inference, and all 28 decoder blocks. The prompt-ending control uses block 18, one block earlier than the large analysis. Model revisions: Qwen a09a35458c702b33eeacc393d103063234e8bc28; OpenThinker 864fb9aefbb91f257c4911e4b9f85a6986f8eab2.
Download the report, data, and source code
- Complete handoff bundle: standalone HTML, original measurement JSON, analysis scripts, literature metadata, and SHA-256 checksums.
- Large-sample representation changes: the source of the 99.974% and 92.65% results.
- All-token scan and code: exact prompts, both models’ activations, and layer conventions.
- Prompt-ending control and linear-predictor transfer audit.
- Large-sample analysis code and original experiment collection.
This HTML works offline: charts and measurements are embedded. External citations require internet. The saved data are sufficient to rebuild the report without running the models again.