Post-training a small vision-language model on a MacBook: SFT vs RL on spatial maps



Mac LLM/VLM Post-training Testing · visual question answering (SpatialEval spatial-map)

November 19, 2026 · Qwen3.5-0.8B and 2B with LoRA · Apple M5 MacBook Pro, 16 GB · 9 runs, 13 hours of training and evaluation

The question

The text-only studies showed what SFT and RL do to a small model on a MacBook. Does the same recipe work when the model has to look at an image, and what does it cost the machine?

We used MilaWang/SpatialEval, spatial-map split: each item is a picture of a map with named locations and a multiple-choice question about it, in three types: direction ("in which direction is X relative to Y?"), which object ("which object is in the northwest of Y?") and count ("how many objects are to the east of Y?"). We trained on a small slice, 500 items, and tested on 100 held-out items, because the point is to benchmark what the Mac does, not to chase a leaderboard. The same six recipes as before ran on Qwen3.5-0.8B; three of them were repeated on Qwen3.5-2B. All use LoRA (rank 16) on the language model, with the vision tower frozen.

The short answers

  • Both families of recipe work, and they work fast. On 0.8B, the untrained model scores 39%; after training, SFT reaches 89%, gentle SFT 91%, and the RL methods GRPO 84%, RLOO 88%, GSPO 87%. Scores come from 100 questions and one seed, so differences of up to about ten points are noise.
  • Most of the failure was format and length, not vision. The untrained model wrote 392 tokens per answer on average, 29% of its answers hit the 640-token cap, and 24% never produced the answer line. Training mostly taught it to finish and to commit.
  • Training changes inference cost by an order of magnitude. The same questions cost 1006 generated tokens per correct answer untrained, 9 after SFT and 44 after GSPO; one answer takes 5.6 s untrained and 0.25 s after SFT on the MacBook.
  • The 2B model fits for RL on 16 GB, barely. GRPO peaked at 7.7 GB of MLX memory with up to 6.5 GB of swap, and the first RL step took 43 s against about 32 s for the first five steps of 0.8B. It was not better than 0.8B after RL: 85% vs 84%.
  • The skill transfers a little, and not predictably. On a different task (maze navigation) trained models scored 55%-67% against 25% chance, but the ranking of recipes changes, and one recipe (self-distillation) collapsed.

1. Setup

Images go through a frozen vision tower once; everything we train is the language model.

  • Model path. Qwen3.5 has a hybrid language model (gated delta-net and attention layers) and a vision tower. Maps are resized to 448 px, which becomes exactly 196 image tokens in the prompt (the mean prompt, image plus question and chat template, is 304 tokens). Vision features are computed once per image and cached; LoRA (rank 16, scale 20) sits on the language-model layers only, about 1.25% of the 0.8B parameters.
  • Data. 500 training items and 100 test items, seeded, drawn from the spatial-map shard; question types are mixed as in the source (Direction: 31, Which object: 30, Count: 39 in the test set).
  • Six recipes. SFT (answer-only targets, The answer is: C. Northwest, lr 2e-5, 5 epochs), gentle SFT (lr 5e-6, 2 epochs), self-distilled SFT (rejection-sample the model's own correct answers, then train on them), and three critic-free RL methods: GRPO, RLOO and GSPO (4 prompts × 4 sampled answers per step, temperature 1.0, correctness-only reward that also needs the answer line, no KL penalty, lr 2e-6, 5 epochs = 625 steps).
  • Scoring. An answer counts only if it finishes within 640 new tokens and ends in a parsable The answer is: X line with the right letter. This is strict on purpose: it is also the RL reward.

2. Where the untrained model fails

The 0.8B model scores 39% and its biggest problem is counting.

Untrained 0.8B accuracy and truncation by question type (test set).
accuracyanswers cut off at the 640-token cap
0255075100Directionaccuracy · Direction: 6161answers cut off at the 640-token cap · Direction: 66Which objectaccuracy · Which object: 5757answers cut off at the 640-token cap · Which object: 3737Countaccuracy · Count: 88answers cut off at the 640-token cap · Count: 4141% of the test questions of that type
What the untrained model did on the 100 test questionsCount
correct39
wrong letter30
cut off at the token cap29
finished, no 'The answer is: X' line2

Direction and which-object questions are only partly answered correctly (61% and 57%); count questions are almost never answered correctly (8%), and 41% of them ran into the token cap while the model enumerated objects one by one. Chance is 25% on four options, and the most common gold letter covers 28% of the test set, so a model that only guessed the most common letter would score about 28%.

3. Supervised fine-tuning

Five epochs of answer-only SFT lifts the 0.8B model from 39% to 89%.

The SFT targets contain only the final line, taken from the dataset's oracle option. That is a deliberate simplification: there are no reference reasoning traces for these maps, so SFT teaches the model to answer directly. It works because the model already has the visual skill; it only needed to commit.

Test accuracy over the epochs on 0.8B, all runs (epoch 0 is untrained).
SFTSFT gentleSFT self-distilledGRPORLOOGSPO
0%25%50%75%100%012345SFTSFT: 39% at 0SFT: 47% at 1SFT: 64% at 2SFT: 76% at 3SFT: 86% at 4SFT: 89% at 5SFT gentleSFT gentle: 39% at 0SFT gentle: 85% at 1SFT gentle: 91% at 2SFT self-distilledSFT self-distilled: 39% at 0SFT self-distilled: 49% at 1SFT self-distilled: 63% at 2GRPOGRPO: 39% at 0GRPO: 78% at 1GRPO: 79% at 2GRPO: 83% at 3GRPO: 82% at 4GRPO: 84% at 5RLOORLOO: 39% at 0RLOO: 75% at 1RLOO: 81% at 2RLOO: 90% at 3RLOO: 89% at 4RLOO: 88% at 5GSPOGSPO: 39% at 0GSPO: 84% at 1GSPO: 84% at 2GSPO: 89% at 3GSPO: 88% at 4GSPO: 87% at 5epochtest accuracy

The aggressive recipe (lr 2e-5, 5 epochs) climbs slowly (39%, 47%, 64%, 76%, 86%, 89%); the gentle recipe (lr 5e-6, 2 epochs) gets to 91% in well under half the time. Self-distilled SFT ends at only 63%: it trains on the model's own correct answers, so it can only reinforce what the model already did, including the rambling style, and its training set was thin on the hard type: only 19% of its examples were count questions (88 of 457), against 39% of the test set, because the untrained model rarely got those right.

4. Reinforcement learning

GRPO, RLOO and GSPO need no reference answers, only the correct-or-not reward.

Training reward (share of sampled answers that finish and are correct), 25-step moving average.
GRPORLOOGSPO
0%25%50%75%100%0100200300400500600GRPORLOOGSPOoptimizer steptraining reward (25-step average)

After the first epoch the three RL runs scored GRPO 78%, RLOO 75%, GSPO 84%, and finished at GRPO 84%, RLOO 88%, GSPO 87%. As in the text studies, RL also changes how the model writes, with no length penalty: shorter answers finish more often and so earn more reward.

Mean tokens per sampled answer during RL training (25-step moving average).
GRPORLOOGSPO
02004006000100200300400500600GRPORLOOGSPOoptimizer steptokens per sampled answer (25-step average)
Share of prompt groups where all four sampled answers got the same reward, so the group teaches nothing.
GRPORLOOGSPO
0%25%50%75%100%0100200300400500600GRPORLOOGSPOoptimizer stepshare of prompt groups with no learning signal

By the end, 81% of GRPO's groups, 85% of RLOO's groups, 96% of GSPO's groups carried no signal, which suggests the later epochs did little (we did not test shorter training).

Final results, 0.8B

Final test accuracy by question type, all six recipes.
SFTSFT gentleSFT self-distilledGRPORLOOGSPO
0255075100DirectionSFT · Direction: 9494SFT gentle · Direction: 9494SFT self-distilled · Direction: 7777GRPO · Direction: 7474RLOO · Direction: 9090GSPO · Direction: 9494Which objectSFT · Which object: 9090SFT gentle · Which object: 9797SFT self-distilled · Which object: 7070GRPO · Which object: 9090RLOO · Which object: 8787GSPO · Which object: 9090CountSFT · Count: 8585SFT gentle · Count: 8585SFT self-distilled · Count: 4646GRPO · Count: 8787RLOO · Count: 8787GSPO · Count: 7979final test accuracy, %
Run (0.8B, LoRA)TrainingDirectionWhich objectCountAllTokens / answerTruncatedHours
Untrained–61%57%8%39%39229%–
SFT5 ep94%90%85%89%80%0.5
SFT gentle2 ep94%97%85%91%80%0.2
SFT self-distilled2 ep77%70%46%63%30212%0.4
GRPO5 ep74%90%87%84%1580%3.7
RLOO5 ep90%87%87%88%870%2.6
GSPO5 ep94%90%79%87%381%1.5

5. Does 2B do better?

Slightly better untrained, equal after training.

Run (2B, LoRA)TrainingDirectionWhich objectCountAllTokens / answerTruncatedHours
Untrained–74%77%8%49%32214%–
SFT5 ep100%90%82%90%80%0.7
SFT gentle2 ep97%93%82%90%80%0.3
GRPO5 ep90%83%82%85%530%3.2
Untrained and final accuracy, 0.8B vs 2B, for the three methods run on both.
0.8B2B
0255075100Untrained0.8B · Untrained: 39392B · Untrained: 4949SFT0.8B · SFT: 89892B · SFT: 9090SFT gentle0.8B · SFT gentle: 91912B · SFT gentle: 9090GRPO0.8B · GRPO: 84842B · GRPO: 8585test accuracy, %
Method0.8B: untrained → final2B: untrained → final0.8B hours2B hours
SFT39% → 89%49% → 90%0.50.7
SFT gentle39% → 91%49% → 90%0.20.3
GRPO39% → 84%49% → 85%3.73.2
Test accuracy over the epochs on 2B.
SFTSFT gentleGRPO
0%25%50%75%100%012345SFTSFT: 49% at 0SFT: 76% at 1SFT: 83% at 2SFT: 88% at 3SFT: 91% at 4SFT: 90% at 5SFT gentleSFT gentle: 49% at 0SFT gentle: 87% at 1SFT gentle: 90% at 2GRPOGRPO: 49% at 0GRPO: 89% at 1GRPO: 89% at 2GRPO: 86% at 3GRPO: 85% at 4GRPO: 85% at 5epochtest accuracy (2B)

The 2B model starts higher (49% vs 39%), but after SFT both reach about 89%-90%, and 2B GRPO ended at 85% (best epoch 89%), no better than 0.8B GRPO (84%). With 100 test questions none of these gaps is significant. The honest reading is that on this small, templated task the extra parameters bought little; the cost of the larger model is in section 9.

6. What the models actually write

The same map question, answered by six different trained models.

The test item below asks for the direction of Jackal's Jams relative to Giraffe's Guitars; the right answer is C, Northwest. Answers are truncated to 700 characters.

Untrained · 228 tokens · wrong

To determine the direction of **Jackal's Jams** relative to **Giraffe's Guitars**, we need to look at the positions of these two objects on the map.

- **Jackal's Jams** is located in the **top-left** part of the map.
- **Giraffe's Guitars** is located in the **center** of the map, slightly below the middle.

So, from Giraffe's Guitars, going **up and to the left** leads to Jackal's Jams.

In standard map directions:
- **North** is up
- **East** is right
- **Southwest** is down and left
- **Northeast** is up and right

Since Jackal's Jams is **up and left** of Giraffe's Guitars, that corresponds to **Southwest**.

Let’s check the options:
A. Southwest → ✅ Correct
B. Northeast → Incorrect
C. [...]

SFT · 7 tokens · correct

The answer is: C. Northwest

SFT self-distilled · 182 tokens · correct

To determine the direction of **Jackal's Jams** relative to **Giraffe's Guitars**, we need to look at the positions of both objects on the map.

- **Jackal's Jams** is located in the **upper-left** part of the map.
- **Giraffe's Guitars** is located in the **center** of the map, slightly below the midpoint.

So, from Giraffe's Guitars, going **up and to the left** would place Jackal's Jams in that direction.

Let’s check the options:

A. Southwest → Down and right → Incorrect  
B. Northeast → Up and right → Incorrect  
C. Northwest → Up and left → ✅ Correct  
D. Southeast → Down and right → Incorrect  

Therefore, the correct answer is **C. Northwest**.

The answer is: C

GRPO · 275 tokens · correct

First, locate Giraffe's Guitars in the middle-lower area of the map. Jackal's Jams is located in the upper-left area of the map. From the perspective of looking at the map from the upper-right area (or from the viewer's viewpoint), Giraffe's Guitars serves as the reference point (origin) for the question "relative to Giraffe's Guitars." Jackal's Jams is situated horizontally to the left and vertically above Giraffe's Guitars. "Northwest" describes the location of Jackal's Jams from the viewpoint of Giraffe's Guitars, indicating that Jackal's Jams is located in the Northwest direction from Giraffe's Guitars. "Northeast" would describe a location in the Northeast direction from Giraffe's Guita [...]

RLOO · 88 tokens · correct

- Giraffe’s Guitars is in the southeast of the map.
- Jackal’s Jams is to the northwest of that.
- This places Jackal’s Jams in the northwest of the map.
- This indicates the direction is Northwest.
- Option C is Northwest.

This matches the northwest of the Guitars.

Therefore, the answer is C.
</think>

The answer is C.

GSPO · 31 tokens · correct

Looking to the northwest from Giraffe's Guitars:

The answer is: C. Northwest

Final line: The answer is: C. Northwest

What the examples show

  • The untrained model reasons correctly and then answers wrongly. Above, it correctly says Jackal's Jams is "up and to the left", then maps that to Southwest and ticks option A. Its visual reading was right; the translation to a letter was not. This is why format-and-commit training pays off so quickly.
  • SFT removes the reasoning altogether. Seven tokens: The answer is: C. Northwest. It is the cheapest and, here, among the most accurate recipes, because the model has to read the map and answer in one step.
  • Self-distillation keeps the old style. It still writes a long worked answer, and it still makes the same kind of mistake on counts.
  • The RL methods each found a different style, from GRPO's long paragraph, through RLOO's short bullet list, to GSPO's near-direct answer (158, 87 and 38 tokens on average). None was rewarded for style; they drifted to whatever earned reward at the lowest cost.

Another failure of the untrained model

A different test question, where it picks a letter that contradicts its own working:

Untrained · 217 tokens · wrong

To determine the direction of **Candy Land Confections** relative to **Pawn Stars**, we need to look at the spatial arrangement of the objects on the map.

From the image:

- **Pawn Stars** is located in the **upper right** part of the map.
- **Candy Land Confections** is located in the **lower right** part of the map.

So, from Pawn Stars to Candy Land Confections, you move **down and to the right**.

In standard map directions:
- **North** = top
- **South** = bottom
- **East** = right
- **West** = left

Since the movement is **down (south)** and **right (east)**, this corresponds to **Southeast**.

Let’s check the options:

A. Southwest → would be down and left  
B. Northeast → would be up and right  
C. Southeast → down and right ✅  
D. Northwest → up and left  

The correct answer is **C. Southeast**.

The answer is: C

GSPO · 29 tokens · correct

Looking to the southwest from Candy Land Confections:

The answer is: A. Southwest

Final line: The answer is: A. Southwest

Counting stays the hard part

RunCount questionsAccuracyTruncatedTokensPredicted letters
Untrained398%41%461A: 3, B: 7, C: 12, D: 4, None: 13
SFT3985%0%8A: 11, B: 5, C: 13, D: 10
SFT gentle3985%0%8A: 9, B: 6, C: 15, D: 9
SFT self-distilled3946%18%307A: 6, B: 4, C: 13, D: 13, None: 3
GRPO3987%0%113A: 12, B: 5, C: 11, D: 11
RLOO3987%0%81A: 12, B: 5, C: 11, D: 11
GSPO3979%3%50A: 14, B: 5, C: 11, D: 9

Counting is where the models differ most. The untrained model almost never gets it right; after training, direction and which-object accuracy is high for most recipes, while count accuracy ranges from 46% to 87%. Whether these gains are counting or learned shortcuts is not something 100 questions can settle; the maze test below asks a related question.

Do the recipes agree?

Share of identical answersSFTSFT gentleSFT self-distilledGRPORLOOGSPO
SFT–89%61%83%86%88%
SFT gentle89%–62%82%89%89%
SFT self-distilled61%62%–58%64%62%
GRPO83%82%58%–85%85%
RLOO86%89%64%85%–94%
GSPO88%89%62%85%94%–

Recipes with similar accuracy do not necessarily give the same answers, so the shared score hides different mistakes. This is a reason to prefer larger evaluation sets in follow-up work.

7. Does it transfer? A maze-navigation test

Different image type, different skill, same answer format.

We took 100 random questions from the maze-navigation task of the same dataset ("how many right turns are on the blue path?", four options, chance 25%, majority-letter baseline 32%) and evaluated every adapter without any further training. The prompt and answer line are the same.

0.8B maze-navigation accuracy (no training on mazes).
0.8B
0255075100Untrained0.8B · Untrained: 00SFT0.8B · SFT: 5555SFT gentle0.8B · SFT gentle: 6565SFT self-distilled0.8B · SFT self-distilled: 99GRPO0.8B · GRPO: 6060RLOO0.8B · RLOO: 6767GSPO0.8B · GSPO: 6666% correct on maze-navigation questions
RunAccuracyTruncatedAnswer line presentTokensPredicted letters
0.8B Untrained0%97%3%636A: 1, B: 1, C: 1, null: 97
0.8B SFT55%0%100%8A: 5, B: 45, C: 35, D: 15
0.8B SFT gentle65%0%100%8A: 20, B: 26, C: 30, D: 24
0.8B SFT self-distilled9%80%20%590A: 6, B: 8, C: 2, D: 4, null: 80
0.8B GRPO60%4%96%274A: 21, B: 29, C: 28, D: 18, null: 4
0.8B RLOO67%0%100%98A: 27, B: 23, C: 25, D: 25
0.8B GSPO66%6%97%119A: 22, B: 24, C: 28, D: 23, null: 3
2B Untrained7%89%11%608A: 2, B: 2, C: 2, D: 5, null: 89
2B SFT45%0%100%8A: 49, B: 4, C: 27, D: 20
2B SFT gentle70%0%100%8A: 12, B: 22, C: 38, D: 28
2B GRPO44%29%74%289A: 18, B: 19, C: 18, D: 19, null: 26
  • The untrained model scores near zero mostly because it runs out of tokens (97% truncated at 640): it rambles, and never gets to the answer line. We did not rerun with a larger cap, so this is a statement about the cap.
  • The adapters that learned to answer directly (SFT, gentle SFT) and the RL adapters all score well above chance. Part of this is that they always produce an answer; part may be real transfer. Without a control (for example a blank image) we cannot separate the two.
  • The self-distilled adapter collapses to 9%: its rambling style carried over unchanged and kept hitting the cap.
  • 2B behaves differently: SFT 45%, SFT gentle 70%, GRPO 44%. Its GRPO adapter leaves 29% of answers truncated on mazes.

8. Token-level inference cost

Training changed how many tokens each answer costs, and with them the time per answer.

Mean tokens per answer and tokens per correct answer, at the end of training (test set, greedy).
mean tokens per answertokens per correct answer
025050075010001250Untrainedmean tokens per answer · Untrained: 392392tokens per correct answer · Untrained: 10061006SFTmean tokens per answer · SFT: 88tokens per correct answer · SFT: 99SFT gentlemean tokens per answer · SFT gentle: 88tokens per correct answer · SFT gentle: 99SFT self-distilledmean tokens per answer · SFT self-distilled: 302302tokens per correct answer · SFT self-distilled: 479479GRPOmean tokens per answer · GRPO: 158158tokens per correct answer · GRPO: 188188RLOOmean tokens per answer · RLOO: 8787tokens per correct answer · RLOO: 9999GSPOmean tokens per answer · GSPO: 3838tokens per correct answer · GSPO: 4444tokens

The untrained model writes 392 tokens per answer, 1006 per correct answer. SFT writes 8; GSPO 38; RLOO 87; GRPO 158. Dividing generated tokens by correct answers is the number that matters for cost: 109x fewer for SFT and 23x fewer for GSPO than the untrained model.

Batched evaluation speed

Throughput while evaluating the 100 test questions 25 at a time (batched decode and prefill).
decode tokens/s (25 answers at once)prefill tokens/s ÷ 10
0200400600Untraineddecode tokens/s (25 answers at once) · Untrained: 487487prefill tokens/s ÷ 10 · Untrained: 210210SFTdecode tokens/s (25 answers at once) · SFT: 514514prefill tokens/s ÷ 10 · SFT: 160160SFT gentledecode tokens/s (25 answers at once) · SFT gentle: 502502prefill tokens/s ÷ 10 · SFT gentle: 140140SFT self-distilleddecode tokens/s (25 answers at once) · SFT self-distilled: 434434prefill tokens/s ÷ 10 · SFT self-distilled: 206206GRPOdecode tokens/s (25 answers at once) · GRPO: 439439prefill tokens/s ÷ 10 · GRPO: 209209RLOOdecode tokens/s (25 answers at once) · RLOO: 471471prefill tokens/s ÷ 10 · RLOO: 214214GSPOdecode tokens/s (25 answers at once) · GSPO: 224224prefill tokens/s ÷ 10 · GSPO: 214214tokens / s

These are aggregate numbers for many answers in flight, so they are higher than what one user sees. Batched decode throughput was about 487 tokens/s untrained and 224-514 tokens/s after training; it was lowest for GSPO (224), whose answers are the shortest of the RL runs, probably because with short answers the evaluation is dominated by prefill and start-up rather than steady decoding (not isolated). What matters more is wall-clock time: the 100-question evaluation took 97 s untrained and 24 s after SFT.

Single stream: what one user would see

The same 100 questions, one at a time, greedy, timed per token.

Time to first token and total time per answer, mean over the test set.
time to first token (s)total time per answer (s)
0.002.004.006.008.00Untrainedtime to first token (s) · Untrained: 0.130.13total time per answer (s) · Untrained: 5.635.63SFTtime to first token (s) · SFT: 0.130.13total time per answer (s) · SFT: 0.250.25SFT gentletime to first token (s) · SFT gentle: 0.130.13total time per answer (s) · SFT gentle: 0.250.25SFT self-distilledtime to first token (s) · SFT self-distilled: 0.130.13total time per answer (s) · SFT self-distilled: 4.514.51GRPOtime to first token (s) · GRPO: 0.130.13total time per answer (s) · GRPO: 2.902.90RLOOtime to first token (s) · RLOO: 0.130.13total time per answer (s) · RLOO: 1.501.50GSPOtime to first token (s) · GSPO: 0.130.13total time per answer (s) · GSPO: 0.730.73seconds
Decode speed and prefill speed (÷10), mean over the test set.
decode tokens/sprefill tokens/s ÷ 10
0100200300Untraineddecode tokens/s · Untrained: 6464prefill tokens/s ÷ 10 · Untrained: 248248SFTdecode tokens/s · SFT: 7171prefill tokens/s ÷ 10 · SFT: 250250SFT gentledecode tokens/s · SFT gentle: 7272prefill tokens/s ÷ 10 · SFT gentle: 250250SFT self-distilleddecode tokens/s · SFT self-distilled: 6464prefill tokens/s ÷ 10 · SFT self-distilled: 249249GRPOdecode tokens/s · GRPO: 6464prefill tokens/s ÷ 10 · GRPO: 249249RLOOdecode tokens/s · RLOO: 6464prefill tokens/s ÷ 10 · RLOO: 248248GSPOdecode tokens/s · GSPO: 6666prefill tokens/s ÷ 10 · GSPO: 248248tokens / s
Inter-token latency by token position in the answer: does decoding slow down as the answer grows?
UntrainedSFTSFT gentleSFT self-distilledGRPORLOOGSPO
0.05.010.015.020.00100200300400500UntrainedUntrained: 15.7 at 0Untrained: 15.7 at 8Untrained: 15.8 at 16Untrained: 15.8 at 32Untrained: 15.8 at 64Untrained: 15.8 at 128Untrained: 15.8 at 256Untrained: 15.7 at 512SFTSFT: 15.7 at 0SFT: 15.7 at 8SFT gentleSFT gentle: 15.7 at 0SFT gentle: 15.8 at 8SFT self-distilledSFT self-distilled: 15.7 at 0SFT self-distilled: 15.7 at 8SFT self-distilled: 15.7 at 16SFT self-distilled: 15.8 at 32SFT self-distilled: 15.8 at 64SFT self-distilled: 15.8 at 128SFT self-distilled: 15.8 at 256SFT self-distilled: 15.7 at 512GRPOGRPO: 15.8 at 0GRPO: 15.8 at 8GRPO: 15.8 at 16GRPO: 15.8 at 32GRPO: 15.8 at 64GRPO: 15.8 at 128GRPO: 15.8 at 256RLOORLOO: 15.8 at 0RLOO: 15.8 at 8RLOO: 15.8 at 16RLOO: 15.8 at 32RLOO: 15.8 at 64RLOO: 15.7 at 128GSPOGSPO: 15.7 at 0GSPO: 15.7 at 8GSPO: 15.7 at 16GSPO: 15.8 at 32GSPO: 15.8 at 64GSPO: 15.8 at 128GSPO: 15.8 at 256GSPO: 15.7 at 512token position in the answerms per token
RunAccuracyPrompt tokensGen tokensTTFT sPrefill tok/sDecode tok/sITL p50 msITL p95 msTotal sPeak MLX GB
Untrained42%3043490.12724826415.716.45.632.26
SFT89%30490.12624967115.716.20.252.09
SFT gentle90%30490.12624977215.716.10.252.09
SFT self-distilled59%3042790.12624926415.716.34.512.09
GRPO82%3041770.12624916415.716.42.902.09
RLOO87%304880.12724776415.716.41.502.09
GSPO88%304390.12624826615.716.20.732.09
  • The image costs a fixed ~0.13 s. Time to first token is the same for every run (0.126-0.127 s): prefill of about 304 tokens at about 2482 tokens/s, which includes the vision tower.
  • Decoding is flat. Inter-token latency is about 15.7 ms (p95 16.4 ms) at every position from the first token to the 512th; so within 512 tokens the cost per token did not grow with the answer length (we did not test longer ones). Single-stream decode speed is 64-72 tokens/s for every adapter.
  • So answer length is the whole story. End-to-end time per answer falls from 5.63 s (untrained) to 0.25 s (SFT), almost entirely because of the number of tokens, since speed per token did not change. After SFT, 0.126 of the 0.25 s is the image prefill.
  • Memory at inference is small. Peak MLX memory was about 2.3 GB for 0.8B with the LoRA adapter applied.

9. How the MacBook performed

Accuracy is only half of a benchmark. This chapter reads the system counters logged at every step to see what the machine did while it trained: where the time went, how busy the GPU and CPU were, and whether the laptop ever slowed down.

These are the nine LoRA runs of this study: six on Qwen3.5-0.8B and three on Qwen3.5-2B. The vision tower is frozen and its features are computed once per image, so the training steps are language-model steps over 196 image tokens plus the text; the cost of the image shows up as longer sequences, not as extra vision compute.

69%GPU busy, average over all steps
14%CPU busy, average over all steps
100%lowest thermal speed limit seen in 13 h of training
14.1 GBhighest system RAM used (of 16 GB)
4.5-9.2 GBswap in use during runs
7.7 GBhighest MLX (GPU) memory

Across 9 runs (13 hours of training in total) the GPU was busy 69% of the time on average and the CPU only 14%: training is a GPU job, and the CPU mostly feeds it. The thermal speed limit never dropped below 100%, so the laptop never throttled, even through multi-hour runs on a plugged-in laptop.

An RL step is mostly generation

Each RL step first samples 16 answers and then trains on them. Sampling took 68-81% of the step time (74% on average). Sampling is memory-bandwidth bound exactly as in the inference benchmark, so the lever that matters is how many tokens you generate, not how clever the optimiser is. Batching helps a lot: 16 sequences decoded together reach 175-377 tokens/s in total, against about 64 tokens/s for a single stream on the same model.

Mean seconds per RL step, split into sampling (generation) and the policy update (training).
generationtraining
0.05.010.015.020.00.8B GRPOgeneration · 0.8B GRPO: 14.814.8training · 0.8B GRPO: 5.95.90.8B RLOOgeneration · 0.8B RLOO: 9.79.7training · 0.8B RLOO: 4.64.60.8B GSPOgeneration · 0.8B GSPO: 6.26.2training · 0.8B GSPO: 2.12.12B GRPOgeneration · 2B GRPO: 14.514.5training · 2B GRPO: 3.43.4seconds per step

Throughput

Tokens per second: generation during RL sampling (batch of 16) and training (forward + backward) for every run.
generationtraining
01002003004005000.8B SFTtraining · 0.8B SFT: 16160.8B SFT gentletraining · 0.8B SFT gentle: 16160.8B SFT self-distilledtraining · 0.8B SFT self-distilled: 2362360.8B GRPOgeneration · 0.8B GRPO: 374374training · 0.8B GRPO: 3343340.8B RLOOgeneration · 0.8B RLOO: 377377training · 0.8B RLOO: 3843840.8B GSPOgeneration · 0.8B GSPO: 376376training · 0.8B GSPO: 2062062B SFTtraining · 2B SFT: 11112B SFT gentletraining · 2B SFT gentle: 11112B GRPOgeneration · 2B GRPO: 175175training · 2B GRPO: 160160tokens / s

GPU and CPU load

GPU utilisation sat between 31% and 87% for every run, while the CPU stayed at 9-18%.

Average GPU and CPU utilisation per run (system counters sampled during every step).
GPU busy %CPU busy %
02550751000.8B SFTGPU busy % · 0.8B SFT: 6868CPU busy % · 0.8B SFT: 17170.8B SFT gentleGPU busy % · 0.8B SFT gentle: 7272CPU busy % · 0.8B SFT gentle: 18180.8B SFT self-distilledGPU busy % · 0.8B SFT self-distilled: 8787CPU busy % · 0.8B SFT self-distilled: 12120.8B GRPOGPU busy % · 0.8B GRPO: 8686CPU busy % · 0.8B GRPO: 14140.8B RLOOGPU busy % · 0.8B RLOO: 8282CPU busy % · 0.8B RLOO: 11110.8B GSPOGPU busy % · 0.8B GSPO: 8787CPU busy % · 0.8B GSPO: 10102B SFTGPU busy % · 2B SFT: 3131CPU busy % · 2B SFT: 17172B SFT gentleGPU busy % · 2B SFT gentle: 3131CPU busy % · 2B SFT gentle: 17172B GRPOGPU busy % · 2B GRPO: 8282CPU busy % · 2B GRPO: 99% busy

Memory: always at the limit

System RAM read 13.9-14.1 GB in every run, almost the whole 16 GB, and the swap file held 4.5-9.2 GB. This is mostly macOS using memory for caches and other apps, so it says more about the machine than about training, but it also means there is no slack: the runs that peaked at 7.7 GB of MLX memory had only a few GB of headroom. The steady swap use did not visibly slow the runs.

What each run cost

RunStepsHourss / stepSampling shareGen tok/sTrain tok/sGPUCPUPeak MLX GBRAM GBSwap GBGain (pts)Pts / hour
0.8B SFT6250.52.5––1668%17%3.714.19.2+50+101.7
0.8B SFT gentle2500.22.4––1672%18%3.714.08.9+52+250.8
0.8B SFT self-distilled2300.45.5––23687%12%7.413.94.5+24+56.5
0.8B GRPO6253.720.772%37433486%14%7.314.17.9+45+12.1
0.8B RLOO6252.614.468%37738482%11%7.414.05.2+49+19.0
0.8B GSPO6251.58.475%37620687%10%7.313.94.7+48+31.5
2B SFT6250.73.3––1131%17%6.713.97.0+41+59.6
2B SFT gentle2500.33.3––1131%17%6.713.97.2+41+136.8
2B GRPO6253.217.981%17516082%9%7.714.06.5+36+11.2

Gain is final minus untrained test accuracy in points; points per hour divides it by the wall-clock hours of the whole run, evaluations included (one evaluation of the test set takes about 61 s). Energy and battery use were not measured: that needs administrator access to the power counters.

Takeaway. On this MacBook training is GPU-bound and thermally comfortable; the cost of RL is almost all sampling time, so it scales with how many tokens the model writes, and the memory headroom, not the clock speed, is the constraint to plan around.

0.8B versus 2B on the MacBook

Runs / stepof which generationof which traininggen tok/strain tok/speak MLX GBRAM GB (max)swap GB (max)hours
0.8B GRPO20.714.85.93743347.314.17.93.7
2B GRPO17.914.53.41751607.714.06.53.2
0.8B SFT2.5–––163.714.19.20.5
0.8B SFT gentle2.4–––163.714.08.90.2
0.8B SFT self-distilled5.5–––2367.413.94.50.4
0.8B RLOO14.49.74.63773847.414.05.22.6
0.8B GSPO8.46.22.13762067.313.94.71.5
2B SFT3.3–––116.713.97.00.7
2B SFT gentle3.3–––116.713.97.20.3
Seconds per RL step, GRPO: 0.8B vs 2B (25-step moving average).
0.8B GRPO2B GRPO
02040608001002003004005006000.8B GRPO2B GRPOoptimizer stepseconds per RL step (25-step average)

On average a 2B GRPO step took 18 s against 21 s for 0.8B, with generation 81% of the step (72% for 0.8B). The first 2B step took 43 s because untrained answers were long; steps got cheaper as the model learned to answer briefly, the same effect seen at 0.8B, and the 2B run took 3.2 h in total for 625 steps.

Where it ran out of room

Swap in use during GRPO: the 2B run pushed several GB to disk.
0.8B GRPO2B GRPO
0.02.04.06.08.010.001002003004005006000.8B GRPO2B GRPOoptimizer stepswap in use, GB
Wall-clock hours per run (training plus evaluations).
hours
0.01.02.03.04.05.00.8B SFThours · 0.8B SFT: 0.50.50.8B SFT gentlehours · 0.8B SFT gentle: 0.20.20.8B SFT self-distilledhours · 0.8B SFT self-distilled: 0.40.40.8B GRPOhours · 0.8B GRPO: 3.73.70.8B RLOOhours · 0.8B RLOO: 2.62.60.8B GSPOhours · 0.8B GSPO: 1.51.52B SFThours · 2B SFT: 0.70.72B SFT gentlehours · 2B SFT gentle: 0.30.32B GRPOhours · 2B GRPO: 3.23.2hours (training + evaluation)
  • Memory is the limit, not compute. The 2B GRPO run needed micro-batch 1 and peaked at 7.7 GB of MLX memory with system RAM at 14.0 GB and up to 6.5 GB of swap. It finished, but it left little headroom: longer answers, more images per prompt or a bigger model would push hard on 16 GB. We did not try to train 4B or larger models, so where exactly the limit lies is untested.
  • Images cost sequence length. Each prompt carries 196 image tokens (and about 300 tokens in total), so the sequence limit was 1,100 tokens and the images were downscaled to 448 px; the number of images per example was kept at one.
  • No thermal throttling. As in the text studies, the laptop never slowed itself, even through multi-hour RL runs.
  • Evaluation is not free. One 100-question evaluation of the untrained 2B model took about 160 s, as long as several RL steps, because untrained answers are long. RL training also spent most of each step in generation, so cost scales with answer length, not with the optimiser.
  • Everything was measured with one machine and one seed. The timing numbers should be read as typical for this laptop under this setup, not as hardware limits.

10. Conclusions

  1. On a small visual QA task, a few hundred examples and a few hours were enough to take a 0.8B model from 39% to roughly 85-90%, with either SFT or RL.
  2. Most of the gain is format and commitment: finishing within the token budget and producing the answer line. Reading the image was mostly already possible.
  3. RL needs no reference answers and learns its own short style; SFT is cheaper (minutes) when targets are as simple as a letter.
  4. Self-distillation did not work here: it cannot teach what the model never gets right.
  5. Training reduced cost per correct answer by roughly 109x with the cheapest recipe, because per-token speed is constant and answer length is what changed.
  6. The 2B model fits for RL on a 16 GB MacBook, but only just, and it did not beat 0.8B on this task.

Caveats

One seed per recipe and 100 test questions (±10 points at 95%). A single dataset slice of a templated, synthetic task. The maze-navigation results have no control (blank image, larger token cap), so transfer is suggestive only. Single-stream speed was measured on 0.8B only; the 2B figures are training and batched-evaluation numbers. We froze the vision tower; training it was not attempted.

Appendix

A. Accuracy by epoch, all runs

RunEpochAllDirectionWhich objectCountTokensTruncated
0.8B SFT039%61%57%8%39229%
0.8B SFT147%32%17%82%80%
0.8B SFT264%32%90%69%80%
0.8B SFT376%71%80%77%80%
0.8B SFT486%90%87%82%80%
0.8B SFT589%94%90%85%80%
0.8B SFT gentle039%61%57%8%39229%
0.8B SFT gentle185%77%90%87%80%
0.8B SFT gentle291%94%97%85%80%
0.8B SFT self-distilled039%61%57%8%39229%
0.8B SFT self-distilled149%68%73%15%32512%
0.8B SFT self-distilled263%77%70%46%30212%
0.8B GRPO039%61%57%8%39229%
0.8B GRPO178%65%80%87%2231%
0.8B GRPO279%65%87%85%2290%
0.8B GRPO383%71%90%87%1641%
0.8B GRPO482%81%83%82%1630%
0.8B GRPO584%74%90%87%1580%
0.8B RLOO039%61%57%8%39229%
0.8B RLOO175%74%67%82%1090%
0.8B RLOO281%84%80%79%1720%
0.8B RLOO390%87%97%87%860%
0.8B RLOO489%94%87%87%870%
0.8B RLOO588%90%87%87%870%
0.8B GSPO039%61%57%8%39229%
0.8B GSPO184%81%83%87%570%
0.8B GSPO284%84%90%79%340%
0.8B GSPO389%97%87%85%340%
0.8B GSPO488%94%90%82%381%
0.8B GSPO587%94%90%79%381%
2B SFT049%74%77%8%32214%
2B SFT176%58%83%85%80%
2B SFT283%71%97%82%80%
2B SFT388%97%90%79%80%
2B SFT491%100%90%85%80%
2B SFT590%100%90%82%80%
2B SFT gentle049%74%77%8%32214%
2B SFT gentle187%84%97%82%80%
2B SFT gentle290%97%93%82%80%
2B GRPO049%74%77%8%32214%
2B GRPO189%90%97%82%420%
2B GRPO289%97%87%85%290%
2B GRPO386%94%80%85%330%
2B GRPO485%94%80%82%490%
2B GRPO585%90%83%82%530%

B. Method and data notes

  • Data: SpatialEval spatial-map, vision-QA split, shard 0; 500 train / 100 test, seed 0; images resized to 448 px (longest side). Maze test: SpatialEval maze-navigation, 100 random questions, seed 0.
  • Hyperparameters: LoRA rank 16, scale 20 (language model only); SFT lr 2e-5 (5 epochs); gentle and self-distilled SFT lr 5e-6 (2 epochs); RL lr 2e-6, 4 prompts × 4 rollouts per step, temperature 1.0, up to 640 new tokens, no KL, 5 epochs; evaluation greedy, 100 questions, 25 at a time.
  • Software: MLX 0.32 and mlx-vlm 0.7.12 on macOS 26, Apple M5, 16 GB; the vision features are cached per image and the language model runs in training mode so the differentiable gated-delta kernel is used.
  • W&B project: macbook-posttrain-spatial. Metrics, rollouts and evaluation answers are kept under runs_spatial/; adapter weights were not committed.
  • Other pages: token and speed analytics (0.8B) · analytics (2B).




Share this article




Copyright © 2022-2026 Azmine Toushik Wasi. All rights reserved.