Mac LLM/VLM Post-training Testing · math-story-problems
November 18, 2026 · Qwen3.5-0.8B with LoRA · Apple M5 MacBook Pro, 16 GB · 6 runs
Code on GitHub · report.html source · Companion: GSM8K post · inference repo · inference post
On GSM8K, the model's own reasoning was already strong, and supervised fine-tuning made it worse. Would the picture change on a dataset where the answers are short and the failure mode is different?
We used azminetoushikwasi/math-story-problems. Every item is one arithmetic problem, three numbers joined by two plus or minus signs, written four ways: a bare equation, a short story, a story with distractor item types, and a long story with many irrelevant sentences. The answer is always a single integer. That makes it a clean test of one skill: pull the right numbers out of a story and calculate.
We trained on 500 items (each shown in one randomly chosen story form) and tested on 100 held-out items in all four forms, so each score rests on 400 prompts. Six recipes were compared on the same model: three kinds of supervised fine-tuning (SFT) and three reinforcement-learning (RL) methods.
It can do the arithmetic. It cannot cope with long stories.
Accuracy falls steadily as the story gets longer: 90% for the bare equation, 74% for the short story, 54% with distractor items, and 8% for the long story. In the long-story form the model writes about 484 tokens on average, and 65% of its answers are cut off at the 512-token cap, so it is failing as much by rambling as by getting things wrong.
Imitating short, correct arithmetic lifts the model from 56% to as high as 97.8%.
The dataset has only final answers, so for SFT we generated the reference solutions from the equation: 2385 + 761 = 3146, then 3146 - 1063 = 2083, then The answer is: 2083. Three variants were tried: the original recipe (learning rate 2e-5, 5 epochs), a gentle one (5e-6, 2 epochs), and self-distillation, which trains on the model's own correct finished answers.
The equation-derived solutions are a very different story from GSM8K's human-written ones. The model learns a ~42-token style it can carry out reliably, and the long-story form jumps from 8% to 96%. The gentle recipe gets there in a single epoch and never shows the early dip on bare equations that the original recipe does. Self-distillation, by contrast, only reinforces what the model already does: its data covers only the 65% of questions the model already solves, so it adds little where the model is weakest (the long story reaches 20%).
The RL methods reach the top of the dataset, and they get there fast.
GRPO, RLOO and GSPO see only the questions and a correct-or-not reward: no reference solutions, no hints about the equation. Learning rate 2e-6, four sampled answers per question, up to 640 new tokens.
GRPO scored 96% on the test set after a single epoch and RLOO scored 99.5% on the test set after a single epoch and GSPO scored 95% on the test set after a single epoch. By the end GRPO reached 99%, RLOO reached 98%, GSPO reached 99.2%. At this level (about 400 prompts, one seed) the differences between methods are within noise; what is clear is that RL matches the best SFT recipe without needing its reference solutions.
Because there is no length penalty, the answer length shrinking is a by-product: shorter answers finish more often and so earn more reward.
Once the model answers almost everything correctly, most question groups have four identical rewards and therefore no gradient: by the end, 98% (GRPO), 98.8% (RLOO), 97.8% (GSPO) of groups carry no signal. Training keeps running, but very little changes after the first epochs.
| Run | Status | Equation | Short story | Distractor items | Long story | All | Tokens | Truncated | Hours |
|---|---|---|---|---|---|---|---|---|---|
| Untrained | – | 90% | 74% | 54% | 8% | 56% | 302 | 18% | – |
| SFT (equation solutions) | done | 89% | 95% | 96% | 96% | 94% | 42 | 0% | 0.3 |
| SFT gentle | done | 96% | 99% | 98% | 98% | 97.8% | 42 | 0% | 0.2 |
| SFT self-distilled | done | 83% | 90% | 61% | 20% | 64% | 286 | 14% | 0.4 |
| GRPO | done | 99% | 99% | 99% | 99% | 99% | 149 | 0% | 1.6 |
| RLOO | done | 97% | 98% | 98% | 99% | 98% | 73 | 0% | 1.1 |
| GSPO | done | 99% | 100% | 99% | 99% | 99.2% | 223 | 0% | 2.3 |
The same recipes behave very differently on GSM8K and on math-story-problems.
| Method (LoRA) | GSM8K (untrained → final) | GSM8K hours | math-story (untrained → final) | math-story hours |
|---|---|---|---|---|
| SFT, original recipe (human solutions on GSM8K; equation solutions here) | 54% → 22% | 0.3 | 56% → 94% | 0.3 |
| SFT, gentle recipe | 54% → 37% | 0.1 | 56% → 97.8% | 0.2 |
| SFT, self-distilled | 54% → 54% | 0.3 | 56% → 64% | 0.4 |
| GRPO | 54% → 64% | 3.8 | 56% → 99% | 1.6 |
| RLOO | 54% → 67% | 3.4 | 56% → 98% | 1.1 |
| GSPO | 54% → 59% | 3.1 | 56% → 99.2% | 2.3 |
On GSM8K the model already reasons well in its own long, step-by-step style, supervised fine-tuning on human solutions pulled it away from that style, and RL gave gains of 5 to 13 points (up to 16 with full fine-tuning). Here the untrained model is held back by rambling, a short templated style is something it can execute, and so both SFT on clean solutions and RL give very large gains. The shared lesson: post-training helps when it teaches something the model can actually do, and the cheapest reliable signal is a verifiable reward.
Accuracy is only half of a benchmark. This chapter reads the system counters logged at every step to see what the machine did while it trained: where the time went, how busy the GPU and CPU were, and whether the laptop ever slowed down.
These are the six 0.8B LoRA runs on math-story. Compared with the GSM8K runs of the same methods, answers are much shorter, so an RL step took 8 s on average instead of 19 s, and the whole RL run 1.7 h instead of 3.4 h.
Across 6 runs (6 hours of training in total) the GPU was busy 74% of the time on average and the CPU only 14%: training is a GPU job, and the CPU mostly feeds it. The thermal speed limit never dropped below 100%, so the laptop never throttled, even through multi-hour runs on a plugged-in laptop.
Each RL step first samples 16 answers and then trains on them. Sampling took 85-87% of the step time (86% on average). Sampling is memory-bandwidth bound exactly as in the inference benchmark, so the lever that matters is how many tokens you generate, not how clever the optimiser is. Batching helps a lot: 16 sequences decoded together reach 451-499 tokens/s in total, against about 72 tokens/s for a single stream on the same model.
GPU utilisation sat between 51% and 92% for every run, while the CPU stayed at 9-20%.
System RAM read 13.6-14.1 GB in every run, almost the whole 16 GB, and the swap file held 3.0-4.2 GB. This is mostly macOS using memory for caches and other apps, so it says more about the machine than about training, but it also means there is no slack: the runs that peaked at 7.0 GB of MLX memory had only a few GB of headroom. The steady swap use did not visibly slow the runs.
| Run | Steps | Hours | s / step | Sampling share | Gen tok/s | Train tok/s | GPU | CPU | Peak MLX GB | RAM GB | Swap GB | Gain (pts) | Pts / hour |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SFT | 625 | 0.3 | 1.1 | – | – | 170 | 53% | 16% | 4.0 | 13.6 | 3.1 | +38 | +127.8 |
| SFT gentle | 250 | 0.2 | 1.1 | – | – | 165 | 51% | 20% | 3.8 | 13.8 | 4.2 | +41 | +271.6 |
| SFT self-distilled | 276 | 0.4 | 3.5 | – | – | 349 | 92% | 14% | 7.0 | 13.7 | 3.0 | +7 | +16.6 |
| GRPO | 625 | 1.6 | 7.9 | 87% | 451 | 125 | 84% | 14% | 6.8 | 14.0 | 4.2 | +43 | +27.1 |
| RLOO | 625 | 1.1 | 5.2 | 85% | 499 | 118 | 82% | 9% | 6.9 | 13.9 | 3.3 | +42 | +39.5 |
| GSPO | 625 | 2.3 | 11.9 | 86% | 480 | 160 | 82% | 9% | 6.8 | 14.1 | 3.9 | +43 | +18.3 |
Gain is final minus untrained test accuracy in points; points per hour divides it by the wall-clock hours of the whole run, evaluations included (one evaluation of the test set takes about 118 s). Energy and battery use were not measured: that needs administrator access to the power counters.
One seed per recipe and 400 test prompts (100 per form), so differences of a few points are not meaningful near the 98-100% ceiling. The SFT targets are an easy deterministic template. Test equations never appear in the training data, but the problem templates do, so this measures skill on this family of problems, not general math. Late in some RL runs (e.g. RLOO), accuracy slipped a few points as answers kept shrinking; the final-epoch score is reported.
| Run | Epoch | Equation | Short story | Distractor items | Long story | All | Tokens | Truncated |
|---|---|---|---|---|---|---|---|---|
| SFT (equation solutions) | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| SFT (equation solutions) | 1 | 56% | 67% | 68% | 68% | 65% | 43 | 0% |
| SFT (equation solutions) | 2 | 69% | 89% | 87% | 89% | 84% | 42 | 0% |
| SFT (equation solutions) | 3 | 87% | 94% | 93% | 94% | 92% | 42 | 0% |
| SFT (equation solutions) | 4 | 89% | 95% | 95% | 95% | 94% | 42 | 0% |
| SFT (equation solutions) | 5 | 89% | 95% | 96% | 96% | 94% | 42 | 0% |
| SFT gentle | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| SFT gentle | 1 | 97% | 97% | 97% | 97% | 97% | 42 | 0% |
| SFT gentle | 2 | 96% | 99% | 98% | 98% | 97.8% | 42 | 0% |
| SFT self-distilled | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| SFT self-distilled | 1 | 89% | 92% | 83% | 10% | 68% | 283 | 10% |
| SFT self-distilled | 2 | 83% | 90% | 61% | 20% | 64% | 286 | 14% |
| GRPO | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| GRPO | 1 | 97% | 98% | 96% | 91% | 96% | 137 | 0% |
| GRPO | 2 | 97% | 97% | 97% | 97% | 97% | 137 | 0% |
| GRPO | 3 | 97% | 98% | 98% | 99% | 98% | 147 | 0% |
| GRPO | 4 | 99% | 100% | 99% | 99% | 99.2% | 149 | 0% |
| GRPO | 5 | 99% | 99% | 99% | 99% | 99% | 149 | 0% |
| RLOO | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| RLOO | 1 | 99% | 100% | 100% | 99% | 99.5% | 123 | 0% |
| RLOO | 2 | 100% | 100% | 100% | 100% | 100% | 95 | 0% |
| RLOO | 3 | 99% | 100% | 100% | 99% | 99.5% | 72 | 0% |
| RLOO | 4 | 97% | 98% | 98% | 99% | 98% | 73 | 0% |
| RLOO | 5 | 97% | 98% | 98% | 99% | 98% | 73 | 0% |
| GSPO | 0 | 90% | 74% | 54% | 8% | 56% | 302 | 18% |
| GSPO | 1 | 100% | 99% | 97% | 84% | 95% | 232 | 1% |
| GSPO | 2 | 96% | 99% | 98% | 96% | 97% | 262 | 0% |
| GSPO | 3 | 100% | 100% | 99% | 97% | 99% | 228 | 0% |
| GSPO | 4 | 99% | 100% | 99% | 99% | 99.2% | 221 | 0% |
| GSPO | 5 | 99% | 100% | 99% | 99% | 99.2% | 223 | 0% |
macbook-posttrain-mathstory; all metrics, rollouts and evaluation answers are kept under runs_mathstory/; model weights were deleted locally after the runs (re-run train.py to regenerate). See also the research journal (JOURNAL.md) and the GSM8K report.Copyright © 2022-2026 Azmine Toushik Wasi. All rights reserved.