Post-training Qwen3.5 on a 16 GB MacBook: what works, what breaks



Mac LLM/VLM Post-training Testing

November 18, 2026 · Apple M5 MacBook Pro, 16 GB · MLX · 9 of 9 runs finished · 0.8B, GSM8K

The question

Our companion post measured how far a 16 GB MacBook goes at running Qwen3.5. This one asks the harder question: how far does it go at training it?

Running a model streams weights through the chip once per token. Training does that and more: it keeps activations for the backward pass, gradients, optimizer state, and for reinforcement learning (RL) it also has to generate whole batches of answers before it can learn from them. On a laptop where macOS alone takes about 6.5 GB of the 16, each of those costs matters.

We set out to compare four post-training methods on the same task: SFT (imitate reference solutions), and three critic-free RL methods, GRPO, RLOO and GSPO (learn from the model's own answers, rewarded when correct). All on the Qwen3.5-0.8B, with LoRA adapters and (for the RL methods) with full fine-tuning; we also tried two gentler SFT recipes after the first one failed. The task is GSM8K grade-school math: 500 training questions for 5 epochs, scored on 100 held-out test questions, with one line per (model, method, adapter) in Weights & Biases and every rollout kept.

All planned runs have finished. A good deal of what broke on the way turned out to be the most informative part, so this post tells that story as well as the final scores. The larger models (2B, 4B) were planned but not trained: see the memory chapter for why we expect them not to fit as configured. A companion post repeats the experiment on a dataset with much shorter answers.

The short version

  • Memory is the first wall, and it comes much sooner than for inference. A single 0.8B training step peaked at 15.8 GB before we turned on gradient checkpointing, and 6.9 GB after.
  • Training required re-implementing a layer. Qwen3.5's fast attention kernel has no gradient; the library fallback was 50x too slow.
  • Three separate hyperparameter and reward mistakes each wrecked a run, and each looked like "the method doesn't work" until we read the curves.
  • SFT on 500 human solutions makes this model worse. On the 0.8B, LoRA SFT took accuracy from 54% to 22% after five epochs (it was 17% after the first and peaked at 29% in between), while the training loss fell to about 0.01. A gentler recipe lost less (37%), and training on the model's own correct answers kept it at its starting 54% but taught nothing new.
  • RL is what improved it. With LoRA, GRPO, RLOO and GSPO ended at 64%, 67% and 59% (from 54%); with full fine-tuning 68%, 68% and 70%. On 100 test questions the differences between the RL methods are within noise; the gap to SFT is not.

1. The setup, in plain words

  • SFT: show the model a reference solution and train it to reproduce it token by token.
  • GRPO: sample 4 answers per question, reward the correct ones, and nudge the model toward answers that did better than their group average.
  • RLOO: same sampling, but each answer is compared to the average of the other answers for that question (a leave-one-out baseline), with a plain policy gradient.
  • GSPO: like GRPO, but the update is weighted at the whole-sequence level instead of per token.
  • LoRA trains small adapter matrices (10.8M parameters on the 0.8B) on top of frozen weights; full fine-tuning updates every transformer weight and needs fp32 master weights and Adam state (about 14 bytes per parameter).

All RL methods here are critic-free and have no reference-model KL penalty, so only one model sits in memory. Each RL step uses 4 prompts x 4 rollouts at temperature 1.0 and up to 640 new tokens. The learning rates are 2e-5 (SFT, LoRA), 1e-5 (SFT, full), 2e-6 (RL, LoRA) and 1e-6 (RL, full). All runs use the full 500 samples x 5 epochs on the 0.8B. The gentler SFT variants use 5e-6 for 2 epochs ("SFT gentle") and rejection-sampled, self-generated correct answers ("SFT self-distilled"). Full-fine-tuning SFT was not completed.

2. The first thing that broke: no gradients through Qwen3.5

Qwen3.5 is not a plain transformer. Most of its layers are "gated delta" linear-attention layers, and MLX ships a fast Metal kernel for them that has no backward pass.

The first training step crashed with Primitive::vjp not implemented for CustomKernel. The library's pure-operations fallback works, but it loops over tokens in Python: 50 seconds for one step on two short sequences, which would have made the whole project impossible. We wrote a chunk-parallel, matmul-based version of the recurrence (pt/gdn.py) that autograd can differentiate. It went through two bugs on the way (a numerically unstable matrix inverse that produced garbage on real, correlated keys, replaced by forward substitution) and now matches the reference: 4% relative error in the gradients on the real model, the same loss at step 1 (1.09 on both paths), and about 1 second per SFT step.

Takeaway. On Apple silicon, "can I fine-tune model X?" can depend on whether every layer type has a backward kernel. Check before you plan.

3. SFT: imitation made the model worse

On the 0.8B, LoRA SFT took accuracy from 54% to 22% after five epochs (it was 17% after the first and peaked at 29% in between), while the training loss fell to about 0.01. The model memorised the 500 reference solutions and lost the stronger, more verbose reasoning style it already had.

SFT training loss. It falls to ~0.01: the model has memorised the 500 solutions.
0.8B SFT lora
00.511.5201002003004005006000.8B SFT loraoptimizer steptraining loss (10-step avg)
Qwen3.5-0.8B: GSM8K test accuracy (first 100 test questions, greedy) after each epoch. Epoch 0 is the untrained model. Dashed = full fine-tuning.
SFT loraSFT gentle loraSFT self-distilled loraGRPO fullGRPO loraRLOO fullRLOO loraGSPO fullGSPO lora
0%25%50%75%100%012345SFT loraSFT lora: 54% at 0SFT lora: 17% at 1SFT lora: 29% at 2SFT lora: 27% at 3SFT lora: 24% at 4SFT lora: 22% at 5SFT gentle loraSFT gentle lora: 54% at 0SFT gentle lora: 32% at 1SFT gentle lora: 37% at 2SFT self-distilled loraSFT self-distilled lora: 54% at 0SFT self-distilled lora: 46% at 1SFT self-distilled lora: 54% at 2GRPO fullGRPO full: 54% at 0GRPO full: 48% at 1GRPO full: 57% at 2GRPO full: 60% at 3GRPO full: 62% at 4GRPO full: 68% at 5GRPO loraGRPO lora: 54% at 0GRPO lora: 56% at 1GRPO lora: 56% at 2GRPO lora: 63% at 3GRPO lora: 63% at 4GRPO lora: 64% at 5RLOO fullRLOO full: 54% at 0RLOO full: 57% at 1RLOO full: 51% at 2RLOO full: 62% at 3RLOO full: 61% at 4RLOO full: 68% at 5RLOO loraRLOO lora: 54% at 0RLOO lora: 48% at 1RLOO lora: 57% at 2RLOO lora: 66% at 3RLOO lora: 60% at 4RLOO lora: 67% at 5GSPO fullGSPO full: 54% at 0GSPO full: 59% at 1GSPO full: 72% at 2GSPO full: 61% at 3GSPO full: 65% at 4GSPO full: 70% at 5GSPO loraGSPO lora: 54% at 0GSPO lora: 52% at 1GSPO lora: 53% at 2GSPO lora: 62% at 3GSPO lora: 66% at 4GSPO lora: 59% at 5epochaccuracy

Our first attempt, at a learning rate of 1e-4, was far worse: accuracy collapsed from 54% to 4% with degenerate repeated output. LoRA with scale 20 is very sensitive; 2e-5 trains stably but still lowers accuracy. A reasonable reading: Qwen3.5 already reasons well in its own style, and 500 terse human-written solutions over five epochs pull it toward a style it cannot execute as well. That is a finding, not a bug: it is exactly the situation RL is supposed to improve on, because RL trains on the model's own outputs.

4. RL: two ways to teach a model to stop thinking

Both of our early GRPO failures were the same disease: the model found a way to get reward without being correct, or was pushed too hard by noise.

Reward hacking

The first GRPO run collapsed from 54% to 1% accuracy within the first epoch. The reason is in the reward. We capped generation at 400 tokens, and this model is verbose: 44% of rollouts were truncated and scored 0. A short, wrong answer in the right format earned a 0.1 format bonus, which beat 0. Within 50 steps the policy had learned that skipping the reasoning pays: completion length fell from 288 to about 35 tokens and every group looked the same. Fix: reward correctness only (the format flag is logged but not rewarded) and allow 640 tokens.

Learning-rate drift

The second attempt no longer collapsed in length, but reward and format rate drifted down steadily (0.42 to 0.1 and 0.89 to 0.39 over 25 steps) at a learning rate of 5e-6. With only 16 rollouts per step, and about half the groups carrying no signal, small noisy updates accumulate quickly under LoRA scale 20. At 2e-6 the run is stable. We verified the gradient code against the reference before blaming the hyperparameters.

RL mean reward on the 0.8B (25-step moving average). A flat or rising line is what we want to see; both failed attempts fell.
0.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO lora
0%25%50%75%100%01002003004005006000.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO loraoptimizer stepmean reward (25-step average)
RL completion length: shorter completions can mean fewer truncations or can mean the policy abandoned its reasoning; read it together with the reward.
0.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO lora
010020030040050060001002003004005006000.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO loraoptimizer stepmean completion length (tokens, 25-step avg)
Takeaway. In RL on a small model, read the reward and the completion-length curves in the first 50 steps. Reward hacking and drift show up there long before the accuracy number does. All discarded attempts are kept in W&B (group discarded) and under runs_discarded/.

5. What changed besides accuracy

Accuracy is one number. The answers themselves change too: their format, length and style, and some of those changes are side effects nobody asked for.

The clearest example is the answer format. Our prompt ends with "End with a final line: 'The answer is: <number>'", and the untrained model follows it almost always. During RL we stopped rewarding that format (after it had been exploited, chapter 4). The result: in the sampled training rollouts the share ending with "The answer is: N" fell from about 0.8 to about 0.13 over the first two to three epochs, and on the greedy test answers it fell from 86% to 56% by epoch 2 (so far, in the GRPO run), while accuracy did not suffer, because the evaluator reads the last number in the answer. The model silently stopped following a formatting instruction that nothing rewarded any more. That is harmless for this benchmark and could be a real problem in a pipeline that parses the format. GRPO also gradually dropped LaTeX (80% of answers at epoch 0, 36% at epoch 2) while keeping its markdown headers.

SFT changes the style far more. After SFT the model stops writing the long, markdown-heavy, LaTeX-heavy step-by-step answers it started with (95% and 80% of answers) and writes terse plain-text solutions like the GSM8K references: mean length falls from 347 to about 90 tokens and markdown and LaTeX disappear entirely. It follows the "The answer is:" format more reliably (99-100%), but gets fewer answers right, and in the first epochs it sometimes degenerates into a repeating arithmetic loop that runs to the token cap (see the example below). So the SFT story is as much about style as about accuracy: the model traded its own, stronger way of reasoning for a shorter one it executes worse.

Share of the 100 test answers that end with the requested "The answer is: N" line, by epoch (epoch 0 = untrained).
0.8B SFT lora0.8B SFT gentle lora0.8B SFT self-distilled lora0.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO lora
0%25%50%75%100%0123450.8B SFT lora0.8B SFT lora: 86% at 00.8B SFT lora: 78% at 10.8B SFT lora: 91% at 20.8B SFT lora: 100% at 30.8B SFT lora: 99% at 40.8B SFT lora: 99% at 50.8B SFT gentle lora0.8B SFT gentle lora: 86% at 00.8B SFT gentle lora: 94% at 10.8B SFT gentle lora: 99% at 20.8B SFT self-distilled lora0.8B SFT self-distilled lora: 86% at 00.8B SFT self-distilled lora: 90% at 10.8B SFT self-distilled lora: 90% at 20.8B GRPO full0.8B GRPO full: 86% at 00.8B GRPO full: 95% at 10.8B GRPO full: 84% at 20.8B GRPO full: 88% at 30.8B GRPO full: 83% at 40.8B GRPO full: 83% at 50.8B GRPO lora0.8B GRPO lora: 86% at 00.8B GRPO lora: 92% at 10.8B GRPO lora: 56% at 20.8B GRPO lora: 18% at 30.8B GRPO lora: 0% at 40.8B GRPO lora: 0% at 50.8B RLOO full0.8B RLOO full: 86% at 00.8B RLOO full: 95% at 10.8B RLOO full: 27% at 20.8B RLOO full: 0% at 30.8B RLOO full: 0% at 40.8B RLOO full: 0% at 50.8B RLOO lora0.8B RLOO lora: 86% at 00.8B RLOO lora: 96% at 10.8B RLOO lora: 93% at 20.8B RLOO lora: 84% at 30.8B RLOO lora: 83% at 40.8B RLOO lora: 89% at 50.8B GSPO full0.8B GSPO full: 86% at 00.8B GSPO full: 93% at 10.8B GSPO full: 94% at 20.8B GSPO full: 9% at 30.8B GSPO full: 8% at 40.8B GSPO full: 4% at 50.8B GSPO lora0.8B GSPO lora: 86% at 00.8B GSPO lora: 98% at 10.8B GSPO lora: 88% at 20.8B GSPO lora: 93% at 30.8B GSPO lora: 93% at 40.8B GSPO lora: 88% at 5epochshare of test answers ending with "The answer is: N"
Mean generated tokens per test answer, by epoch: how verbose the model is.
0.8B SFT lora0.8B SFT gentle lora0.8B SFT self-distilled lora0.8B GRPO full0.8B GRPO lora0.8B RLOO full0.8B RLOO lora0.8B GSPO full0.8B GSPO lora
01002003004005000123450.8B SFT lora0.8B SFT lora: 347.25 at 00.8B SFT lora: 166.15 at 10.8B SFT lora: 119.96 at 20.8B SFT lora: 86.96 at 30.8B SFT lora: 89.19 at 40.8B SFT lora: 92.17 at 50.8B SFT gentle lora0.8B SFT gentle lora: 347.25 at 00.8B SFT gentle lora: 109.56 at 10.8B SFT gentle lora: 87.54 at 20.8B SFT self-distilled lora0.8B SFT self-distilled lora: 347.25 at 00.8B SFT self-distilled lora: 323.26 at 10.8B SFT self-distilled lora: 342.66 at 20.8B GRPO full0.8B GRPO full: 347.25 at 00.8B GRPO full: 194.5 at 10.8B GRPO full: 290.6 at 20.8B GRPO full: 312.82 at 30.8B GRPO full: 320.43 at 40.8B GRPO full: 321.77 at 50.8B GRPO lora0.8B GRPO lora: 347.25 at 00.8B GRPO lora: 288.67 at 10.8B GRPO lora: 376.73 at 20.8B GRPO lora: 346.98 at 30.8B GRPO lora: 335.62 at 40.8B GRPO lora: 338.74 at 50.8B RLOO full0.8B RLOO full: 347.25 at 00.8B RLOO full: 273.58 at 10.8B RLOO full: 317.5 at 20.8B RLOO full: 264.94 at 30.8B RLOO full: 288.26 at 40.8B RLOO full: 290.65 at 50.8B RLOO lora0.8B RLOO lora: 347.25 at 00.8B RLOO lora: 217.34 at 10.8B RLOO lora: 220.47 at 20.8B RLOO lora: 299.72 at 30.8B RLOO lora: 315.56 at 40.8B RLOO lora: 295.23 at 50.8B GSPO full0.8B GSPO full: 347.25 at 00.8B GSPO full: 213.55 at 10.8B GSPO full: 208.68 at 20.8B GSPO full: 254.55 at 30.8B GSPO full: 271.57 at 40.8B GSPO full: 238.03 at 50.8B GSPO lora0.8B GSPO lora: 347.25 at 00.8B GSPO lora: 187.3 at 10.8B GSPO lora: 281.95 at 20.8B GSPO lora: 294.8 at 30.8B GSPO lora: 317.84 at 40.8B GSPO lora: 320.64 at 5epochmean generated tokens per test answer

Same question, before and after: 0.8B SFT lora

Test question 14; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 55 (gold 60) · wrong · 484 tokens
…porary}) = 55\%$$
So, 55% of the total students are in hip-hop dance.

**Step 4: Calculate the percentage of the entire students.**
We know the total number of students is 20.
$$55\% \text{ of } 20 = 0.55 \times 20 = 11$$
So, 11 students are in hip-hop dance.

**Step 5: Calculate the percentage of the entire students.**
$$\frac{11}{20} = 0.55$$
Converting to a percentage:
$$0.55 \times 100 = 55\%$$

The answer is: 55
epoch 5 · answer 25 (gold 60) · wrong · 512 tokens
…x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x 16 = 10 - 10 x 25% x 16 = 10 - 25% x 16 = 10 - 4 x 25% x

Same question, before and after: 0.8B SFT gentle lora

Test question 85; the end of the answer at epoch 0 (untrained) and at epoch 2.

epoch 0 · answer 11 (gold 44) · wrong · 301 tokens
…ock."
    Therefore, the number of blocks per vacation is 1.
    $$ \text{Blocks per vacation} = 1 $$

3.  **Calculate the total number of blocks:**
    To find the total number of blocks, multiply the number of vacations by the number of blocks per vacation.
    $$ \text{Total blocks} = \text{Number of vacations} \times \text{Blocks per vacation} $$
    $$ \text{Total blocks} = 11 \times 1 = 11 $$

The answer is: 11
epoch 2 · answer 11 (gold 44) · wrong · 512 tokens
…since he was 23 years old, so he has been on 11 vacations.
He has been on vacation since he was 23 years old, so he has been on 11 vacations.
He has been on vacation since he was 23 years old, so he has been on 11 vacations.
He has been on vacation since he was 23 years old, so he has been on 11 vacations.
He has been on vacation since he was 23 years old, so he has been on 11 vacations.
He has been on vacation since

Same question, before and after: 0.8B SFT self-distilled lora

Test question 8; the end of the answer at epoch 0 (untrained) and at epoch 2.

epoch 0 · answer 135 (gold 45) · wrong · 428 tokens
…s} - 2.5 \text{ hours} = 1.5 \text{ hours}$
*   **Speed:** 80 mph
*   **Distance:** $1.5 \text{ hours} \times 80 \text{ mph} = 120 \text{ miles}$

**Step 4: Calculate the total distance**
*   Total Distance = (Distance from Step 1) + (Distance from Step 2) + (Distance from Step 3)
*   Total Distance = $0 \text{ miles} + 15 \text{ miles} + 120 \text{ miles}$
*   Total Distance = $135 \text{ miles}$

The answer is: 135
epoch 2 · answer 80 (gold 45) · wrong · 512 tokens
…ies to get home in 4 hours but spends the first 2 hours in standstill traffic."
    *   Does he drive the *remaining* 4 hours in the fourth segment?
        *   Time spent driving so far: 3 hours + 0.5 hours = 3.5 hours.
        *   Total time available: 4 hours.
        *   Remaining time to drive: $4 \text{ hours} - 3.5 \text{ hours} = 0.5 \text{ hours}$.
    *   So, the fourth segment is indeed 0.5 hours at 80 mph

Same question, before and after: 0.8B GRPO full

Test question 4; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 40 (gold 20) · wrong · 282 tokens
…{ cups} $$

**Step 3: Calculate the total feed needed for the final meal.**
To find the total amount needed for the final meal, we add the morning feed and the afternoon feed.
$$ \text{Total Feed} = \text{Morning Feed} + \text{Afternoon Feed} $$
$$ \text{Total Feed} = 15 + 25 $$
$$ \text{Total Feed} = 40 \text{ cups} $$

Wendi needs to give her chickens 40 cups of feed in the final meal of the day.

The answer is: 40
epoch 5 · answer 13.33 (gold 20) · wrong · 512 tokens
… her chickens in the **final meal** of the day?"

Let's re-parse the timeline:
1.  **Morning:** 15 cups.
2.  **Afternoon:** 25 cups.
3.  **Total given:** 15 + 25 = 40 cups.
4.  **Number of meals:** 3.
5.  **Cups per meal:** 3 cups (per chicken).

So, 40 cups total for 3 meals = 13.33 cups per meal.

The question asks for the amount in the **final meal**.

Total meals = 3
Total cups = 40
Cups per meal = 40 / 3 = 13.33

Same question, before and after: 0.8B GRPO lora

Test question 0; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 18 (gold 18) · correct · 382 tokens
…the number of eggs remaining.**
Subtract the eaten and baked eggs from the total laid.
$$16 \text{ (total)} - 3 \text{ (eaten)} - 4 \text{ (baked)} = 9 \text{ eggs remaining}$$

**Step 4: Calculate the revenue from the remaining eggs.**
She sells the remainder at $2 per fresh duck egg.
$$9 \text{ eggs} \times \$2/\text{egg} = \$18$$

**Conclusion:**
Janet makes $18 every day at the farmers' market.

The answer is: 18
epoch 5 · answer 18 (gold 18) · correct · 407 tokens
…Step 8: Calculate the number of ducks.**
She has 9 eggs remaining. Therefore, she sells 9 ducks.

**Step 9: Calculate the total revenue.**
She sells 9 ducks at $2 per egg.
Total = $9 \times 2 = 18$ dollars.

**Summary of the day:**
- 16 eggs laid.
- 3 eaten (breakfast).
- 4 baked.
- 9 left.

**Calculation:**
$16 - 3 - 4 = 9$.

**Revenue:**
9 ducks $\times$ $2 = $18.

The answer is 18.

**Final line:** 18.

\boxed{18}

Same question, before and after: 0.8B RLOO full

Test question 0; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 18 (gold 18) · correct · 382 tokens
…the number of eggs remaining.**
Subtract the eaten and baked eggs from the total laid.
$$16 \text{ (total)} - 3 \text{ (eaten)} - 4 \text{ (baked)} = 9 \text{ eggs remaining}$$

**Step 4: Calculate the revenue from the remaining eggs.**
She sells the remainder at $2 per fresh duck egg.
$$9 \text{ eggs} \times \$2/\text{egg} = \$18$$

**Conclusion:**
Janet makes $18 every day at the farmers' market.

The answer is: 18
epoch 5 · answer 18 (gold 18) · correct · 182 tokens
…y**: 16 eggs.
2.  **Eaten for breakfast**: 3 eggs.
3.  **Baked muffins**: 4 muffins (muffins do not produce eggs, so they don't count towards the market sales).
4.  **Remaining eggs**: $16 - 3 - 4 = 9$ eggs.
5.  **Price per egg**: $2 per fresh duck egg.
6.  **Total revenue from market**: $9 \text{ (eggs)} \times \$2 \text{ (price per egg)} = \$18$.

The answer is that Janet makes $18 every day at the farmers' market.

Same question, before and after: 0.8B RLOO lora

Test question 5; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 64 (gold 64) · correct · 444 tokens
…oup: $5 \times 8 = 40$
*   Cost of second group: $3 \times 8 = 24$

Total Cost = $40 + 24 = 64$

**Alternative Method (Summing the arithmetic series):**
The cost of the first group is an arithmetic sequence: $5, 5, 5, \dots, 5$ (8 terms).
Sum = $8 \times 5 = 40$.

The cost of the second group is an arithmetic sequence: $3, 3, 3, \dots, 3$ (8 terms).
Sum = $8 \times 3 = 24$.

Total = $40 + 24 = 64$.

The answer is: 64
epoch 5 · answer 6 (gold 64) · wrong · 512 tokens
…applies to "every second glass".
   Indices $2, 4, 6, 8, 10, 12, 14, 16$ are multiples of 2.
   For these indices, the price is $60\%$ of $5 = 3$.
   For indices $1, 3, 5, 7, 9, 11, 13, 15$, the price is $5$.
5. Calculate the sum:
   Sum = (price for 1st) + (price for 2nd) + ... + (price for 16th).
   Count of multiples of 2 is 8.
   Number of non-multiples is $8$.
   Sum = $8 \times 3 + 8 \times 5 = 24 + 40 = 64$.
6

Same question, before and after: 0.8B GSPO full

Test question 0; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 18 (gold 18) · correct · 382 tokens
…the number of eggs remaining.**
Subtract the eaten and baked eggs from the total laid.
$$16 \text{ (total)} - 3 \text{ (eaten)} - 4 \text{ (baked)} = 9 \text{ eggs remaining}$$

**Step 4: Calculate the revenue from the remaining eggs.**
She sells the remainder at $2 per fresh duck egg.
$$9 \text{ eggs} \times \$2/\text{egg} = \$18$$

**Conclusion:**
Janet makes $18 every day at the farmers' market.

The answer is: 18
epoch 5 · answer 18 (gold 18) · correct · 178 tokens
…:

$$
16 - 3 = 13 \text{ eggs remaining}
$$

These **13 eggs** are used to bake muffins for her friends every day with **4 muffins per day**.

Now, calculate how many eggs remain after baking muffins:

$$
13 - 4 = 9 \text{ eggs remaining}
$$

These **9 eggs** are sold at the farmers' market for **$\$2$ per egg**.

To find the total daily income from selling the remainder:

$$
9 \times 2 = 18
$$

The answer is: **18**

Same question, before and after: 0.8B GSPO lora

Test question 5; the end of the answer at epoch 0 (untrained) and at epoch 5.

epoch 0 · answer 64 (gold 64) · correct · 444 tokens
…oup: $5 \times 8 = 40$
*   Cost of second group: $3 \times 8 = 24$

Total Cost = $40 + 24 = 64$

**Alternative Method (Summing the arithmetic series):**
The cost of the first group is an arithmetic sequence: $5, 5, 5, \dots, 5$ (8 terms).
Sum = $8 \times 5 = 40$.

The cost of the second group is an arithmetic sequence: $3, 3, 3, \dots, 3$ (8 terms).
Sum = $8 \times 3 = 24$.

Total = $40 + 24 = 64$.

The answer is: 64
epoch 5 · answer 8 (gold 64) · wrong · 512 tokens
…Pair 3 (Glasses 5 and 6)**:
- Glass 5: $5 \times 0.60 = 3$
- Glass 6: $3$

- **Pair 4 (Glasses 7 and 8)**:
- Glass 7: $5 \times 0.60 = 3$
- Glass 8: $3$

- **Pair 5 (Glasses 9 and 10)**:
- Glass 9: $5 \times 0.60 = 3$
- Glass 10: $3$

- **Pair 6 (Glasses 11 and 12)**:
- Glass 11: $5 \times 0.60 = 3$
- Glass 12: $3$

- **Pair 7 (Glasses 13 and 14)**:
- Glass 13: $5 \times 0.60 = 3$
- Glass 14: $3$

- **Pair 8 (Glasses

Style statistics per run and epoch

ModelMethodAdapterEpochAccuracyTruncatedMean tokens"The answer is:"\boxed{}Markdown bold/headersLaTeX
0.8BSFTlora054%14%34786%0%95%80%
0.8BSFTlora117%22%16678%0%0%0%
0.8BSFTlora229%8%12091%0%0%0%
0.8BSFTlora327%0%87100%0%0%0%
0.8BSFTlora424%1%8999%0%0%0%
0.8BSFTlora522%1%9299%0%0%0%
0.8BSFT gentlelora054%14%34786%0%95%80%
0.8BSFT gentlelora132%6%11094%0%0%0%
0.8BSFT gentlelora237%1%8899%0%0%0%
0.8BSFT self-distilledlora054%14%34786%0%95%80%
0.8BSFT self-distilledlora146%10%32390%0%97%76%
0.8BSFT self-distilledlora254%10%34390%0%97%78%
0.8BGRPOfull054%14%34786%0%95%80%
0.8BGRPOfull148%5%19495%0%89%71%
0.8BGRPOfull257%15%29184%0%100%23%
0.8BGRPOfull360%10%31388%0%100%1%
0.8BGRPOfull462%15%32083%0%100%8%
0.8BGRPOfull568%15%32283%0%100%4%
0.8BGRPOlora054%14%34786%0%95%80%
0.8BGRPOlora156%8%28992%0%95%67%
0.8BGRPOlora256%27%37756%0%99%36%
0.8BGRPOlora363%20%34718%7%75%39%
0.8BGRPOlora463%15%3360%85%94%35%
0.8BGRPOlora564%13%3390%87%95%43%
0.8BRLOOfull054%14%34786%0%95%80%
0.8BRLOOfull157%5%27495%0%95%84%
0.8BRLOOfull251%13%31827%0%89%100%
0.8BRLOOfull362%10%2650%0%86%89%
0.8BRLOOfull461%13%2880%0%88%83%
0.8BRLOOfull568%14%2910%0%88%82%
0.8BRLOOlora054%14%34786%0%95%80%
0.8BRLOOlora148%6%21796%0%0%11%
0.8BRLOOlora257%6%22093%0%67%19%
0.8BRLOOlora366%17%30084%0%97%26%
0.8BRLOOlora460%17%31683%0%99%14%
0.8BRLOOlora567%12%29589%0%98%14%
0.8BGSPOfull054%14%34786%0%95%80%
0.8BGSPOfull159%3%21493%4%57%96%
0.8BGSPOfull272%5%20994%0%3%81%
0.8BGSPOfull361%8%2559%10%95%99%
0.8BGSPOfull465%10%2728%5%100%100%
0.8BGSPOfull570%10%2384%2%100%100%
0.8BGSPOlora054%14%34786%0%95%80%
0.8BGSPOlora152%2%18798%0%37%95%
0.8BGSPOlora253%12%28288%0%93%99%
0.8BGSPOlora362%8%29593%0%100%99%
0.8BGSPOlora466%7%31893%0%100%99%
0.8BGSPOlora559%12%32188%0%100%98%
Takeaway. Track style alongside accuracy: format-following, length, markdown and LaTeX use. A reward that ignores a behaviour lets that behaviour drift, and a benchmark score will not tell you.

6. Memory: the wall comes sooner

A training step that peaked at almost 16 GB on a model with 1.6 GB of weights.

While the second GRPO attempt was running, system swap climbed to 5 GB and step times swung between 11 and 46 seconds. Profiling one forward and backward pass explained it: without gradient checkpointing, the activations of the 0.8B for a batch of two 800-token sequences needed almost the entire 16 GB. Turning on checkpointing (recomputing activations in the backward pass) cut that to 6.9 GB at the cost of some speed, and the restarted GRPO run peaked at 6.8 GB.

Peak MLX memory of one LoRA training step on the 0.8B (tests/profile_memory.py). Batch size and gradient checkpointing change it by 2-3x.
peak MLX memory (GB)
0.05.010.015.020.0usableSFT-like, micro-batch 2peak MLX memory (GB) · SFT-like, micro-batch 2: 8.88.8RL-like, micro-batch 2peak MLX memory (GB) · RL-like, micro-batch 2: 15.815.8RL-like, micro-batch 1peak MLX memory (GB) · RL-like, micro-batch 1: 8.88.8RL-like, micro-batch 2 + ckptpeak MLX memory (GB) · RL-like, micro-batch 2 + ckpt: 6.96.9RL-like, micro-batch 4 + ckptpeak MLX memory (GB) · RL-like, micro-batch 4 + ckpt: 12.212.2GB

This is the number to carry to the larger models, and it is not reassuring. Even with checkpointing, the 0.8B needs about 5 GB of working memory on top of its 1.6 GB of weights. The 4B has 8.5 GB of weights and bigger activations, so as configured it very likely does not fit in the ≈9.5 GB we have: it would need a micro-batch of 1, shorter sequences, or a more memory-efficient backward pass for the gated-delta layers. We therefore did not train the 2B or 4B: with RL at roughly 3 hours per 0.8B run, the time cost alone would have been days per run, on top of the memory risk.

Peak MLX memory, system RAM and swap per run, against the 16 GB limit.
peak MLX (GPU) memorymax system RAM usedmax swap
0.05.010.015.020.016 GB unified memory0.8B SFT lorapeak MLX (GPU) memory · 0.8B SFT lora: 9.89.8max system RAM used · 0.8B SFT lora: 13.913.9max swap · 0.8B SFT lora: 3.63.60.8B SFT gentle lorapeak MLX (GPU) memory · 0.8B SFT gentle lora: 4.44.4max system RAM used · 0.8B SFT gentle lora: 13.413.4max swap · 0.8B SFT gentle lora: 2.52.50.8B SFT self-distilled lorapeak MLX (GPU) memory · 0.8B SFT self-distilled lora: 7.17.1max system RAM used · 0.8B SFT self-distilled lora: 13.613.6max swap · 0.8B SFT self-distilled lora: 3.53.50.8B GRPO fullpeak MLX (GPU) memory · 0.8B GRPO full: 11.111.1max system RAM used · 0.8B GRPO full: 14.014.0max swap · 0.8B GRPO full: 4.54.50.8B GRPO lorapeak MLX (GPU) memory · 0.8B GRPO lora: 6.86.8max system RAM used · 0.8B GRPO lora: 14.014.0max swap · 0.8B GRPO lora: 4.14.10.8B RLOO fullpeak MLX (GPU) memory · 0.8B RLOO full: 11.111.1max system RAM used · 0.8B RLOO full: 14.014.0max swap · 0.8B RLOO full: 4.64.60.8B RLOO lorapeak MLX (GPU) memory · 0.8B RLOO lora: 6.96.9max system RAM used · 0.8B RLOO lora: 13.913.9max swap · 0.8B RLOO lora: 7.97.90.8B GSPO fullpeak MLX (GPU) memory · 0.8B GSPO full: 11.111.1max system RAM used · 0.8B GSPO full: 14.014.0max swap · 0.8B GSPO full: 4.44.40.8B GSPO lorapeak MLX (GPU) memory · 0.8B GSPO lora: 6.86.8max system RAM used · 0.8B GSPO lora: 14.014.0max swap · 0.8B GSPO lora: 3.63.6GB

7. What a run costs in time

SFT is cheap; RL is generation-bound and slow.

An SFT step (4 sequences) takes about a second, so five epochs of 500 questions finish in roughly 15 minutes. An RL step must first generate 16 answers of up to 640 tokens at decode speeds of a few hundred tokens/s, then train on them: 15-45 seconds on the 0.8B. A 625-step RL run took roughly 3-4 hours with LoRA and about 3 hours with full fine-tuning on the 0.8B; larger models would be several times slower.

Mean seconds per optimizer step (SFT: 4 sequences; RL: 4 prompts x 4 rollouts).
mean seconds per optimizer step
0.05.010.015.020.025.00.8B SFT loramean seconds per optimizer step · 0.8B SFT lora: 1.31.30.8B SFT gentle loramean seconds per optimizer step · 0.8B SFT gentle lora: 1.61.60.8B SFT self-distilled loramean seconds per optimizer step · 0.8B SFT self-distilled lora: 3.73.70.8B GRPO fullmean seconds per optimizer step · 0.8B GRPO full: 17.117.10.8B GRPO loramean seconds per optimizer step · 0.8B GRPO lora: 21.621.60.8B RLOO fullmean seconds per optimizer step · 0.8B RLOO full: 15.815.80.8B RLOO loramean seconds per optimizer step · 0.8B RLOO lora: 18.918.90.8B GSPO fullmean seconds per optimizer step · 0.8B GSPO full: 16.016.00.8B GSPO loramean seconds per optimizer step · 0.8B GSPO lora: 17.517.5s / step
Wall-clock hours per run.
hours so far
0.01.02.03.04.05.00.8B SFT lorahours so far · 0.8B SFT lora: 0.30.30.8B SFT gentle lorahours so far · 0.8B SFT gentle lora: 0.10.10.8B SFT self-distilled lorahours so far · 0.8B SFT self-distilled lora: 0.30.30.8B GRPO fullhours so far · 0.8B GRPO full: 3.03.00.8B GRPO lorahours so far · 0.8B GRPO lora: 3.83.80.8B RLOO fullhours so far · 0.8B RLOO full: 2.82.80.8B RLOO lorahours so far · 0.8B RLOO lora: 3.43.40.8B GSPO fullhours so far · 0.8B GSPO full: 2.82.80.8B GSPO lorahours so far · 0.8B GSPO lora: 3.13.1hours

8. How the MacBook performed

Accuracy is only half of a benchmark. This chapter reads the system counters logged at every step to see what the machine did while it trained: where the time went, how busy the GPU and CPU were, and whether the laptop ever slowed down.

All numbers below are from the 0.8B runs on GSM8K: 500 questions for 5 epochs (2 for the gentle SFT variants), one run at a time on the 16 GB MacBook.

81%GPU busy, average over all steps
12%CPU busy, average over all steps
100%lowest thermal speed limit seen in 20 h of training
14.0 GBhighest system RAM used (of 16 GB)
2.5-7.9 GBswap in use during runs
11.1 GBhighest MLX (GPU) memory

Across 9 runs (20 hours of training in total) the GPU was busy 81% of the time on average and the CPU only 12%: training is a GPU job, and the CPU mostly feeds it. The thermal speed limit never dropped below 100%, so the laptop never throttled, even through multi-hour runs on a plugged-in laptop.

An RL step is mostly generation

Each RL step first samples 16 answers and then trains on them. Sampling took 64-73% of the step time (68% on average). Sampling is memory-bandwidth bound exactly as in the inference benchmark, so the lever that matters is how many tokens you generate, not how clever the optimiser is. Batching helps a lot: 16 sequences decoded together reach 381-516 tokens/s in total, against about 72 tokens/s for a single stream on the same model.

Mean seconds per RL step, split into sampling (generation) and the policy update (training).
generationtraining
0.05.010.015.020.0GRPO fullgeneration · GRPO full: 10.910.9training · GRPO full: 6.26.2GRPO lorageneration · GRPO lora: 15.015.0training · GRPO lora: 6.66.6RLOO fullgeneration · RLOO full: 10.410.4training · RLOO full: 5.55.5RLOO lorageneration · RLOO lora: 13.713.7training · RLOO lora: 5.25.2GSPO fullgeneration · GSPO full: 10.310.3training · GSPO full: 5.75.7GSPO lorageneration · GSPO lora: 12.412.4training · GSPO lora: 5.05.0seconds per step

Throughput

Tokens per second: generation during RL sampling (batch of 16) and training (forward + backward) for every run.
generationtraining
0200400600SFT loratraining · SFT lora: 295295SFT gentle loratraining · SFT gentle lora: 233233SFT self-distilled loratraining · SFT self-distilled lora: 352352GRPO fullgeneration · GRPO full: 516516training · GRPO full: 424424GRPO lorageneration · GRPO lora: 427427training · GRPO lora: 425425RLOO fullgeneration · RLOO full: 500500training · RLOO full: 413413RLOO lorageneration · RLOO lora: 381381training · RLOO lora: 407407GSPO fullgeneration · GSPO full: 488488training · GSPO full: 424424GSPO lorageneration · GSPO lora: 396396training · GSPO lora: 462462tokens / s

GPU and CPU load

GPU utilisation sat between 71% and 94% for every run, while the CPU stayed at 9-21%. Full fine-tuning kept the GPU less busy than LoRA in the RL runs (73% vs 85%): a likely reason is memory pressure: with roughly 11 GB of MLX memory there is less room to keep the GPU fed (we did not isolate the cause).

Average GPU and CPU utilisation per run (system counters sampled during every step).
GPU busy %CPU busy %
0255075100SFT loraGPU busy % · SFT lora: 8686CPU busy % · SFT lora: 2121SFT gentle loraGPU busy % · SFT gentle lora: 8181CPU busy % · SFT gentle lora: 1717SFT self-distilled loraGPU busy % · SFT self-distilled lora: 9494CPU busy % · SFT self-distilled lora: 1313GRPO fullGPU busy % · GRPO full: 7474CPU busy % · GRPO full: 1111GRPO loraGPU busy % · GRPO lora: 8686CPU busy % · GRPO lora: 99RLOO fullGPU busy % · RLOO full: 7474CPU busy % · RLOO full: 99RLOO loraGPU busy % · RLOO lora: 8585CPU busy % · RLOO lora: 99GSPO fullGPU busy % · GSPO full: 7171CPU busy % · GSPO full: 1010GSPO loraGPU busy % · GSPO lora: 8383CPU busy % · GSPO lora: 99% busy

Memory: always at the limit

System RAM read 13.4-14.0 GB in every run, almost the whole 16 GB, and the swap file held 2.5-7.9 GB. This is mostly macOS using memory for caches and other apps, so it says more about the machine than about training, but it also means there is no slack: the runs that peaked at 11.1 GB of MLX memory had only a few GB of headroom. The steady swap use did not visibly slow the runs. The one real slow-down we saw came from a different cause, an activation footprint too large for memory before we turned on gradient checkpointing (chapter 6).

What each run cost

RunStepsHourss / stepSampling shareGen tok/sTrain tok/sGPUCPUPeak MLX GBRAM GBSwap GBGain (pts)Pts / hour
SFT lora6250.31.3––29586%21%9.813.93.6-32-113.2
SFT gentle lora2500.11.6––23381%17%4.413.42.5-17-116.9
SFT self-distilled lora2680.33.7––35294%13%7.113.63.5+0+0.0
GRPO full6253.017.164%51642474%11%11.114.04.5+14+4.6
GRPO lora6253.821.670%42742586%9%6.814.04.1+10+2.6
RLOO full6252.815.866%50041374%9%11.114.04.6+14+5.0
RLOO lora6253.418.973%38140785%9%6.913.97.9+13+3.9
GSPO full6252.816.065%48842471%10%11.114.04.4+16+5.6
GSPO lora6253.117.571%39646283%9%6.814.03.6+5+1.6

Gain is final minus untrained test accuracy in points; points per hour divides it by the wall-clock hours of the whole run, evaluations included (one evaluation of the test set takes about 44 s). Energy and battery use were not measured: that needs administrator access to the power counters.

Takeaway. On this MacBook training is GPU-bound and thermally comfortable; the cost of RL is almost all sampling time, so it scales with how many tokens the model writes, and the memory headroom, not the clock speed, is the constraint to plan around.

9. So what can this MacBook train?

Small models, comfortably, with LoRA; the full fine-tune of even the smallest model is a stretch.

  • Budget: macOS takes ≈6.5 GB, leaving ≈9.5 GB for weights, gradients, optimizer state, activations and (for RL) the generation cache.
  • LoRA is the practical route. The adapter is ~11M parameters, so optimizer state is negligible; the real costs are the frozen weights and the activations (use gradient checkpointing and micro-batches of 1-2).
  • Full fine-tuning needs about 14 bytes per parameter (bf16 weights, fp32 master copy, two fp32 Adam moments): ≈11 GB for the 0.8B, so it only fits the smallest model and leaves little room for activations. Larger models are out of reach for full fine-tuning.
  • RL is slow, not impossible. The cost is generation time, which is set by memory bandwidth exactly as in the inference post.
  • Bigger models: a 9B would need LoRA on a 4-bit base; we did not try it.

What fits (estimates)

ModelTraining modeState GB (est.)On this Mac (usable ≈9.5 GB)How estimated
0.8BLoRA (bf16)≈2.0fitsbf16 weights + LoRA + activations (measured activations dominate for small models)
0.8Bfull fine-tune≈11.1does not fitbf16 weights 2 + fp32 master 4 + Adam m,v 8 = 14 bytes/param + activations
2BLoRA (bf16)≈4.5fitsbf16 weights + LoRA + activations (measured activations dominate for small models)
2Bfull fine-tune≈28.5does not fitbf16 weights 2 + fp32 master 4 + Adam m,v 8 = 14 bytes/param + activations
4BLoRA (bf16)≈8.9tight (swap)bf16 weights + LoRA + activations (measured activations dominate for small models)
4Bfull fine-tune≈59.3does not fitbf16 weights 2 + fp32 master 4 + Adam m,v 8 = 14 bytes/param + activations
9BLoRA on 4-bit base≈6.0fits4-bit base + LoRA (--quantize-base 4)
9BLoRA (bf16)≈18.5does not fitbf16 weights + LoRA + activations (measured activations dominate for small models)

Estimates are weights plus optimizer state and a 0.5 GB allowance; activations come on top and, as chapter 5 shows, can dominate. Treat "fits" as "should fit with small batches and checkpointing".

10. Conclusions

  1. A 16 GB MacBook can post-train small Qwen3.5 models, but you must budget for activations, not just weights. Gradient checkpointing changed a 15.8 GB step into a 6.9 GB one.
  2. Check backward support layer by layer. Qwen3.5's hybrid architecture needed a custom differentiable kernel before anything trained.
  3. SFT on a small human-written set hurt a model that already reasons well. A gentler recipe lost less, and self-distillation preserved the starting accuracy, but neither improved it: SFT only helps when the reference solutions are better than what the model already produces.
  4. RL needs a careful reward and a small learning rate. Reward that can be earned without being correct will be found within 50 steps; watch reward and length together.
  5. Unrewarded behaviour drifts. After we stopped rewarding the "The answer is:" format, the share of answers using it fell from ~0.8 to ~0.13 without hurting the score. Track style, not just accuracy.
  6. RL beat SFT here, at a price. The three RL methods improved the 0.8B (59-70% vs 54% untrained), and full fine-tuning fit in memory with a micro-batch of 1, but RL took hours where SFT took minutes. RL also shortened answers a lot (to roughly 75-220 tokens).
  7. Time scales brutally with model size for RL. Plan for about 3 hours per 0.8B run; larger models would take days.

What we did not measure, and caveats

Test accuracy is on 100 questions (about ±10 points) with one seed per setup; learning rates were corrected after the failures above rather than swept, so the method comparison is not a tuned head-to-head; power draw and batteries are untested.

What is next

Multiple seeds and a larger test set would tighten the method comparison; SFT with full fine-tuning and the 2B model are the obvious missing runs. The inference-side companion post covers running these same models.

Appendix

A. Results table

ModelMethodAdapterStatusBaseline accFinal accBest accGainTrainableLRStep sPeak GBSwap GBHours
0.8BSFTloradone54%22%54%-32 pts10.8M2e-051.39.83.60.3
0.8BSFT gentleloradone54%37%54%-17 pts10.8M5e-061.64.42.50.1
0.8BSFT self-distilledloradone54%54%54%+0 pts10.8M5e-063.77.13.50.3
0.8BGRPOfulldone54%68%68%+14 pts498.1M1e-0617.111.14.53.0
0.8BGRPOloradone54%64%64%+10 pts10.8M2e-0621.66.84.13.8
0.8BRLOOfulldone54%68%68%+14 pts498.1M1e-0615.811.14.62.8
0.8BRLOOloradone54%67%67%+13 pts10.8M2e-0618.96.97.93.4
0.8BGSPOfulldone54%70%72%+16 pts498.1M1e-0616.011.14.42.8
0.8BGSPOloradone54%59%66%+5 pts10.8M2e-0617.56.83.63.1

B. Resources per run

ModelMethodAdapterPeak MLX GBMax RAM GBMax swap GBStep sTrain tok/sRollout tok/s
0.8BSFTlora9.813.93.61.3295–
0.8BSFT gentlelora4.413.42.51.6233–
0.8BSFT self-distilledlora7.113.63.53.7352–
0.8BGRPOfull11.114.04.517.1424516
0.8BGRPOlora6.814.04.121.6425427
0.8BRLOOfull11.114.04.615.8413500
0.8BRLOOlora6.913.97.918.9407381
0.8BGSPOfull11.114.04.416.0424488
0.8BGSPOlora6.814.03.617.5462396

C. Method and data notes

  • Data: 500 GSM8K train questions (seeded subset saved with each run); evaluation on the first 100 test questions, greedy, thinking off, 512-token cap.
  • Hyperparameters: LoRA rank 16, scale 20, no dropout; gradient clipping 1.0; gradient checkpointing on for all runs except the first SFT run (0.8B LoRA), which finished before checkpointing was enabled for the 0.8B (it only changes memory and speed, not the maths); seed fixed.
  • Everything is kept: per-run metrics, rollouts and evaluation generations under runs/ (model weights were deleted locally after the runs; re-run train.py to regenerate them), the discarded attempts under runs_discarded/; W&B project macbook-posttrain has one line per (model, method, adapter).




Share this article




Copyright © 2022-2026 Azmine Toushik Wasi. All rights reserved.