Forge · RL with verifiable rewards

Teaching a 1.5B model to reason with RL

Qwen2.5-1.5B, trained with GRPO (reinforcement learning against a math checker, the technique behind DeepSeek-R1) on a single 8GB RTX 5060. Compare the base model and the tuned model on the same problem, side by side.

Run it liveHow it works
Training signal · mean group reward
1.23to2.80 of 3.25

750 steps x 8 generations. Full curve and every training figure on Results.

GSM8K pass@1
58.8% → 70.0% +11.2 pts
Forgetting (ARC)
69.5 → 68.5 ≈flat
Rewards
4 verifiable
Served
228 tok/s
01The demo

Pick a problem

Replaying real cached outputs from both models, recorded offline through Ollama (Q4_K_M) so this page always works.
02Comparison

The same problem, twice

Base Qwen2.5-1.5B-Instruct

reasoning trace
answer

Tuned Qwen2.5-1.5B + GRPO

reasoning trace
answer
solving…
Base 1/6 · Tuned 5/6 on these examples
03Go deeper

The rest of the evidence

PlaygroundRun it yourself, liveType any problem. Both models answer side by side on a real GPU, streaming.MethodHow GRPO actually worksThe reward stack, and the bug where every reward was zero so the run learned nothing.ResultsEvery number, and its sourceStrict vs lenient scoring, the forgetting control, quantization cost, serving latency.TracesRead the reasoningUnedited completions on held-out problems, including the one both models miss.