Decision heads on a small local model
Over four days in September 2026 I trained tiny classification heads on top of frozen, 4-bit Qwen2.5 models, to see how well a small local model could read decisions out of documents. Two public repositories hold phases one and two (depth and distillation sweeps) and phase three. On the task families they were trained on, the heads beat the backbone's own zero-shot read-out by 14.8 points. On the six unseen task families I tested, they lost.
- six unseen task families, vs zero-shot
- −3.25 pts
- workflow suite, zero-shot 0.650
- 0.582
- 20 of 28 layers, latency saved
- 28%
Limits. These heads are not production-ready and do not reproduce or match the commercial system I compared against; its published numbers were never run on my hardware. In held-out-task probes, accuracy on the held-out families fell to 0.0063 when two large-label families (150 and 100 labels) were left out, and to 0.594 when two small-label families were. No safety, robustness or calibration testing was done.