On-device fine-tuning in 608 bytes — GravOptMini optimizer + W-Twin health monitor for STM32
Built a minimal on-device training stack for STM32 — fits in 608 bytes RAM total.
The problem: Adam optimizer needs exp_avg + exp_avg_sq — 2× parameter RAM. On STM32F103 (20KB SRAM) with a 5K parameter model that's 40KB just for optimizer state. Doesn't fit.
What I built:
GravOptMini — momentum optimizer with quantile-based gradient freeze. Only exp_avg (1× param RAM). Freezes low-gradient parameters to reduce compute.
W-Twin Lite — power-law baseline fitted on early steps, tracks deviation. Pure float32, no scipy, no numpy. Rolling window of 20 values. ~240 bytes.
Benchmark (Python simulation, STM32 memory constraints):
- Model: 46 params, 184 bytes
- GravOptMini state: 368 bytes (exp_avg only)
- W-Twin Lite: 240 bytes
- Total: 608 bytes
- MNIST accuracy: 93.2% vs Adam 97.4%
- W-Twin overhead: 3.9% per step
- Detection: anomaly caught 96 steps before visible loss divergence, 0 false alarms on clean run
Next step is a real STM32 test with physical hardware. C port of both components is the plan.
Has anyone done on-device fine-tuning on STM32? Curious what optimizer you used and what constraints you hit.
Code: github.com/Kretski/WTwin
