Five Agents, One Model

The original design was clean on paper: five specialized agents, five fine-tuned LoRA adapters, hot-swapped in and out of the base model depending on which agent was handling a given request.

In practice, adapter swapping under load caused GPU memory to climb until the process ran out of memory. The fix was to stop swapping — load the base model once, and differentiate agent behavior through role-specific system prompts instead.

It's a less elegant architecture than per-agent fine-tuning, and it gives up some of the specialization LoRA adapters bought. But a system that's reliably 90% as specialized beats one that's 100% as specialized and crashes under load.