Building a 4-Node Local AI Fleet on Apple Silicon
How I orchestrated GUS, Mama, DJ, and DRU into a coherent inference fabric — routing models, managing VRAM, and cutting cloud spend by 95%.
The Problem
Running every LLM call through a cloud API is expensive and slow. At scale — dozens of agent dispatches per session — it compounds fast.
The Architecture
Four machines, one inference fabric:
- GUS (Mac Mini M4) — orchestration boss, always-on, qwen3:14b
- Mama (Mac Studio M1) — heavy compute drone, gemma4:26b
- DJ (MacBook Pro M1) — tertiary worker, light multimodal tasks
- DRU (MacBook Pro) — Director workstation, advisor sessions only
The Numbers
After wiring this up, cloud API spend dropped ~95%. Most inference now runs fully local, fully private, with sub-150ms round-trips for classification tasks.
Lessons
Model diversity matters more than raw compute. Running decorrelated families (Gemma, Qwen, DeepSeek) gives genuinely independent verification — not the same failure modes in three containers.
DG
Drew Garman
Forward Deployed Engineer — Phoenix, AZ. drewgarman.dev