# Local models that can drive a GrowBot

The Custom-agent mode (growbot.dev/tester → Connect robot → pair → **Custom agent**) points the robot at any
OpenAI-compatible endpoint. That covers **Ollama**, **LM Studio**, llama.cpp's server, and vLLM —
so a model running on your own laptop can be the robot's brain, offline, free, private.

The job is small: read a one-screen prompt + a JSON observation, reply with ONE line like
`forward(meters=0.3)`. That's instruction-following, not deep reasoning — small models do it.
What separates models here is **format discipline** (no prose, no markdown, one verb) and latency.

## Recommended models

| model (Ollama tag) | size (Q4) | min RAM | verb discipline | notes |
|---|---|---|---|---|
| `qwen3:4b` | ~2.6 GB | 8 GB | very good | best small pick; disable thinking (`/no_think`) or expect slower replies |
| `llama3.2:3b` | ~2.0 GB | 8 GB | good | fastest usable; occasionally adds a chatty tail — the app salvages the first verb |
| `gemma3:4b` | ~2.5 GB | 8 GB | good | solid all-rounder |
| `qwen3:8b` | ~5 GB | 16 GB | excellent | noticeably steadier goal-following; the sweet spot if you have the RAM |
| `llama3.1:8b` | ~4.7 GB | 16 GB | very good | reliable classic |

**Not recommended:** 1–2 B models (`llama3.2:1b`, `qwen3:1.7b`) — they frequently break the
one-verb format or invent verbs. They *sometimes* work; you'll spend your time rereading the trace.
Reasoning-heavy models (or `qwen3` with thinking left on) work but burn 5–30 s "thinking" about a
wiggle — the app strips `<think>` blocks, but you still wait.

## Minimum specs, honestly

- **Apple Silicon (M1 or later), 8 GB**: 3–4 B models run well. 16 GB comfortably runs 8 B.
- **PC with a GPU (RTX 3060 / 8 GB VRAM or better)**: everything above is fast.
- **CPU-only laptops**: a 3 B model works but replies take several seconds; fine for a slow, deliberate robot.
- **Raspberry Pi 5 (8 GB)**: `llama3.2:3b` ≈ 5–8 tok/s → 5–15 s per decision. Workable, glacial.

## Honest latency

A verb reply is ~10–25 tokens, but the model also has to read the prompt + observation
(~600–900 tokens; the system prompt caches after the first call). Measured end-to-end per decision
(request → verb), typical:

| hardware | 3–4 B | 8 B |
|---|---|---|
| M1/M2/M3 Mac | **1–3 s** | 2–5 s |
| RTX 3060+ | **0.5–1.5 s** | 1–2.5 s |
| CPU-only x86 | 3–8 s | 8–20 s |
| Raspberry Pi 5 | 5–15 s | not realistic |

For scale: the hosted brain the creature uses lands around 2–5 s per thought. So a local 4 B on a
Mac *feels the same or snappier*. This is **turn-based cognition, not realtime control** — and that's
fine, because the reflexes never wait for the brain: the walk policy runs in the browser at 30 Hz,
the firmware dead-man limps the legs 500 ms after silence, tip-abort and STOP are local. A slow
brain makes a thoughtful robot, not an unsafe one.

## Setup

**Ollama** (the browser calls your endpoint, so CORS must be open):

```bash
ollama pull qwen3:4b
OLLAMA_ORIGINS='*' ollama serve
# endpoint field in the app:  http://localhost:11434/v1
```

**LM Studio**: load a model → Developer/Server tab → start the server → **enable CORS** →
endpoint field: `http://localhost:1234/v1`.

Gotchas, honestly:

- **Open the Custom-agent page on the same machine as the model** and use `http://localhost:…`.
  Browsers exempt localhost from mixed-content blocking — but **Safari doesn't**; use Chrome,
  Edge or Firefox for endpoint mode.
- **The robot's phone can't reach your laptop's `http://192.168.x.x`** from the https app
  (mixed content, all browsers). If you want the model on the laptop *and* the closed-loop IMU
  walk on the robot's phone, use the **inbound agent mode** instead: a 20-line script on the
  laptop dials the relay over `wss://` (no CORS, no mixed content) and the robot phone stays the
  safety layer. Handshake + example: [/agent-docs.html](/agent-docs.html).
- Endpoint mode on a laptop has no IMU, so `forward`/`turn` fall back to a rough open-loop
  shuffle. The trained closed-loop gait needs the page running on the robot's phone.