Drive a GrowBot with your own agent

The robot's phone stays the safety layer; your brain sends high-level verbs. The web app validates each verb against the robot's body truth, clamps every number, enforces a motion duty budget, and runs the walk policy and reflexes (dead-man, tip-abort) in the browser at full rate. In the intended flow your agent sends verbs and never touches servos, firmware, or the raw wire protocol (see the safety notes at the bottom for the trust model).

your agent ──verbs──▶ relay ──▶ GrowBot web app (clamps · body truth · reflexes) ──▶ relay ──▶ chip ──▶ legs

Open growbot.dev/tester → Connect robot → pair → pick Custom agent. One field, two ways in:

• OpenAI-compatible endpoint (local models: Ollama, LM Studio, llama.cpp, vLLM — anything speaking /v1/chat/completions): paste the base URL. The page runs the loop: it sends your model an observation each turn and executes the one verb it replies. See LOCAL_MODELS.md for model picks, specs, and honest latency.
• Inbound agent (your own script, any language, running anywhere): leave the field empty and dial the relay yourself. Handshake below.

The verb menu

Out-of-range numbers are clamped (the ack says clamped:true). Off-menu verbs and unknown params are rejected — the robot does nothing. One motion at a time; a new motion verb replaces the current one.

The minimal agent prompt

Give your model this as its system prompt (the app sends the canonical copy — including your robot's real gesture list — in the menu envelope and uses the same text in endpoint mode):

You drive a small two-legged desk robot (GrowBot) by replying with exactly ONE verb call per turn.

BODY: two-leg desk walker. Two paddle legs that swing from the hips. It can walk, turn,
gesture, and speak aloud through its phone. No arms, no wheels, no odometry.

VERBS (the complete menu; anything else is rejected):
  speak(text="...")                 say it aloud (max 180 chars)
  forward(meters=0.05..2)           walk forward; ~0.05 m/s, meters are an estimate not a measurement
  turn(degrees=-180..180)           turn in place (an arc-turn where spins are unavailable); negative = left, positive = right (approximate)
  stop()                            halt everything, legs go limp
  gesture(name=wiggle|bow|sway)     a canned expressive move

RULES:
- Reply with ONLY the verb call on one line. No prose, no markdown, no code fences, no explanation.
- One verb per turn. Each turn you receive a fresh observation (sensors + the result of your last verb).
- Out-of-range numbers are clamped; off-menu verbs are rejected and the robot does nothing.
- A safety layer enforces servo limits and a motion duty budget (about 20s of leg motion per minute).
  If an ack says the budget is spent, speak or wait instead of moving.

Test your local model first (curl)

Before wiring anything up, prove your model emits clean verbs. Ollama example:

curl -s http://localhost:11434/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "qwen3:4b",
  "messages": [
    {"role":"system","content":"<the minimal agent prompt above>"},
    {"role":"user","content":"observation: {\"robot\":1,\"duty_left_ms\":20000}\ngoal from the human: say hello, then take a short walk.\nReply with exactly ONE verb call."}
  ],
  "max_tokens": 120
}'
# good reply:  {"choices":[{"message":{"content":"speak(text=\"hello! stretching my legs.\")"}}, …

Then paste http://localhost:11434/v1 into the Custom agent field. For the browser to reach Ollama, start it with OLLAMA_ORIGINS='*' ollama serve (CORS). LM Studio: enable CORS in the server settings.

"curl works, but the app says it can't reach my endpoint"

This is expected, not a contradiction — curl and the browser are not the same client. The app's page makes the outbound call, and browsers enforce two rules curl has no concept of:

Once the request does leave the browser, CORS is the second gate: LM Studio needs its server-tab CORS toggle on; Ollama needs OLLAMA_ORIGINS='*'. A missing/blocked model server, a wrong port, and a CORS rejection all surface as the same bare network error, so if the endpoint field is a real https/localhost URL and the app still can't see it, check CORS next.

Using ngrok specifically: ngrok's free tier serves an HTML "you are about to visit this site" warning page to any request that doesn't send ngrok-skip-browser-warning: true — this app sends that header automatically, but if you're curling through the same tunnel to sanity-check it yourself, add -H 'ngrok-skip-browser-warning: true' or you'll see the warning page's HTML instead of LM Studio's JSON.

Empty model list? /v1/models reports whatever LM Studio currently has loaded — if nothing is loaded/selected in its UI, the list is legitimately empty and the dropdown will say so.

Inbound handshake (your own script)

Your agent connects to the shared relay on the robot's agent side-channel — the pairing code plus -agent. The web app must be open on the Custom agent tab (it is the safety layer; no app, no motion).

Minimal Python agent (pip install websockets):

import asyncio, json, websockets

RELAY = "wss://growbot-relay.growbot.workers.dev"
CODE  = "gb-XXXXXX"                    # your robot's pairing code
CHAN  = f"{RELAY}/d/{CODE}-agent"

async def main():
    async with websockets.connect(CHAN) as ws:
        await ws.send(json.dumps({"t":"attach","id":f"{CODE}-agent","code":f"{CODE}-agent"}))
        await ws.send(json.dumps({"t":"hi","name":"demo"}))
        rid = 0
        async for raw in ws:
            m = json.loads(raw)
            if m.get("t") == "menu":
                print("verbs:", [v["verb"] for v in m["menu"]])
                rid += 1
                await ws.send(json.dumps({"t":"verb","rid":rid,"action":'gesture(name="wiggle")'}))
            elif m.get("t") == "ack":
                print("ack:", m)        # ...decide the next verb here (or ask an LLM to)
            elif m.get("t") == "obs":
                pass                    # ~2 Hz sensors: m["robot"], m["orientation_deg"], m["duty_left_ms"]

asyncio.run(main())

Quick poke without code (brew install websocat): run websocat wss://growbot-relay.growbot.workers.dev/d/<code>-agent, paste the attach line, then a verb line.

Safety model — what the app enforces, honestly

• Body truth: verbs and params are validated against the robot's body file; servo targets are clamped to the safe band before anything is sent.
• Duty budget: ~20s of leg motion per rolling 60s. Exhausted → motion verbs return an error ack with the rest time; speech still works.
• Reflexes stay local: the 500ms firmware dead-man, the browser tip-abort (walk stops if the body pitches over), and STOP all run at full rate regardless of how slow your brain is.
• Verbs, not servos — for a cooperative brain: an agent that speaks this protocol sends verbs, and the app clamps them to the body band and duty budget before any motor command goes out. This is what makes it safe to point a small, low-capability local model at the robot: a buggy or confused brain still can't drive the legs out of band.
• This is not a security boundary. The relay is unauthenticated today (a known TODO, exactly like the existing manual/policy/LLM remotes). Anyone who holds the pairing code — which you hand your agent as <code>-agent — can open the chip's main channel /d/<code> directly and move the legs with raw envelopes, bypassing these clamps. So the clamps are a safety rail for a cooperative brain, not a defense against a hostile one. Treat the pairing code like a password and only run agents you trust.

← back to the app · local models guide: LOCAL_MODELS.md · putting your own hardware on the other end: body-docs (experimental)