Local Fleet — the all-local swarm mode

• Status: Proposed
• Date: 2026-07-05

Local Fleet is a mode, not a replacement: run the entire swarm — the lead brain and its coding workers — on a single Apple Silicon machine with no frontier-API dependency. It trades peak quality for cost, privacy, and offline operation. Frontier-API workers (Claude Code, Codex on their subscriptions) remain the quality bar; Local Fleet is what you reach for when the run must stay on the box, run without a network, or cost nothing per token.

Reference hardware for the sizing here is a MacBook Pro M5 Max with 128 GB unified memory. The design is not pinned to that box — it is the current yardstick.

This mode does not exist end-to-end today. The pieces it stands on mostly do: Clanky already has a local brain route and a local worker launcher. What is missing is per-spawn model/effort control and a memory-aware scheduler. Each section below marks Exists today vs Proposed explicitly, and the table collects that.

What it is and is not

• It is a routing decision, not a new architecture. Clanky stays the lead agent that plans, spawns, watches, and harvests; workers stay visible clanky:<slug> panes on Herdr. Only the models behind the brain and the workers change.
• It is not the default and not the quality ceiling. Local models trail frontier models on hard judgment and long-horizon coding. Local Fleet is a deliberate mode toggle, chosen per run or per deployment.
• Policy stays in the brain. Consistent with Clanky's invariant that the brain applies policy and the data planes stay mechanism, model routing and memory pressure are the brain's concern, not Herdr's or a model server's.

The shape

The mode is assembled from existing spawn/model seams plus one new control surface. Each seam is cited to where it lives so the "exists today" claims are checkable.

1. Brain model swap

Exists today. Point the brain (the Eve lead-agent runtime) at a local OpenAI-compatible endpoint instead of the Codex subscription. CLANKY_MODEL_PROVIDER=local selects the local route; createLocalModel (clanky-agent/agent/lib/local-model.ts) builds an @ai-sdk/openai Chat-Completions model against CLANKY_LOCAL_BASE_URL (default http://127.0.0.1:11434/v1) and CLANKY_LOCAL_MODEL. The endpoint is just a base URL + model id, so it works against any OpenAI-compatible server — an MLX server, llama-server, Ollama, or LM Studio — or a self-hosted orchestrator fronting a local pool (SPEC.md §4.6).

Reasoning effort forwards to thinking models as reasoning_effort via CLANKY_LOCAL_EFFORT (low/medium/high). Because eve cannot resolve an off-catalog model's context window, CLANKY_LOCAL_CONTEXT_TOKENS sets it (the face auto-injects it from Ollama metadata; default 32768, clanky-agent/agent/lib/local-context.ts).

The leader needs judgment — it plans, adjudicates, and unblocks. Target a ~100B-class MoE at 4–6-bit quantization (≈55–70 GB resident), served locally. Named models are deliberately not pinned here; see open questions.

2. Harness allowlist

Exists today. Restrict which harnesses the lead may spawn to local-capable ones through the existing allowlist seam. /harness allow writes CLANKY_CODING_HARNESSES (clanky-agent/agent/lib/coding-harness.ts; enforced at assertCodingHarnessAllowed, so herdr_spawn rejects a disallowed harness). For an all-local run, allow only clanky (the Clanky runtime worker, which rides the same local brain) plus whichever launchable harnesses are configured for local serving.

Each launchable harness (claude, codex, opencode) has an Ollama launcher profile: set CLANKY_CODING_HARNESS_<ID>_LAUNCHER=ollama and the seam renders ollama launch <id> --model <model> instead of the native CLI (SPEC.md §4.3; ollamaHarnessCommand). Ollama-launched Codex rewrites its config dir, so those workers get an isolated CODEX_HOME via CLANKY_CODEX_OLLAMA_HOME (ollamaCodexHome) — a local Codex worker and a subscription Codex worker can run side by side without clobbering ~/.codex. That isolated-home pattern is the precedent the model-control proposal builds on.

3. Worker models

Partly exists (config-level). Run two to three small coders (14–32B, quantized, ≈10–20 GB each) as the workers. Per-harness model pinning already exists for the Ollama launcher: CLANKY_CODING_HARNESS_<ID>_MODEL (e.g. CLANKY_CODING_HARNESS_CLAUDE_MODEL) becomes the --model flag on ollama launch. What does not exist is choosing that model per spawn — see the next section.

Capacity fits on 128 GB, but the true constraint is unified-memory bandwidth under concurrent decode, not resident size. Workers are bursty (they think, edit, run a check, wait on I/O) rather than decoding continuously, so a small fleet is workable — but simultaneous decode across the leader and every worker is where it degrades. This is what motivates the fleet policy layer.

Indicative budget on 128 GB unified memory (conservative fit):

The high end overruns 128 GB: you cannot hold the largest leader, three mid-size coders, and full contexts resident at once. A conservative config (a mid-leader plus two coders) fits; a maximal one forces load/unload. That trade — capacity ceiling and bandwidth contention — is exactly the scheduling problem the policy layer owns.

4. Subagent model/effort control (the missing piece)

Proposed. This is the core proposal. Today model selection is brain-global: the /harness <id> --model command writes a CLANKY_CODING_HARNESS_<ID>_MODEL env var (Ollama launcher only) and requires a brain restart to take effect. The spawn seam itself carries no model or effort field — herdr_spawn's input schema (clanky-agent/agent/tools/herdr_spawn.ts) has harness, performer, command, cwd, but nothing to say "spawn this worker on the 32B coder at high effort and that one on the 14B at low." A lead running a mixed fan-out cannot route by task size without editing global config and restarting.

Two ways to close that gap:

(a) Clanky edits harness settings files per spawn/profile. Write the model and effort into each harness's own config (~/.claude*/settings, ~/.codex config) before launch. Powerful — it reaches every harness knob — but it mutates the user's own harness configuration, and concurrent spawns race the same files. If pursued, it must use the isolated-home pattern (as CLANKY_CODEX_OLLAMA_HOME already does for Ollama Codex) so a Clanky-managed config never clobbers the user's, with one isolated home per model/effort profile.

(b) A Clanky command that renders per-spawn launcher flags — recommended. Add /harness model <harness> <model> [--effort <level>] (and a matching per-spawn override field on the spawn seam) that turns the selection into launch flags at spawn time — claude --model <model>, codex -c model=<model> -c model_reasoning_effort=<level>, ollama launch <id> --model <model> — with no shared-config mutation. It composes with the existing launcher/allowlist resolution, gives per-run and per-worker control, and is visible in the spawn manifest and orchestration graph. It also generalizes the seam that today only renders --model for the Ollama launcher: extend it to native launchers and to effort, scoped per spawn.

Recommendation: (b). No global mutation, no restart, no config races, auditable per spawn. Reserve (a) for knobs that only a harness's own config file can express, and only behind an isolated home.

5. Fleet policy layer

Proposed. A scheduler in the brain that is aware of unified-memory pressure: load and unload models as demand shifts, route by task size (judgment → leader, iteration → small coders), and cap concurrent decodes so bursty workers do not all contend for bandwidth at once. This lives in the brain, not in Herdr or a model server — consistent with Clanky's existing invariant that policy is the brain's and the data planes stay deterministic mechanism. It builds directly on per-spawn model/effort control (b): the scheduler decides model + effort per task, and the spawn seam renders it.

Topology

flowchart TB
  owner["Owner / run request"]

  subgraph brain["Clanky brain (Eve lead agent)"]
    lead["Lead agent<br/>plan, spawn, watch, harvest"]
    policy["Fleet policy layer<br/>(Proposed): route by task size,<br/>cap concurrent decode, load/unload"]
    allow["Harness allowlist<br/>CLANKY_CODING_HARNESSES"]
    modelctl["Per-spawn model/effort control<br/>(Proposed): renders launch flags"]
  end

  subgraph mux["clanky-herdr"]
    w1["Worker pane clanky:a<br/>coder 14-32B"]
    w2["Worker pane clanky:b<br/>coder 14-32B"]
    w3["Worker pane clanky:c<br/>coder 14-32B"]
  end

  subgraph serve["Local model server(s)"]
    leader["Leader model<br/>~100B MoE, 4-6 bit"]
    coders["Coder models<br/>14-32B quantized"]
  end

  owner --> lead
  lead --> policy
  policy --> allow
  policy --> modelctl
  modelctl -->|spawn seam: model + effort flags| w1
  modelctl -->|spawn seam| w2
  modelctl -->|spawn seam| w3
  lead -->|CLANKY_MODEL_PROVIDER=local<br/>OpenAI-compatible /v1| leader
  w1 -->|ollama launch / --model| coders
  w2 --> coders
  w3 --> coders

  serve -.->|shared unified memory<br/>bandwidth is the real limit| policy

The allowlist and brain-model swap are live seams; the model/effort control and the policy layer (dashed/marked) are proposed.

What exists today vs proposed

Open questions

• Serving layer. MLX server vs Ollama vs both. MLX fits Apple Silicon best for the leader; Ollama is what the existing worker launcher already speaks. A mixed setup (MLX for the leader, Ollama for coders) is plausible but doubles the operational surface. Pick one to lead with.
• Model curation, not model names. Concrete model picks (leader and coders) stale within weeks. This doc pins classes (≈100B MoE leader, 14–32B coders) and defers the named picks to a curation process (the local-llm-curator workflow) rather than hard-coding a roster that rots.
• Effort mapping per harness. Effort levels do not line up across harnesses (Codex: minimal/low/medium/high/xhigh; local reasoning_effort: low/medium/high; Claude has its own). The /harness model --effort surface needs a per-harness mapping so one requested level renders correctly per launcher.
• Where load/unload lives. The policy layer decides, but the mechanism (Ollama keep-alive/unload, MLX process lifecycle) differs by serving layer and ties back to the first question.