Architecture

Clankie is one service plus the surfaces that reach it. The service owns the captain (a pi-based agent with durable sessions), his tools, his game bodies, and the HTTP API every surface speaks.

This Mermaid diagram, ADR 0128, ADR 0129, and ADR 0145 are the canonical current architecture diagrams. The JPG/tldraw exports under docs/diagrams/ are historical snapshots.

The public gateway routes to the configured Clankie host. It keeps live host connections, expiring pairing-route hashes, and bounded in-flight exchanges. Optional APNs delivery also persists device-authorized routing registrations and revocation versions (ADR 0159). The Mac remains authoritative for offers, devices, grants, conversations, terminal sessions, Herdr, and credentials. Pairing begins at the stable global origin, then the Mac returns /h/{hostId} as the control and relay base for the device session. See ADR 0151.

A Mac earns its place at that doorway by signing in, not by an operator copying a bearer. One Cognito user pool is the account authority: /gateway completes an email one-time code, the broker keeps the access and rotating refresh token as clankie-account, and the Mac presents the access token on its outbound WebSocket. The gateway verifies that JWT against Cognito's published keys and derives the host route from the account subject plus a per-installation id, so it needs no account or host database and every Mac under one account keeps a distinct route. Cognito identifies Macs only; phones are identified by device sessions the Mac itself signs. See ADR 0153 and the secret-by-secret trust map in docs/credentials.md.

How a message becomes a turn

The older message-to-captain JPG is a historical snapshot; the present flow is described below.

A Discord message reaches the active bridge. A text-only message in the live voice channel's attached chat enters that room's existing VoiceFloor; the realtime room thread may answer aloud, ask the captain to act, or stay silent, and no separate text turn races it (ADR 0124). Every other message posts to POST /v1/captain/channel-turns. The service normalizes it — untrusted body fenced and labelled, images resolved to bytes at the last hop, channel context attached — and prompts a pi session. Every room gets a continuing session (a pi JSONL tree that survives restarts): operator conversations, voice channels under ~/.clankie/captain/voice/, and text channels under ~/.clankie/captain/rooms/ (ADR 0118). A message that arrives while that room's run is streaming is steered into it and reported absorbed, so a burst of messages gets one merged reply rather than one reply each (ADR 0091). The channel backlog still rides in with the request, and is used only when the lane does not already hold that conversation. A privileged turn drops to a one-shot, which writes its own tree under ~/.clankie/captain/turns/ so the tools it ran are readable afterwards (ADR 0107). The reply carries the turn's last screenshot or generated image with it — and when that artifact cannot be resolved, the words still post and say the picture did not — while replying with the silence sentinel sends nothing: silence is a real answer. Nothing caps how long a turn may take — looking something up properly is work, not a fault — but a turn that emits no event at all for 5 minutes is a dead stream, so the stall watchdog aborts its pi session and settles it as captain_turn_stalled. While someone waits on a slow requested turn, he can post one short send_text_update message to the channel ("hang on, pulling the bracket up") without ending it; work he elects to do on his own stays quiet. In channels where the owner enables /clankie tools mode:on, the host also edits one quiet tool-activity card with public-safe work categories, counts, elapsed time, and a terminal state; tool names, arguments, and results stay in the local Pi trail (ADR 0134). Discord shows him typing from his first words to the end of the turn: the captain signals the body when his reply stream can no longer be the silence sentinel, so a turn he ends in silence never shows the room a reply being written.

The TUI and relay speak the same operator-conversation contract (/operator/v1/dispatch): durable agent personas, their current fleet seats, one coherent cursor-long-polled fleet snapshot, revision-fenced sends, cursored replay, and long-polled tails. A tail carries two things: the durable events, and the message the captain is typing right now — a volatile draft held in memory, never in the event log, that the settled message event replaces in the block it streamed into (ADR 0141). Herdr persona conversations are direct-send lanes with no Pi session. The persona owns its name, full appearance tuple, DM, and channel memberships; its current Herdr seat supplies live status and terminal routing. Their readable history folds the complete active user/assistant branch from Herdr's native Claude Code, Codex, Pi, or Grok session identity; raw terminal bytes stay on the terminal lane (ADR 0135). The app and controlled swarm-home Discord project those same host-owned records and logs. Discord faces are app-baked PNGs served under content-hashed HTTPS paths by the existing Activity origin (ADR 0147). Herdr's native event subscription advances the volatile fleet cursor; persona, seat, channel, and stance changes advance the same cursor. Foreground apps therefore render one current seats/personas/channels moment without polling or persisting a second world projection (ADR 0150). The operator seat is a place any harness can sit (ADR 0152). clankie seat opens Claude Code, on the owner's own plan, as Clankie: the plugin at integrations/claude-plugin forces his identity as the output style, injects the owner persona, reach, address, and service model card at session start (clankie prompt) and the newest memory card on every turn (clankie memory-card), and names one stdio MCP server, clankie mcp, that bridges to the service's lane tool bank at /v1/mcp with the operator bearer read from the broker. The bank is the same authored registry the pi session is built from, wrapped once at runtime and scoped by the bearer's lane, so a Codex pane with the same entry is the same seat. A herdr pane named clankie is his head: the census binds it to his own persona rather than a fleet contact and projects its transcript into the conversation the app pins. While a seat is bound, self-wakes, herdr completion watches, and room escalations reach it as channel events pushed by clankie mcp; with no seat open they run the TUI operator lane on pi as before. Social lanes never sit in the seat: the owner's plan carries only the owner. Every fleet seat has a mailbox of its own, and a Claude Code seat launched with the channel runs clankie mcp --seat, a channel-only bridge that polls it: a DM or room turn then lands as a channel event instead of keystrokes typed into the pane's pty, and a Codex seat takes it through codex queue, so nothing the operator is drafting there is touched (ADR 0161).

A TUI process opens the existing main Clankie conversation unless --chat selects another. /new creates a fresh conversation explicitly. A captain conversation and its Pi session are one lifetime: bounded retention removes their shared directory, while public event logs rotate with typed cursor recovery (ADR 0111). /btw temporarily selects an ephemeral child made with Pi's native current-leaf fork, opening it on a clean screen at that boundary. A hidden boundary makes the inherited branch reference-only; Ctrl+X swaps between the child and its parent without discarding either, and Ctrl+C cancels and deletes the child, restores the parent transcript, and replays any parent events that arrived meanwhile (ADR 0143). Conversations are files under ~/.clankie/captain/. Each settled operator or Discord captain turn also appends one metrics line to ~/.clankie/captain/turn-settled.jsonl: tool-name counts, first mutating tool, context-token occupancy, the model/provider/effort that actually executed the turn, and the provider-reported totalTokens summed over the turn with the number of reports that contributed. Execution identity is read off the live pi session as the turn executes, so a /model or /effort change under a live conversation lands on the next turn to execute rather than being reconstructed from a settings snapshot afterwards. Unknown is said out loud: a row from before the capture, or a provider that reported nothing, reads back as null — never zero, and context occupancy is never treated as usage or a charge. GET /v1/captain/turn-metrics and clankie metrics return the same bounded rows, newest first. The file sits beside autonomy.json, outside the conversation directory the retention pass deletes. It is not ~/.clankie/events.jsonl — that log already uses captain.turn.settled for presence idle/waiting_user, and the captain does not write domain events. An absorbed steer (ADR 0091) shares the owning run's line. The full HTTP surface is listed in apps/clankie/openapi.yaml; apps/clankie/scripts/setup-yaak.py imports that canonical catalog into Yaak and adds a Keychain-backed Local environment for authenticated requests.

The service also keeps autonomy.json: one owner-approved goal and one replaceable self-wake per operator conversation, plus a global enable switch. An unreadable file fails closed and surfaces state_unreadable to operator clients instead of silently re-enabling autonomous work. An active goal queues host-authored continuation turns through the same conversation chain as operator messages, so every tool call and reply stays in the existing Pi session and public event log. A human message that arrives while that run is streaming steers it by default; in-flight tool calls still finish. Explicit delivery: "steer" also joins a human-started Pi turn, while delivery: "queue" waits for a separate turn on the conversation FIFO (ADR 0091). A token budget moves a goal to budget_limited; /goal owns activation, pause/resume, and clearing, while /autonomy off stops new continuations and wakes. A due wake queues one turn with Clankie's recorded reason and may be replaced by another. Neither path changes the conversation's tool set or authority (ADR 0130). Each conversation also keeps an append-only goal decision journal under ~/.clankie/captain/goal-journal/ — one line per real choice made while working a goal, written through note_goal_decision and returned by get_goal so a continuation resumes from what was already decided (ADR 0132).

The native macOS menu-bar app uses that same contract to list continuing Pi sessions and tail expanded transcripts. Its microphone opens a private local realtime room over an authenticated loopback WebSocket; social speech stays in the room, while ask_clankie sends actionable work through the operator conversation service. Raw PCM remains in memory. Exact Discord speech is a separate, bounded captain read that returns content only while owner-controlled transcript retention is enabled (ADR 0125).

Operator input can invoke an exact loaded skill as /name task or /skill:name task. The service rewrites that verified invocation to Pi's native skill command and enables expansion for that prompt only. Discord input and ordinary operator prompts keep expansion disabled.

Before each Pi run, a hidden host extension reads the newest bounded episode card into the system prompt. The host supplies the destination lane, filters operator-private notes out of ambient lanes, and refreshes recall without persisting duplicate cards in the conversation. Discord turns also receive the newest visible person facts for their authenticated guild/user identity. The global 128-episode ring and per-person fact files live under ~/.clankie/memory/; the TUI's /memory command browses, edits, and forgets that same store through operator-only routes. docs/memory.md is the full picture — what each store holds, who may read it, and what bounds it.

A second hidden extension appends the model card: the name, ref, and provider he is actually running on, his reasoning effort, and his context and output limits. It resolves the same selection Pi executes, on every run rather than once per session, because /model and /effort swap the model under a live conversation. Asked what he runs on, he answers from the prompt like he answers with his own address — no tool call, no guess, and silence if the selection cannot be resolved.

Where things run

• Captain tools. Coding tools (read/bash/edit/write) are pi built-ins. They attach to the operator console and to Discord turns authorized by the machine-control grants (ADR 0095, ADR 0105, ADR 0133). An individually granted actor gets a one-shot tool-bearing turn in shared rooms and a durable tool-bearing lane in an official-bot DM. Explicitly trusted guilds, optionally narrowed to channels, give every admitted member the same durable tool-bearing lane. Social and system histories have separate session keys, so revocation routes the next message away from the old tool bank. They land in the conversation's workspace — the directory a workspace-scoped operator conversation names, this repository for every other lane (ADR 0104). Voice join/leave are the same argument-free tools on Discord and the operator console: a Discord turn follows the authenticated speaker, an operator turn follows the configured owner (ADR 0062). The canonical authored-tool registry is apps/clankie/src/captain/tools.ts, connected-service additions live in captain/connect-tools.ts, and the HTTP surface is apps/clankie/openapi.yaml. This document does not duplicate their changing census.
• Browser catalog. The service registers the complete paginated agent-browser catalog with pi, but only everyday navigation tools and browser_tool_search start active. Browser calls are sequential across rooms; the subprocess receives no Clankie credentials, but true filesystem/network isolation requires a VM or remote broker (ADR 0082). The persistent profile holds his own accounts, signed up for by hand: the catalog's headed argument relaunches the browser visible on the operator's screen, so he can hand over the window for a signup, a CAPTCHA, or a phone check rather than grinding at it. A headed session is exempt from the browser's idle timeout (ADR 0127).
• Leading agents. Clankie leads coding agents through the herdr CLI over bash, guided by skills — there is no worker protocol. The service is his durable body. On first startup, auto starts the private bundled runtime and saves the binding; the fleet is Clankie's own session, never one the service was launched inside, and a pane joins it by being created there (ADR 0164). Every later console and restart follows that choice; a session the owner named selects an external one (ADR 0157). The service subscribes to the chosen socket's native events to wake fleet readers across all of its workspaces (ADR 0150). Any operator turn whose session is up attaches a live agent census so he can lead, route, and harvest without rediscovering the room; a turn from the operator console sitting in a pane is additionally a join — that pane is him. The herdr-lead board is the companion dashboard (ADR 0097). Agents coordinate through herdr and plain files.
• His body. runFreePlay drives one seam, GbaDriverIo (packages/play); its mind, voice, progress, learned transitions, and behavior loop hold no emulator and never learn what implements the seam. One body implements it: Clankie's separately credentialed seat in a PokeAgents world, reached through WorldPlayerClient on @pokeagents/world-protocol/ipc (WORLD_ADDRESS unix, tcp, or tls — defaulting to the world's own socket under WORLD_STATE_DIR) and entered with the pokeagent_join_mmo tool (ADR 0103, ADR 0145). The captain is the parent of that sitting; @clankie/play is the driver — the same split other harnesses get from an MCP Task, a subagent, or a CLI loop. That seam consumes verified FireRed adapter-v2 and Emerald adapter-v2 payloads, selected by the observation's (gameId, adapterVersion) pair; unknown pairs and game-specific extras the selected schema does not verify fail closed. A hosted world cannot be paused, changes without him acting, and can replace his body under him, so the loop offers no save, load, or restart action — the world persists its own cartridge. pokeagentMmoEnabled is the owner setting; with it off, or with no world reachable, the ask refuses out loud rather than falling back. Frames flow to the Discord activity surface. Every sitting carries a stable journey identity separate from its run id, the bounded story spans that journey, and the next sitting receives the last self-authored notes and objective while exact world state stays with the cartridge save (ADR 0126). The journal records body provenance at each causal stage, and its venue still reads both values because journals written before ADR 0145 are on disk. Other harnesses reach the same world through PokeAgents' own front doors — @pokeagents/world-mcp, its CLI, or the pokeagent-mmo skill — each on its own credentialed seat, as the same parent-plus-driver sitting (MCP Task, host subagent, or CLI loop; PokeAgents ADR 0023), with no control over Clankie, Activity publication, play voice, or room input. EnvironmentRuntime leases remain internal action/session fences within the owning runtime; they are not cross-process possession.
• PokeAgents boundary. The sibling PokeAgents repository owns the WORLD_OPERATIONS catalog, capability schemas, native client transport, and the MCP projection derived from that catalog. MCP carries calls; the world contract and host enforce player identity, authority, and gameplay semantics. Clankie currently imports only the pinned @pokeagents/world-protocol package (including /ipc) and keeps host, emulator, persistence, and world-MCP packages out of product source. Hosted play composes WorldPlayerClient. Every catalog operation is classified body or mind; unclassified names stay off pokeagent_world until classified. The play loop owns BODY (world.join, world.leave, play.observe, play.act, play.frame, play.watch); the mind owns session, who, regions, travel, and challenges.
• Auth. Provider keys and OAuth tokens live in the credential broker (Keychain), written by the TUI /auth flow and read by pi through a credential-store bridge. Compatibility model/media provider keys may fall back to existing shell values or the gitignored root .env.local when the broker has no entry; Discord account and body credentials remain broker-only except documented operator/captain test overrides. Persona is owner-authored in ~/.config/clankie/settings.json and can never be set by a caller. /connect stores Linear and mailbox credentials the same way; Discord remains a body configured by /discord (credential guide, ADR 0093). The mailbox is his own address, not the owner's inbox: email.fromAddress carries the identity when the provider login differs, the captain states that address from settings, and mail stays console-only because sign-in codes arrive there (ADR 0127). That address is public, so every message the mail tools return is labelled untrusted sender text the way a Discord body is (ADR 0081). A seat in a hosted world is a broker credential too — pokeagent_mmo_world, with the environment variant refused outright. Each media-enabled active Discord body owns one clankvox child through the Apache @clankie/vox-client boundary. A text-only official-bot process does not spawn Vox. Both media-enabled bodies use its primary role for voice, TTS, and music; the lab user body can concurrently watch screen shares and publish Go Live through separate roles (Discord media guide, ADR 0128). /discord Active body picks which process is the mouth; the launcher starts only that one (ADR 0048). Who may ask him to drive this machine from Discord is configured under discord.systemActorUserIds, systemActorGuildIds, and systemActorChannelIds (ADR 0133).

Native media plane

Each media-enabled active bot or user-session body owns exactly one clankvox child. A text-only official-bot process owns none. Apache product code speaks through @clankie/vox-client; the AGPL executable owns DAVE, RTP/RTCP, codecs, capture, TTS/music pacing, screen-watch, and Go Live publishing. The TypeScript DiscordVoiceSession retains consent, attribution, floor, realtime-provider, and captain-handoff policy.

Readiness is role-specific: versioned process_ready must exactly match the client IPC protocol and proves only that the child can serve IPC; transport_state=ready proves a role's Discord media transport; and positive dave_state=ready proves that role's negotiated DAVE session. Primary voice ready, connection, transport, DAVE, and error events are correlated by the caller's connectionId. The detailed current diagram and evidence rules live in ADR 0128.

Current architecture constraints

Clankie uses pi's ModelRuntime and createAgentSession for the captain's models, sessions, tools, skills, and compaction. The captain, HTTP surface, and play host share one service (ADR 0101). Herdr exposes the coding-agent fleet as visible panes coordinated through its CLI and plain files. Untrusted input stays fenced, secrets stay in the credential broker, and every report describes observed outcomes rather than intentions.

adr/ records the active decisions for play mechanics, voice, presence, media, browsing, and operator control.

Distribution

The macOS Apple silicon release preserves these process boundaries inside one self-contained, versioned directory. A native clankie launcher starts the bundled TUI and Node runtime; the supervisor starts compiled service entrypoints instead of pnpm workspace scripts. Runtime state and credentials remain outside the immutable release. ADR 0136 records the decision, and distribution.md documents the artifact and installer. Product skills and clankie doctor travel with the release so he can describe and set up this machine without a git tree (ADR 0142).

Canonical Homes