ClankVox
ClankVox is Clanky's presence plane: the dedicated native media plane that puts Clanky where the humans are. Today that means Discord — voice (Opus, E2EE via DAVE, real 20ms send pacing) and Go Live streaming and screen share. It is the layer Clanky delegates to whenever presence needs realtime sockets, codec work, packet timing, encryption, or low-level media telemetry that the Node brain is too slow and unequipped to run.
Discord is the only platform ClankVox targets today, and this repository documents that Discord transport. Another platform's media transport would live at this same layer, behind the same invariant.
The invariant: ClankVox stays deterministic; Clanky applies policy. Clanky stays agentic — prompts, settings, gateway control, Realtime sessions, tools, Pi delegation, and product behavior. ClankVox stays deterministic — media transport mechanics, RTP/RTCP, codecs, transport encryption, playback pacing, media capture/publish, and IPC. For Discord that means Opus, DAVE, H264/VP8, music PCM, and Go Live watch/publish.
1. What You Can Do
ClankVox makes native media workflows available to Clanky. Its Discord-backed workflows include:
- join a Discord voice channel and capture speaker audio
- stream assistant speech or music back into Discord with correct pacing
- keep transport truth local instead of guessing from Node-side queued bytes
- receive native Go Live frames for screen-watch workflows
- publish narrow H264-backed Go Live sources when Clanky orchestrates the source
- keep voice, screen, and playback telemetry available to Clanky's floor-control policy
2. What Clanky Lets ClankVox Handle
Clanky delegates realtime media transport work that should not live in the Node runtime. For Discord, that includes:
- Discord voice and stream-server sockets
- UDP/RTP send and receive
- codec advertisement and packet framing
- DAVE session lifecycle and encryption/decryption
- Opus encode/decode and PCM normalization
- inbound audio/video capture events
- outbound TTS/music playback cadence
- native Go Live watch and self-publish media paths
ClankVox does not decide whether the agent should answer, interrupt, remember, search, or delegate. It reports transport truth. Clanky applies product policy.
3. Mental Model
The canonical media-plane map is docs/diagram.md.
Clanky owns product policy and ClankVox owns deterministic media mechanics; they
meet at the stdin/stdout IPC boundary. The Discord transport exposes three roles
— voice, stream_watch, and stream_publish — because Go Live is a separate
stream-server leg, not an extra field on the normal voice socket.
Runtime Shape
The entrypoint is src/main.rs. At startup ClankVox:
- installs rustls crypto
- starts a single IPC writer and reader
- creates shared
AppState - enters one
tokio::select!loop - multiplexes IPC, voice events, music events, reconnect timers, and the 20ms send tick
Most behavior is split across supervisor-style modules:
- src/app_state.rs: shared state and transport slots
- src/connection_supervisor.rs: connect, disconnect, and reconnect control
- src/capture_supervisor.rs: inbound audio/video events and subscriptions
- src/playback_supervisor.rs: TTS/music playback and periodic send tick
- src/stream_publish.rs: outbound Go Live sender pipeline
- src/voice_conn.rs: Discord voice/stream transport, WebSocket, UDP, RTP, codec negotiation, DAVE, and packetization
- src/ipc.rs: Clanky <-> Rust message contracts
What To Read
- Architecture: process model, ownership boundaries, transport roles, IPC, and module map.
- Diagram: docs-UI-friendly media-plane map.
- Audio Pipeline: capture, TTS, music, playback pacing, and telemetry.
- Go Live: native screen watch, native self publish, stream discovery, sender/receiver flows.
For build/test commands, logging knobs, edit-location maps, and cross-repo
coordination points, read the clankvox-dev agent skill
(.agents/skills/clankvox-dev/SKILL.md);
CONTRIBUTING.md covers the human-contributor workflow.
For the product layer above ClankVox, jump to Clanky Start Here.
Build And Test
cargo test
cargo build --release
pnpm docs:dev
Plain cargo commands work: .cargo/config.toml pins
the build environment (OPUS_STATIC=1, OPUS_NO_PKG=1, and
CMAKE_POLICY_VERSION_MINIMUM=3.5 — audiopus_sys builds bundled libopus
through cmake, and cmake >= 4 refuses its vendored cmake_minimum_required
without that floor).
From the sibling agent repo (../clanky-agent), build through Clanky's
idempotent wrapper and exercise the IPC boundary with the voice smokes:
pnpm clankvox:setup # build the release binary (idempotent; --force to rebuild)
pnpm smoke:clankvox:ipc # IPC framing round-trip
pnpm smoke:voice:config
pnpm smoke:voice:control
pnpm smoke:voice:fault
pnpm smoke:voice:realtime
Discord Boundaries
- ClankVox is Clanky's native voice/media transport plane; Discord voice and Go Live are the transports it implements, and Discord is the only platform it targets today.
- Inbound native screen watch is integrated end to end through
stream_watch. - Outbound publish exists and is intentionally narrow: YouTube-backed music/video URLs plus browser-session PNG frames, H264 sender transport, and Clanky-owned source orchestration.
- Native Go Live behavior depends on Discord user-token/selfbot flows.
- Go Live DAVE video decrypt and raw UDP keyframe feedback remain the important transport constraints; see Go Live for detail.