▋
platform/local-llm-curator/SKILL.md

name: local-llm-curator description: Audit, maintain, and serve local LLM coding models on the user's Apple Silicon machine. Use when asked to choose, install, compare, inventory, update, prune, or sync local coding models across MLX, Ollama, LM Studio, llama.cpp/GGUF, or Hugging Face caches — and when asked to run, start, stop, or troubleshoot a local OpenAI-compatible endpoint (omlx on 127.0.0.1:8000) that an agent or harness points at.

local-llm-curator

Use this skill to keep local coding models current without loading model-maintenance procedure on every turn.

Workflow

  1. Run scripts/model-sync.sh audit to inventory local models and caches. It covers ollama, oMLX, LM Studio, and the Hugging Face cache, and reports a size per entry — on a fixed-RAM machine size is the decision, so lead with it. Pipe through column -t -s$'\t' to read it.
  2. For recommendations or installs, verify current availability and benchmarks from primary sources — official model cards, release posts, technical reports, runtime docs — before naming a model. Recognizing a model name is not knowing its current state; local coding models turn over within months, so search the name as the user wrote it even when it looks familiar.
  3. Apply the user's retention policy from ~/AGENTS.md: keep one daily driver, one fast fallback, and optionally one experimental model; prune superseded near-duplicates.
  4. Prefer MLX on Apple Silicon when a suitable build exists. Fall back to GGUF/Ollama/LM Studio when easier deployment or compatibility is the deciding factor.
  5. Before deleting anything, run scripts/model-sync.sh prune-plan --want <file> and present the removal commands for approval.
  6. Only run destructive removal commands after explicit user approval.

Serving a Model Over HTTP

Inventorying a model does not make it reachable. Agents that talk to a "local model" (Clankie's model add-local, Codex, opencode) need an OpenAI-compatible endpoint, and omlx (uv tool, ~/.local/bin/omlx) is the server on this machine — defaults to 127.0.0.1:8000, which is what those configs point at.

omlx serve --port 8000    # foreground, multi-model; background it yourself
omlx diagnose
curl -s http://127.0.0.1:8000/v1/models   # the only honest list of what it serves

omlx start/stop/restart are advertised in --help but only work for the macOS-app and Homebrew installs. This machine's omlx is a uv tool, where they exit with "Background start is available for the macOS app and Homebrew installs." Use omlx serve and detach it (nohup … &, or a herdr pane).

omlx serve discovers models from subdirectories of model_dir (default ~/.omlx/models), so a model id is a directory name, not a Hugging Face repo id and not a slug — populate it by pointing model_dir at the download or placing the snapshot there. Never trust a config's declared model ids; curl /v1/models while the server is up and reconcile.

Two things that read as "the client is broken" when the server is fine:

  • ~/.omlx/settings.json sets auth.api_key with skip_api_key_verification: false by default. A caller sending no key, or the wrong one, gets a 401 — including agent harnesses that assume a local endpoint is keyless, and including client-side model probes that then report the endpoint as unreachable rather than as unauthorized. Read the key out of settings and store it in the caller's own credential store, or set skip_api_key_verification: true for a loopback-only server.

  • Cold decode is not the model's speed. The first request after load pages weights in off SSD; a published tok/s only shows up once warm. Measure the second request, not the first.

  • A model 50x slower than its card is swapping, not misconfigured. iogpu.wired_limit_mb is unset by default, so MLX buffers are pageable and macOS evicts model weights under pressure — decode then pays a disk read per token. Measured here: DeepSeek-V4-Flash-0731-2.4bit-mixed at 0.7 tok/s against a published 36 tok/s on the same machine, with sysctl vm.swapusage showing 51 GB of 52 GB used. Check swap first; it is the single most likely cause of a wildly slow local model, and no cache or scheduler setting will fix it.

    sysctl vm.swapusage                        # used ~= total means thrashing
sudo sysctl iogpu.wired_limit_mb=124518    # omlx prints the right number for the machine at startup

    Two traps when reading memory during this: a loaded MLX model shows a small ps RSS (Metal allocations are not RSS, so omlx-server reports ~17 GB while holding 79 GB), and memory_pressure reporting most memory free right after a request proves the weights were paged out, not that there is headroom. Trust vm.swapusage and the server's own Loaded model: … (actual: …) line.

  • A large quant evicts everything else. ~/.omlx/logs/server.log shows the real reason a request stalls or the engine restarts, e.g. Settle barrier timed out ...: freed=78.62GB (need>=86.24GB). Check that log before blaming the client. The memory guard ceiling lives under memory in settings.json. A ~79 GB model wants the machine to itself — close the browser/Electron pile before benchmarking or the measurement is of swap, not the model.

Desired-Model Files

Use a plain text file for --want, one model identifier per line. Blank lines and # comments are ignored. Identifiers may be exact installed names or substrings such as:

# daily
Qwen3-Coder-Next

# fast fallback
Qwen3-Coder-30B

Substring matching keeps every model containing the identifier, so a want entry that is a prefix of another installed model silently keeps both (e.g. qwen3.6:27b-mlx also keeps qwen3.6:27b-mlx-bf16). Always check the prune-plan output for unintended keep rows and remove those models explicitly.

A model oMLX serves is an install like any other: it lives as a directory under its model_dir and is invisible to ollama list and to the Hugging Face cache listing, while being the largest thing on the machine. The audit covers it; do not reach for a per-runtime command instead.

The script intentionally does not encode a permanent "best model" list. Current model choice belongs in the skill workflow, after fresh research.