▋
creative/song-stems/SKILL.md

name: song-stems description: >- Separate a song into stems (vocals, bass, drums, keys...), analyze key/tempo/per-bar chords and the vocal melody, reharmonize with a new progression rendered as tempo-aligned MIDI, and remix the stems. Use for "isolate the vocals", "what key/chords is this", "make the harmony darker/jazzier", stem-swap remixes, acapella extraction, de-reverbing/drying a vocal, regenerating the instrumental in the original's texture with a local model (ACE-Step cover), or diagnosing "my remix sounds the same as the original".

song-stems

Pipeline: demucs (separation) → scripts/analyze.py (tempo/key/chords/melody → chords.json) → you pick the new harmony → scripts/progression.py (MIDI) → fluidsynth or Logic (render) → ffmpeg (remix). Everything is headless except the optional Logic last mile.

Work in a scratch dir per song. Keep every intermediate at the source sample rate — never resample mid-pipeline or stems drift out of alignment.

1. Separate

demucs -n htdemucs_6s -d mps -o stems "song.wav"
# -> stems/htdemucs_6s/<songname>/{vocals,drums,bass,guitar,piano,other}.wav
  • If demucs is missing: uv tool install --with numpy demucs (plain uv tool install demucs crashes at runtime — demucs doesn't declare numpy).
  • First run downloads the model (hundreds of MB); output nests under stems/<model>/<songname>/, which surprises everyone.
  • -n htdemucs gives 4 stems (vocals/drums/bass/other) and is faster; use it when keys/guitar don't need separating. --two-stems=vocals for just acapella+instrumental.
  • Before remixing, verify that the vocal stem is not tracking the full mix: scripts/check_bleed.py song.wav stems/htdemucs_6s/song/vocals.wav. It reports high-passed envelope correlation, quiet-frame fraction, and relative energy; treat them as comparative diagnostics across separation models, not universal thresholds.
  • Run check_bleed on the INSTRUMENTAL stem too before blaming separation: vocals corr high + instrumental corr low means the mix is vocal-dominant, and every model will "fail" the vocal check identically. On such tracks (wet cloud-rap etc.) the song's identity — including its old harmony — rides in the vocal's autotune and delay/reverb tails, so a remix keeping the wet vocal keeps sounding like the original no matter what plays underneath. Strip tails from the acapella with audio-separator <vocal> --model_filename UVR-DeEcho-DeReverb.pth, and A/B wet vs dry vocal over the new bed by ear. For top-tier isolation first: audio-separator song.wav --model_filename model_bs_roformer_ep_368_sdr_12.9628.ckpt (install: uv tool install --with audioread "audio-separator[cpu]" — audioread is an undeclared dep, same disease as demucs/numpy).

2. Analyze

scripts/analyze.py song.wav \
  --stems stems/htdemucs_6s/song/piano.wav stems/htdemucs_6s/song/guitar.wav stems/htdemucs_6s/song/bass.wav \
  --melody stems/htdemucs_6s/song/vocals.wav \
  -o analysis

Writes analysis/chords.json — tempo, key, first_downbeat_sec, per-bar chord/roman/confidence, per-bar melody_pcs (vocal pitch classes, strongest first) — plus a .mid transcription of each stem (import these into Logic too; they're the editable versions of the parts).

  • --stems feeds the chord detector transcribed notes (cleaner). Without it, chords come from chroma of whatever audio you pass — passing the mix works but is noisier.
  • --melody is kept OUT of the chord histogram on purpose; it exists so you can reason about melody-vs-harmony.
  • Assumes 4/4; pass --beats-per-bar otherwise. Chord labels are maj/min triads only — read confidence and treat low values as "ambiguous", not gospel.
  • basic-pitch requires Python <3.12 (the script pins this; uv fetches 3.11 automatically). On macOS it runs via CoreML — the tflite/onnx/tensorflow warnings at startup are noise, not errors.

3. Reharmonize (this is your job, not a model's)

Read chords.json and reason like an arranger. The style prompt ("darker", "jazzier", "melancholic") maps to concrete moves — minor iv borrowings, tritone subs, added 7ths/9ths, deceptive cadences, relative-minor recentering — but the constraint that matters is the melody: each bar's melody_pcs should land as chord tones or intended tensions of the new chord. Propose, then verify:

scripts/progression.py "Am9 Fmaj7 Dm7 E7" --bpm <tempo_bpm> \
  --offset-sec <first_downbeat_sec> --loops <n> \
  --check analysis/chords.json -o new_chords.mid

--check warns per bar where a melody pc sits a b9 above a proposed chord tone — the one interval that reliably sounds broken. Warnings are advisory; everything else is taste (on a vocal that glides between adjacent semitones, flags against both pitches are transcription artifacts — compare against how the ORIGINAL chords would flag). Chord symbols are music21 harmony syntax (Am, G7, Cmaj7, F#m7b5, Bb/D, BbmM7; flat roots are auto-normalized to music21's B- form).

A progression can be one looped pattern (--loops) or the full song written out bar-by-bar (loops=1) — write it out when the song has sections; a fixed loop fights them. Vamps need this too, and it is not optional: irregular bar counts are the norm (Bbm x2, Gb x1, Bbm x4...), so an N-bar loop walks out of phase within seconds and reads to the listener as "off-rhythm and out of key" even when every chord is diatonic. Before rendering, count how many bars of your progression land on a different root than chords.json — a fixed loop scored 72% wrong on a two-chord vamp; generating from the song's own change points scored 17%. Build the sequence by grouping the analyzed bars into runs of the same chord and substituting per run, so changes happen exactly where the song already changes; add motion by varying the substitute inside long static runs, not by imposing a faster grid.

Also generate the matching bassline: --roots --octaves -1 --program 38 (root notes in 808 register, synth bass); keys sit better as --octaves 1 --program 88 (warm pad) since default voicings land in muddy octave 2-3.

Harmonic rhythm: read the bass transcription, not the bar chords

chords.json is quantized to whole bars, which throws away the thing the listener actually tracks. Before writing any bassline, dump the note times from analysis/bass.mid — on an 808 track the bass note IS the chord, so its onsets are the ground truth for cadence. What reads as "Bbm for four bars" is usually a syncopated riff whose pitch changes rarely land on a bar line; whole-bar roots over that are rhythmically wrong no matter how right the chords are, and the user hears it as "off-sync" and "out of key" even when every pitch is diatonic.

So keep the source's rhythm and change only the pitches:

scripts/riff808.py analysis/bass.mid prog.txt -o riff808.wav \
  --bpm <tempo> --offset-sec <first_downbeat_sec> [--decay 1.3] [--drive 2.5]

It reads each transcribed note's onset/duration verbatim and snaps its pitch to the nearest tone of the new chord for that bar (root/3rd/5th only — 7ths and 9ths turn to mud in the sub register), preserving both groove and contour. Measured against a rigid bar grid on the same song: whole-bar roots drifted −0.61s across the track while the onset-locked version showed no systematic drift, and envelope correlation with the original bass went 0.07 -> 0.22. prog.txt is one chord symbol per bar (see above).

Caveat worth stating to the user: this changes which chord the riff outlines, not the riff's interval shape — inside a long tonic run it will still oscillate the way the original did.

4. Render the new harmony

Headless (iteration):

fluidsynth -ni -F new_keys.wav -r 44100 \
  ~/Library/Audio/Sounds/Banks/MuseScore_General.sf3 new_chords.mid
  • Flag order is strict in fluidsynth 2.6+: all options BEFORE the positional soundfont/MIDI args, or it errors with "illegal option at this place".
  • Pass -g 1.2: the default gain (0.2) renders around −40dB mean — inaudible under a mix. Always volumedetect a render before mixing it.
  • Match -r to the stems' sample rate.
  • If fluidsynth is missing: brew install fluid-synth. It ships no soundfont; probe ~/Library/Audio/Sounds/Banks/ and if empty fetch the GM soundfont (Homebrew's build plays compressed .sf3 fine): curl -fsSL -o ~/Library/Audio/Sounds/Banks/MuseScore_General.sf3 https://ftp.osuosl.org/pub/musescore/soundfont/MuseScore_General/MuseScore_General.sf3

For taste (final version): import new_chords.mid into Logic and pick a real instrument instead — see step 6.

4b. Generative texture bed (ACE-Step cover)

When GM/synth renders sound nothing like the source's production, repaint the harmony in the original's texture with ACE-Step 1.5 (local, ~/dev/ACE-Step-1.5, MLX on Apple Silicon at ~2.3x realtime; see that repo's docs and any per-song ACESTEP_NOTES.md for the full API): task_type=cover with reference_audio = original song (timbre/mix) and src_audio = a rendered instrumental of the NEW harmony (structure), plus explicit bpm / key_scale / time_signature. Uploads must be multipart file fields — the server rejects absolute paths despite what API.md says.

Server start, model init, poll codes, run budget, src_audio construction, and the audio_cover_strength timing sweep are in references/ace-step-cover.md — read it before the first call; the API's status codes and lazy-load behaviour are not guessable.

  • Never let the bed supply the bass. It is a texture generator, not a bass synth: on an 808 track whose original bass measured −6dB below 150Hz, the bed's bass stem came back at −22dB and the remix lost 8dB of sub. Render the new roots with render_808.py and mix that in explicitly; let the bed carry pads and texture only. Gate every remix on lowpass=f=150,volumedetect against the original before shipping.
  • Rebuilding a mix from stems always reads a few dB light above 200Hz (demucs can't recover correlated/phase content), so don't chase the original mix's hp200 number — compare against the sum of the stems you kept. Also check the source 808's own hp200 before adding --drive: a pure sub 808 can have less upper harmonic content than the renderer's default, and over-driving moves you away from its character.
  • The bed comes back at 48kHz regardless of source rate. This is the one place to resample: convert it to the stems' rate once, up front. Duration is preserved, so nothing drifts.
  • The bed contains its own drums — mix it with the acapella ONLY, or drums flam. To keep the original drums instead, run demucs on the bed a second time and discard its drum stem (bass+other+vocals of the bed = its harmonic content in the original's texture); mix that under the untouched original vocals + drums. Costs one extra separation pass and is the right default when the brief is "harmonic/bass remix, keep the feel".

5. Remix

Don't rebuild the song from stems if you are only changing one band. Reassembling all six untouched stems already measures ~3.6dB down above 200Hz against the source mix — demucs cannot put back what it took apart, and that loss lands on every version you ship no matter how good the musical work is. Run that control FIRST whenever a user says a remix "feels off compared to the original": mix the unmodified stems and compare highpass=f=200,volumedetect to the source. It separates "my part is wrong" from "separation cost me 4dB", which no amount of chord-fixing will recover.

Prefer, in order:

  1. Add to the untouched source mix. A pad layer high-passed above the bass costs ~0.9dB and keeps the original master, vocals and drums bit-for-bit.
  2. Band-swap the source mix. To replace an 808, high-pass the ORIGINAL at ~120Hz (two 2-pole stages) and mix the new bass underneath — measured ~2.3dB down, still better than a rebuild. Check first that the target band really is that instrument alone: lowpass=f=120,volumedetect on the mix, the bass stem, and the drum/vocal stems — when the mix and bass stem read within a dB of each other and everything else is ~30dB down, the crossover costs nothing.
  3. Full stem rebuild. Only when most stems actually change.

Adding anything to an already-brickwalled master makes its limiter clamp the sub; expect ~2dB of sub loss and don't chase it with gain, which just limits harder.

Note the interaction with reharmonization: if the user asks for a colour-only reharm (extensions, no root substitutions), the new chord roots ARE the original's, so any bass you synthesize is trying to reproduce notes already on the record. Keep the original bass and put the reharm in the keys — re-synthesis can only lose.

ffmpeg -i vocals.wav -i drums.wav -i bass.wav -i new_keys.wav -filter_complex \
  "[0]volume=1.0[a];[1]volume=0.9[b];[2]volume=1.0[c];[3]volume=0.7[d];\
   [a][b][c][d]amix=inputs=4:normalize=0:duration=first" remix.wav
  • Before choosing which stems to drop, volumedetect each one: the harmony often lives somewhere unexpected — in 808-heavy genres it's the BASS stem (keys/guitar stems can be near-empty at −35dB). Reharmonizing means REPLACING the bass with a bassline on the new roots, or the old harmony stays; a remix that keeps the old bass and adds quiet new keys sounds identical to the original.
  • normalize=0 is load-bearing: amix silently divides by input count without it.
  • Boosted stems can sum past 0dB — finish the filtergraph with alimiter=limit=0.95:level=false (level defaults to TRUE and auto-gains the output straight back to full scale, silently undoing the ceiling) and check max_volume on the output.
  • If the user says a remix "sounds the same": measure the audible band, not the full mix — highpass=f=200,volumedetect. Sub-heavy tracks carry nearly all their energy below 150Hz, which small speakers can't reproduce, so a change confined to sub bass (or a pad 6dB under the vocal) is inaudible on laptops. New harmonic content must land above 200Hz at roughly vocal loudness: bass roots at octave 2 (not 1), keys boosted until their in-mix mean rivals the vocal stem's.
  • duration=first trims the fluidsynth decay tail (the render runs seconds past the last note-off) so the mix stays the source length — put a stem first.
  • All demucs stems are already time-aligned with the source; new_keys.wav aligns because of --offset-sec. If anything sounds flammy, check sample rates first.

6. Logic Pro last mile

Logic has no scriptable API — community MCPs drive it by keystroke/Accessibility simulation, fine for transport toys, too fragile for a pipeline. So hand Logic finished files and let the human mix:

open -a "Logic Pro" stems/htdemucs_6s/song/*.wav new_chords.mid analysis/*.mid

MIDI opens as a new Logic project with a software instrument track; audio files drag in as tracks. If the user wants a DAW fully in the agent loop, that's Ableton (real Remote Script API + mature MCP), not Logic — say so rather than fighting it.

Verification

analyze.py, progression.py, render_808.py, riff808.py, and check_bleed.py have --self-test; run the relevant checks after editing.