▋
platform/record-unreal/SKILL.md

name: record-unreal description: >- Record real video of an Unreal Engine window on macOS — an editor viewport playing an animation, a PIE session, or a packaged game — for review or evidence, and know which capture method survives a shared desktop. Use for any "record the editor/game", "get me a clip of it running", or "the capture came out black/occluded/empty" request. Not for in-engine offline renders (Movie Render Queue) or for per-frame sampled screenshots meant to verify poses; those are deterministic checks, not recordings.

record-unreal

Read the capture method and its traps before the first attempt. For a new or changed capture path, run the short native and recording check before a long take. Reuse that evidence for unchanged inputs; a mock-engine pass or a sampled render cannot establish that the real recording path works.

Choose the method first

Reuse the project's proven window-ID recorder first. Grapple's successful live gameplay captures use ScreenCaptureKit (SCContentFilter with desktopIndependentWindow, then SCRecordingOutput). Reuse the complete launcher, input path, first-frame inspection and finalizer, not just the Swift recorder. See the retained Grapple recipe.

Native CLI window capture (when no proven project recorder exists). screencapture -v -V<seconds> -l<id> is a bounded native alternative. Earlier measurements showed it recording the window's own pixels through overlap; later Unreal attempts failed to create an image or movie. Neither result proves all window capture works or fails on the current desktop. Inspect the actual short take before adopting it; do not replace a working SCK path by default.

Screen-region capture (only when a fixed rate matters). ffmpeg's AVFoundation screen device at a requested 60 fps, cropped to the window's bounds. It records whatever is on screen, so it needs a genuinely clear region, and "clear" is harder to establish than it looks (below).

Native CLI window capture: incantation and facts

Find the window from the process id with the CoreGraphics window list; the first window listed for a process is often a notification toast, so pick the largest one and skip any that report no size:

osascript -l JavaScript -e '
ObjC.import("CoreGraphics");
var ws = ObjC.deepUnwrap(ObjC.castRefToObject(
  $.CGWindowListCopyWindowInfo($.kCGWindowListOptionOnScreenOnly, $.kCGNullWindow)));
JSON.stringify(ws.filter(w => w.kCGWindowOwnerPID === PID_HERE && w.kCGWindowLayer === 0)
  .map(w => ({id: w.kCGWindowNumber, name: w.kCGWindowName, b: w.kCGWindowBounds})));'

Then record, bounded, and remux without re-encoding:

screencapture -v -V8.5 -l<windowid> take.mov          # -V accepts a float
ffmpeg -i take.mov -map 0:v:0 -c copy -movflags +faststart take.mp4
  • The frame rate is content-driven. An idle terminal window produced 7.5 fps, the same window while scrolling about 26, an actively redrawing window 35.1 over 3 s, a running packaged game 33.5, and an editor viewport playing an animation 47.2 (151 frames over 3.20 s); rate follows redraws. Read the decoded frame count from the output and write it into the packet; never state a requested rate. A paused viewport yields a sparse movie, not a slow one.
  • Let screencapture -v -V run to its own end. Unlike ffmpeg, it does not finalize on SIGINT: a finalizer that interrupted it before -V elapsed left no movie at all. Keep the captured window alive until the process exits, and give any outer timeout room for the full -V plus finalization.
  • Window capture works on any display. The primary-screen constraint that region capture needs does not apply, so do not refuse an off-primary window or file a capture error as an occlusion.
  • The window list reports geometry in points, not pixels. On this Retina display a 1280x720 play-in-editor window comes back as 640x384 (360 plus the title bar), while the movie it yields is still 1280x720. A size filter written in pixels rejects the real window forever and reads as "no play window". Match the play window by its name (GrappleGame Preview [NetMode: ...]) or by owner pid plus layer 0, never by pixel size; and take the owner pid from the editor binary itself, not from a slot wrapper whose command line also names the project.
  • Window-ID capture survives overlap, not movement: a measured +529,+8-point translation shifted the movie +1058,+16 pixels inside a fixed canvas and clipped its right edge. Establish the owned window's intended position/size before recording and verify the actual PID, window ID and bounds. Monitor them through playback and timed finalization; a move, resize or replacement fails the take. Restore placement only before a new take, never silently continue a clipped one. Window snapshots do not identify who moved it.
  • Bound every capture with -V. A recorder that waits for something else to finish is the thing that hangs.
  • Wrap every osascript call in a timeout. A blocked Apple Event returns nothing, forever, and the only symptom is an empty log.

Region capture: incantation and the shield

ffmpeg -f avfoundation -list_devices true -i ""     # find "Capture screen N"
ffmpeg -f avfoundation -framerate 60 -capture_cursor 0 -i "<N>:none" \
  -filter_complex 'crop=W:H:X:Y' -c:v h264_videotoolbox -realtime 1 -b:v 10M out.mp4

Crop values are window bounds multiplied by the screen's backing scale (2 on a Retina display) and rounded to even numbers.

  • This machine keeps a permanent, invisible, full-screen window at layer 24 named Screenshot (ScreenshotControls.appex, alpha 1, no title, no content). Any occlusion test that asks the window list "is something in front of my region" reads occluded forever. If you must gate on occlusion, prove it by pixels: grab the region and grab the window alone, subtract a noise floor measured by grabbing the window twice (a live viewport differs from itself by 0.6–0.9%), and call it occluded only on the excess. Or use window capture and skip the question.
  • A layer-0 window that gets focused after your pre-flight does not fall behind a raised editor. Pre-flight rules out floating windows only.
  • Raising or moving windows from a headless process needs the Accessibility entitlement it usually lacks: activateWithOptions returns true and does nothing, and System Events replies with AppleEvent error -10000. Measure where the window actually is; do not plan on placing it.
  • Screen Recording permission is per process (CGPreflightScreenCaptureAccess in a JXA probe reports it). Check permission first for a black recording; black pixels alone do not identify the cause.
  • A uniform one-colour full-screen probe is the display asleep, and the wake tap fixes it. This is the common outage and it is not a permission problem: the grant is intact, the window is listed on screen, and SCK simply never receives a frame because nothing is being composited. caffeinate -u -t 5, wait ~2 s, then re-probe until the screen has content; hold caffeinate -d -i for the take's whole duration, because a display that sleeps mid-route truncates the recording to a handful of frames without failing anything. Build both into the recorder rather than relying on someone holding the machine awake.
  • A locked screen is the one the colour probe cannot see, and it is the likeliest cause of a take that records but at two frames a second. macOS composites almost nothing for background windows while the session is locked, so the recorder's first frame is a perfect editor and the movie is still 600 frames over 300 s. The lock screen is a full-colour wallpaper, so the colour probe passes happily. Check the flag directly and refuse the take: ioreg -n Root -d1 -w 0 and look for "CGSSessionScreenIsLocked"=Yes (the neighbouring CGSSessionScreenLockedTime is a unix timestamp, which is how you match it against the take that went bad). Only the person at the machine can unlock it. On 2026-09-10 a lock at 04:46 turned a 57 fps path into a 2 fps one, and the frame rate, not any error, was the only symptom.
  • Probe before every take, and judge it on colour count rather than brightness. screencapture -x shot.png then magick shot.png -resize 32x20! -format "%[fx:mean] %k" info:. A live desktop is megabytes with hundreds of distinct colours in that sample; a sleeping one is ~144 KB, uniform, %k of 1 to 5, and reads mean 0.25 rather than 0 — so a brightness test calls it a normal dark screen. On 2026-09-10 a session went 17198 frames over 300 s, then 46 frames over 300 s as the display dozed mid-take, then none.
  • The two failures are different and only one needs a human. Display sleep: uniform probe, window listed, could not create image from window, no first frame — wake it and carry on. Lapsed Screen Recording grant: an actually black PNG and the SCK recorder failing with RPRecordingErrorDomain -5822 Failed due to failure to process first sample buffer — that one needs the terminal re-granted in System Settings and no amount of waking helps.
  • The CoreGraphics window list looks perfectly healthy through both, so it proves nothing. During a total outage the target window still reported kCGWindowLayer 0, kCGWindowIsOnscreen true, kCGWindowSharingState 1, kCGWindowAlpha 1 and its correct bounds and title. Do not conclude a window is capturable from the window list, and do not go looking for occlusion, Spaces or dimensions first — that detour cost several takes and a wrong cause published on a ticket.
  • LaunchServices activation is not the fix. open -a on the editor's app bundle (/Users/Shared/Epic Games/UE_5.8/Engine/Binaries/Mac/UnrealEditor.app) does raise an already-running editor without the Accessibility entitlement and does not start a second instance, which is worth knowing — but it does nothing for a sleeping display.
  • The permission lapses: macOS re-asks for Screen Recording periodically, and when the terminal's grant has lapsed every path fails at once with the same signature, none of which names permission: screencapture -x of the whole screen returns an all-black PNG, screencapture -l<id> says could not create image from window even for the frontmost window, and the SCK recorder dies with RPRecordingErrorDomain -5822 Failed due to failure to process first sample buffer at the first frame. A JXA CGPreflightScreenCaptureAccess probe can still say true because osascript is judged separately. Run the full-screen screencapture -x first and look at it; if it is black, stop taking editor takes and get the grant restored in System Settings › Privacy & Security › Screen & System Audio Recording for the terminal app (Ghostty here). Window dimensions, occlusion and window order were all red herrings for three takes.
  • Game, presentation and capture clocks are independent. A 30 fps cap under 60 fps capture does not guarantee exact pairs: retained 1/2/3-frame groups coexist with near-33.3 ms engine frames. Compare capture timestamps with engine/presentation timing before attributing an uneven stride to game pacing or animation; duplicated capture frames alone cannot decide it.

Frame for the window you record, not the one you render

For an owned level viewport, LevelEditor.ToggleImmersive in -ExecCmds uses Unreal's native immersive view before recording. Inspect the short take afterward; the command toggles the current state and does not fix window geometry.

A recording of an editor window is not a render. Two things follow, and both cost a full native run to learn:

  • Frame the camera for the viewport's aspect. A level viewport inside a 1400x900 editor window measures about 1105x430 logical — aspect 0.39, far wider and shorter than 16:9. A camera positioned to fit a body in 16:9 necessarily overflows that viewport vertically, and what overflows is the feet. Compute the camera distance from the frame ratio you will actually record in.

  • A sampled-render check does not prove recording framing. Per-frame screenshots render at their own resolution and can pass every framing and luminance rule while the recording of the same scene cuts the character in half. A non-black first frame only proves the capture was not black. If the recording is the deliverable, check the recording.

  • The level viewport does not use the piloted camera's FOV. Placing a camera actor, piloting it and driving the viewport still renders at the viewport's own field of view — measured at about 82 degrees horizontal where the camera actor said 60. Sizing a shot from the camera's FOV predicted a half-frame subject and delivered a third of one. Calibrate instead: record once, measure what share of frame height the subject occupies, and solve the distance from that. Keep the measured vertical tangent as a named constant and re-measure when the layout or window size changes.

  • Do not store a crop as pixels measured on one capture. screencapture -l returns the window plus a drop shadow whose width is not fixed: two takes of the same 1400x900 window came back 2892x1892 and 3024x2024. Stored offsets then take in the toolbar at the top and cut the subject's feet off at the bottom. Store the region in the window's own logical points and derive the pixel offset from each capture's margin, (capture - window * scale) / 2.

  • Frame on the body, not the path. Bounds over every frame describe the box the subject travels through, so the camera retreats far enough to see the whole journey. Take the furthest bone from that frame's root, track the root, and the subject stays the same size throughout.

Crop to the viewport rather than shipping the whole editor window, and tie the crop to the window placement it was measured on — apply it only when the capture is the size that placement produces, so an unrecognised window records whole instead of being cropped to someone else's rectangle. Suppress the engine's own on-screen messages with DisableAllScreenMessages in -ExecCmds; unbuilt lighting and competing directional lights both print over the viewport you are recording, and the engine names that command in the message itself.

Unreal-side traps that end a recording

  • -unattended on the editor makes the -ExecutePythonScript executer exit on its own prompt before the script runs. Do not add it to suppress dialogs.
  • A modal dialog (Delete Assets, Save, Overwrite) blocks Slate ticks, so a script that finishes through a register_slate_post_tick_callback never resumes and the run sits until its outer timeout. Watch from the host (window list, layer ≥ 3 or a matching title), screenshot the dialog into the packet, and stop the editor; a missing end-of-script record marks the run incomplete; on its own it proves neither a modal nor a defect in the thing that would have written it.
  • Asset editor windows open over the level viewport without asking; an animation's editor that was absent at scene staging was present 8 and 13 seconds into two later waits, and what opens it is not yet identified. Close them by path before capture and again on every poll while waiting: enumerate the run's own folder with EditorAssetLibrary.list_assets, resolve each with unreal.find_asset (it returns None for anything not loaded, so an asset with no editor is skipped) and call close_all_editors_for_asset. Report what you asked to close, not what closed: that call returns nothing you can build a receipt from. AssetEditorSubsystem.get_all_edited_assets exists in C++ but is NOT on the 5.8 Python binding — calling it raises AttributeError and aborts the run after every earlier step has passed. Verified on 5.8.2.
  • A test double may only carry APIs proven in a real native run. The binding above went unnoticed because the double defined the method, and the same double returned an object from find_asset for every path where the real one returns None. An offline suite that mocks the engine cannot fail on an API the engine does not have, so mocking is where invented APIs go to look green.
  • Close asset editors before the editor shuts down. Leaving an animation editor open crashes the exit in FAnimationEditor's destructor during Slate teardown; the crash reporter then puts a "quit unexpectedly" alert on screen above layer 0, over your next capture.
  • After any editor exit, reap its helpers by identity before releasing whatever lock you hold: UnrealTraceServer … --sponsor=<editorpid> exits on SIGTERM; CrashReportClient ignores SIGTERM and has ignored a first SIGKILL, so escalate and poll until absent; both reparent to init, so the helper's own argv (--sponsor=<pid>, pid-<pid> in the crash directory) is the only surviving identity. Reap on clean exit too, and sample the editor's pids while it is alive because they are unreadable once it is gone. Recheck the retained process start identity and argv before each signal; PID reuse never transfers ownership. Keep escalation within the original cleanup deadline and report incomplete cleanup if it expires. A successful signal is not proof of absence.
  • Any timeout on the editor side must be longer than any wait on the recorder side, or the editor kills the recorder mid-wait and the packet shows an empty recorder log.
  • unreal.Rotator(...) takes (roll, pitch, yaw); passing (pitch, yaw, roll) aims lights at the sky and the scene renders black.
  • get_socket_location resolves a same-named socket before a bone, case-insensitively; read bones with transform_from_bone_space when a mesh has Foot_L-style sockets.
  • Actors in the editor world do not advance play_animation; set the position per frame and refresh instead.

Packaged game

Launch windowed with an explicit -WinX -WinY -ResX -ResY (or r.SetRes at runtime) so the window's bounds are known, then window-capture it by ID. Two game windows on one Mac record independently; capture the one whose camera shows what you need.

What the packet must carry

Method, window id and bounds, measured frame count and duration, encoder, process exits, a before/after window snapshot, and a label: realtime window/screen recording versus sampled screenshots. Never let a screenshot sequence stand in for a recording; a viewer reads an activity post as a showcase. Project-specific rules (shared editor locks, evidence archives) live in that project's own native-proof skill; this file is only the capture.