Recording How-To Demo Videos of a Native macOS App (Semi-)Automatically: vhs and a Self-Driving Demo Mode

This article was co-authored with generative AI. Facts have been checked against public documentation where feasible, but errors may remain. Please verify primary sources before relying on this for important decisions.

What this is about

This covers how to record how-to demo videos (as automatically as possible) for a macOS native app (built with SwiftUI) I'm making for digital archives. The subject is a tool that "creates a SIP (accession) and an AIP (long-term preservation) from a source folder," but the technique itself applies to any native macOS / SwiftUI app.

There are 2 key points.

1. The CLI flow can be turned into mp4/gif fully automatically (unattended, CI-capable) with vhs
2. For GUI operations, rather than sending clicks from the outside, building a self-driving demo mode into the app itself is coordinate-independent and robust, and it becomes just a matter of recording with screencapture

And I'll write about the "here's where you get stuck" parts specific to native macOS apps (the most impactful being not relying on WindowGroup's automatic window creation, but explicitly creating a window with AppKit).

Premise: why the previous "fully automated recording" doesn't work on native

With the Docker version (Linux GUI) of the same workflow, I could record demos fully automatically. It grabs the X11 display :1 inside the container with ffmpeg x11grab and sends operations with xdotool (there was also a derivative that drove noVNC with Playwright). It needs neither a person nor a real screen, and can run in CI.

None of this works for a native macOS app.

• There is no X11, and AppKit/SwiftUI windows can't be operated with xdotool/Playwright
• Screen recording (ScreenCaptureKit / screencapture) requires a real login session running in the background (Aqua) + the "Screen Recording" permission (you can record even from SSH if you enter that same session via launchctl bsexec, but not fully headless or without a session)

So the "grab the container's virtual display and hit it" approach isn't available. Instead, I use A/B below.

A. Fully automate the CLI demo with vhs

vhs (by Charm) runs a terminal session from a script called a .tape and renders it headlessly straight into .mp4/.gif/.webm. The required components are ttyd and ffmpeg. It draws the terminal with ttyd (xterm.js), uses headless Chromium via go-rod for frame capture (auto-downloaded the first time), and converts to video with ffmpeg. It needs neither a screen nor a person, so it's ideal for CLI how-to demos.

brew install vhs

For the demonstration itself, it's easiest to maintain if you prepare a single narrated walkthrough script (where # comment lines can serve as the basis for subtitles) and just run it from the .tape.

Output docs/media/cli-demo.mp4
Output docs/media/cli-demo.gif

Set FontSize 15
Set Width 1280
Set Height 820
Set Padding 18
Set Theme "Dracula"

Hide
Type "export DEMO_PAUSE=1.0 && clear" Enter
Show
Sleep 500ms
Type "./scripts/demo-walkthrough.zsh" Enter
Sleep 34s    # script's real time + a tail. Take it generously

This outputs the whole pipeline from source → SIP → AIP, plus format conversion, metadata, and validation, as a roughly 30-second mp4/gif. Japanese terminal output is captured as-is.

vhs gotchas

• Sleep does not wait for command completion. vhs just advances along the timeline, so after Type ... Enter you must secure at least the script's real time with Sleep. It helps to measure with time ./script beforehand.
• Japanese fonts. vhs's default font has no CJK, so you can get tofu (missing glyphs). On my machine (macOS) Japanese showed up via the system fallback, but in environments where it doesn't, specify a CJK monospace font (HackGen / Cica / Sarasa, etc.) with Set FontFamily "...". Producing a single PNG from a short .tape to check is the fast way.
• Dependencies. vhs launches ttyd and headless Chromium (go-rod). In environments where local ports or browser launch are restricted (inside a sandbox, etc.), it fails on browser launch failure (Failed to launch the browser / No usable sandbox!, i.e. failure to connect to ttyd). Under a sandbox you may need VHS_NO_SANDBOX=1.

B. For GUI walkthroughs, build a self-driving demo mode into the app

For the actual tutorial that shows the GUI, I first considered the "click from outside using coordinates" plan (cliclick / AppleScript UI scripting). But on native it's fragile, and in particular automating the file picker (NSOpenPanel) is the crux of the difficulty.

Changing the approach: adding a --demo mode in which the app itself operates its own state to demonstrate makes both external clicks and dialog operations disappear and instantly becomes robust. Recording is just a matter of running screencapture -v on the outside.

In short, you just call "the same processing that the GUI buttons call" in order, from code, right after the app launches.

// Parse the launch arguments --demo <source> <output> [--quit] (DemoLaunch).
// The app just operates its own state in order (no external clicks or dialog operations).

@MainActor
enum DemoAutopilot {
    static func runSequence(root: RootState, full: FullAppState) async {
        guard let sample = DemoLaunch.sample, let out = DemoLaunch.out else { return }

        func beat(_ k: Double = 1) async {
            try? await Task.sleep(nanoseconds: UInt64(k * 1_400_000_000))
        }

        root.mode = .home;  await beat(1.0)     // show the entry point (mode selection)
        root.mode = .full;  await beat(0.9)     // ③ go to the end-to-end flow
        full.inputURL  = sample;       await beat()   // fill the input fields in order (no dialog)
        full.title     = "総務課 移管文書"; await beat(0.6)
        full.outputURL = out;          await beat()
        full.run()                              // the same run() as the GUI "Create" button

        while full.isRunning {                  // the progress log streaming is captured as-is
            try? await Task.sleep(nanoseconds: 200_000_000)
        }
        await beat(2.0)                          // completion dialog / result tail
        // Exit with exit(0). With NSApp.terminate(nil) it saves the "window state," and on the next
        // launch it tries to restore it, leading to an accident where no window is created (discussed below).
        if DemoLaunch.quitWhenDone { exit(0) }
    }
}

Make the state (RootState / each screen's AppState) a static let shared singleton so the self-driving code can touch it directly.

The biggest gotcha: create the demo window explicitly with AppKit

Getting to this point took the longest, so I'll write the conclusion first.

SwiftUI's WindowGroup automatic window creation was unstable in a context where you repeatedly launch with open -n. The symptom was "the 1st or 2nd time records, but after that no window is created at all" (not captured in the recording / a different frontmost app is captured). With logging in place, applicationDidFinishLaunching ran every time, yet the WindowGroup's content View didn't appear (NSApp.windows.count == 0 remained), and the .task placed as the self-driving trigger didn't fire either.

Piling on symptomatic fixes — changing .task to fire on applicationDidFinishLaunching, bringing it to the front, removing state restoration — the intermittent misfires didn't go away.

The decisive move was not relying on WindowGroup during the self-driving demo, and explicitly creating an NSWindow + NSHostingView in AppDelegate. applicationDidFinishLaunching always runs exactly once (given a GUI launch into a real login session) regardless of launch context, so if you create the window here it definitely exists.

final class AppDelegate: NSObject, NSApplicationDelegate {
    var demoWindow: NSWindow?  // hold onto it
    func applicationDidFinishLaunching(_ n: Notification) {
        NSApp.setActivationPolicy(.regular)
        NSApp.activate(ignoringOtherApps: true)
        guard DemoLaunch.isRequested else { return }   // leave normal launch to WindowGroup

        // Put the SwiftUI View onto an AppKit window (not relying on auto-creation = reliable).
        let content = RootView()
            .environmentObject(RootState.shared) /* …other shared state too… */
        let win = NSWindow(contentRect: .init(x: 0, y: 0, width: 1280, height: 800),
                           styleMask: [.titled, .closable, .miniaturizable, .resizable],
                           backing: .buffered, defer: false)
        win.contentView = NSHostingView(rootView: content)
        win.isReleasedWhenClosed = false
        win.center(); win.makeKeyAndOrderFront(nil); win.orderFrontRegardless()
        demoWindow = win
        // Pass the CGWindowID to the recording script, telling it "record only this window."
        try? "\(win.windowNumber)".write(to: winidURL, atomically: true, encoding: .utf8)

        Task { @MainActor in
            await waitForGoSignal()              // the cue to start recording (below)
            await DemoAutopilot.runSequence(...) // operate the state in order to demonstrate
        }
    }
}

Keep the WindowGroup side empty (Color.clear) only during the demo, so it doesn't double up with the explicit window. With this, even with open -n from an agent or the background, a window is always created no matter how many times you repeat, and recording became stable.

Record "per window"

With screencapture -v (the whole screen), accidents happen: if another app is in front it gets captured, and with multiple displays it records a different screen. If you record "only that window" with screencapture -v -l <CGWindowID>, it is completely unaffected by z-order, frontmost app, or display (it captures the window's rendered content directly, so no bringing to front or fullscreen is needed).

Note: screencapture's video recording (-v) is macOS 14 Sonoma or later (it uses ScreenCaptureKit internally). Combining -v with -l (window selection) is not documented in man screencapture, but it works on real hardware from Sonoma onward. The id passed to -l is the CGWindowID, which, as described below, the app writes out via NSWindow.windowNumber and hands over.

The skeleton of the recording wrapper (zsh):

pkill -9 -f 'MyApp.app/Contents/MacOS'; while pgrep -f …; do sleep 0.5; done  # wipe out remnants completely
open -n "MyApp.app" --args --demo "$SRC" "$OUT" --quit \
  --winid-out "$WINID" --go-signal "$GO" --demo-schedule … --cue-out …
until [[ -s "$WINID" ]]; do sleep 0.25; done                  # wait for the app to write the winid
WID="$(cat "$WINID")"
screencapture -v -l "$WID" out.mov &                          # record per window
REC=$!; sleep 1; : > "$GO"                                    # signal recording started → self-drive starts
# after waiting for self-drive to finish (cues output / app exit)…
kill -INT "$REC"

Other gotchas

• Signal the recording start with a go-signal. screencapture has a startup lag of a few hundred milliseconds to a few seconds. If self-driving starts before recording reliably begins, you miss the first cut (cue0). So use the ordering "the app writes the winid → the recording script launches screencapture → waits about 1 second, then creates the signal file (: > $GO) → the app starts self-driving only after seeing the signal." This also aligns with the head-trim (contentStartMs) discussed later.
• Normal termination (NSApp.terminate) saves the window state, and the next launch tries to "restore" it (state restoration). When you've built up state for a demo, this can become the underlying cause of "next time it restores with no window" → no window created. It's safer for self-driving termination to use exit(0) (which doesn't ride the normal-termination save flow). Further, before recording, defaults write <bundle-id> NSQuitAlwaysKeepsWindows -bool false and deleting ~/Library/Saved Application State/<bundle-id>.savedState gives you a double defense of "don't let it save + delete any existing save," and combining that with open -n (a new instance each time) plus a thorough pre-launch pkill improves reproducibility.
• Long recordings are ruined by display sleep. Run caffeinate -dimsu -w $$ alongside during recording to suppress sleep/screensaver (-w $$ ties it to the lifetime of the recording script).
• If you launch the self-drive Task from applicationDidFinishLaunching, do it after creating the window. Occupying the main thread before window creation interferes with WindowGroup's window creation (with the explicit-window approach this ordering problem disappears entirely).
• The "Screen Recording" permission is required (System Settings → Privacy & Security → Screen Recording → turn ON the terminal you run from). A one-time grant.
• Separate validation from recording. If you make it so launching with --demo --quit produces output (e.g. the artifact directory), you can determine that the self-driving mechanism is working, which lets you tell apart "is the code bad or the environment bad." After recording, always pull one frame and inspect it visually (ffmpeg -ss <sec> -i out.mov -frames:v 1 frame.png) to confirm the intended picture is captured (this catches frontmost-app contamination or no window created early).
• The file picker dialog and the Finder display of results don't appear in the recording (because per-window recording captures only the target window). If you want to show the post-conversion folder structure or file contents in the video, it's reliable to give the app a built-in result viewer (an output tree + text preview) and switch the selection from the self-drive to show it. If the star of the video is XML (METS, etc.), adding simple syntax highlighting (color-coding element names, attribute names, attribute values, and comments; AttributedString is enough, no external library needed) makes the readability on video vastly better.
• Prepare the input material deterministically with --headless of the same binary. If you give the app a --headless CLI mode, you can reuse the same binary both for the CLI demo's demonstration and for pre-generating the input of another mode's demo (e.g. the SIP passed to the AIP demo). It's both safe and reproducible to generate synthetic data (dummy PostScript/PNG, fictional emails at example.invalid, etc.) each time rather than using real data.

C. Adding narration and subtitles (synchronizing audio and video)

The self-driving demo is silent video. On top of it we add Japanese narration (TTS) and subtitles. The crux is matching the timing of audio and video. Even if you slow the video down with fixed wait times, if you don't know each step's "actual display time" then aligning audio and subtitles afterward becomes manual and drifts.

It breaks down if the order is reversed: script → duration → recording

The correct order is ① write the narration script → ② record at the script's duration → ③ overlay audio and subtitles onto the video. If you record first and then fit the script, the durations won't match. Concretely:

1. Write the script. For each cut (cue), keep separately "the display text to show in the subtitle (display)" and "the reading for the TTS to speak (spoken, with proper nouns in kana)."
2. Synthesize each cue with TTS and measure the audio duration. Use that audio duration + margin as the hold seconds for each step of the self-driving demo (schedule.json). This guarantees that "that cut in the video stays on screen long enough to finish reading the audio."
3. Advance the self-driving demo at the schedule's durations and record each cue's start time. Add a "narration mode" on the app side that writes each cut's start wall-clock epoch (milliseconds) to JSON (--cue-out).
4. Convert to in-video times and composite. startMs = cut-start epoch − recording-start epoch. The recording script saves the epoch just before launching screencapture. Using this startMs, place the TTS audio and subtitles (VTT).

Synchronization tips (implementation notes)

• Align with the wall-clock epoch. The app records each cue's start with Date().timeIntervalSince1970 * 1000, and the recording script saves the same epoch just before screencapture -v &. The difference becomes the in-video time. Both look at the same clock, so there's no drift.
• Absorb screencapture's recording-start lag (a few hundred milliseconds to a few seconds) by trimming the head with contentStartMs (leaving only about 0.5 seconds before the first cue).
• If the audio becomes longer than the video, extend the last frame. With ffmpeg's tpad (freeze the final frame), you can extend the video to the audio duration so the last narration isn't cut off.
• Burned-in subtitles require libass. If your local ffmpeg isn't built with --enable-libass, you can't burn subtitles with the standard subtitles filter. In that case, either use a sidecar VTT (loaded as YouTube subtitles), or render subtitles to PNG and overlay them with overlay.
• Show display, not spoken, in subtitles. The payoff of splitting the script into display (display text) and spoken (reading kana) in ① is right here. Have TTS read spoken ("エスアイピー," "メッツ") while subtitles show display (SIP, METS, Archivematica), so the audio is read correctly while the subtitles are easy to read. When building the sidecar VTT, use each cue's display for the body.
• Speech synthesis is Azure Neural TTS; synthesis, concatenation, and mux are ffmpeg. These don't depend on the recording method, so you can reuse an existing narration pipeline as the back end as-is.

You can add narration to the CLI demo (vhs) the same way. Moreover, since vhs advances on a fixed timeline, no epoch handshake like the GUI is needed. From the script's prerollSec + the cumulative hold seconds of each cut, compute each cue's start time deterministically (treating recording start as 0), and just have the tape side sleep at the same hold seconds to match the pace. Absorb the drift from the keystrokes at the head of vhs with a fixed offset (SKEW_MS). Don't assume "CLI demos are silent" — on the contrary, the CLI is where narration is easiest to add.

In other words, "adding a narration mode to the app that emits each cue's start time" is the key to overlaying audio and subtitles onto native GUI recordings without drift. This reproduces, in the native self-driving demo, the same idea as back when I recorded Web with Playwright and "recorded each scene's start time as real time from recording start."

D. Record parts separately and combine them afterward (maintainability)

Once a tutorial gets long, "shoot it in one take" breaks down. It's impractical to re-record the whole thing every time a part of the app's features changes. So make each part (chapter) self-contained in its own directory. One part = (script script.json + silent recording mov + cut-start times cues.json + audio + that part's final mp4).

narration-<part>/
  script.ja.json        # script (display / spoken / self-drive actions)
  videos/ch01.mov       # silent self-driving demo recording
  cues.json             # each cut's start epoch (generated by the recording)
  final-voice/...mp4    # this part's final video (with audio) + .vtt

Then just concatenate each part's final mp4 in the specified order. If you re-record only one part, re-running the combine script updates the full version, and no other part is touched at all (partial updates are safe).

# resolve part name → directory, and concatenate in order
combine sip aip full intro …      # reorder and cherry-pick via arguments too

When combining videos of different resolutions, unify them with scale+pad

Per-window recording captures at real pixels, so GUI recordings (2560×1600, etc. on Retina) and the vhs terminal (1280×800) tend to differ in both resolution and aspect ratio. Concatenating them directly fails, so scale to a common canvas keeping the aspect ratio and center-pad before concatenating.

# preprocess each input with this filter, then concat
scale=2560:1600:force_original_aspect_ratio=decrease,pad=2560:1600:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30

If you also concatenate the subtitles (VTT) with each part's duration cumulatively offset, the full version's subtitles are produced automatically. Each part's cues use display for the body, and align the times to the final video with "cut start − head trim."

Summary: how to use each

For native macOS app demos, "adding a self-driving mode to the app" turned out easier in the end than "operating it from outside," making both the file-dialog problem and coordinate dependence disappear. If you nail two points — placing the trigger at applicationDidFinishLaunching, and creating the window explicitly and recording per window with screencapture -l <CGWindowID> — you get practical automatic recording even for the GUI (no bringing to front or going fullscreen needed).

To summarize the key points, a native macOS / SwiftUI app's demo video can be (semi-)automated in three stages:

1. The CLI is fully automatic with vhs (unattended, CI-capable)
2. The GUI: add a --demo self-driving mode on the app side and record with screencapture (trigger at applicationDidFinishLaunching, create the window explicitly with AppKit, record per window with screencapture -l <CGWindowID> = no bringing to front or fullscreen needed)
3. Narration and subtitles in the order "script → duration → recording → compositing," emitting each cut's start time (wall-clock epoch) to synchronize audio and subtitles (TTS + ffmpeg)

The first few takes were rough going with GUI foregrounding and .task misfires, but the conclusion is that on native, "giving the app itself a self-driving mode designed on the premise of being recorded" is the most robust.