# OverlayMotion > Remotion overlay templates and an edit-spec system built for AI agents: > one JSON spec in, a branded rendered video out. Generated from docs/*.md. Human version: the #docs page on the site. --- # Quick start From clone to rendered video in under five minutes. One repo, no extra services. (The `overlay-motion` npm package is coming; today the repo is the distribution.) ## 1. Install and explore ```bash git clone && cd overlay-motion npm install npm run dev # the site: template gallery, examples, docs npm run studio # Remotion Studio: every template as a composition ``` The gallery at `http://localhost:5199` shows all templates with live players. Every template detail page displays the exact JSON spec that renders the preview, ready to copy. ## 2. Write a spec An edit is one JSON document. Start from any template's "copy spec" button, or from scratch: ```json { "version": 1, "format": "vertical", "fps": 30, "durationSec": 8, "source": { "type": "none" }, "overlays": [ { "template": "hero-title", "region": "center", "time": { "start": "0.5s", "appear": 1, "hold": 6 }, "props": { "title": "Ship it", "subtitle": "The overlay system for agents" } } ] } ``` For an agent editing real footage, start with the [Editing agent playbook](agent-playbook.md): analyze the source, preserve protected subjects, record assumptions and validate inexpensive checkpoints. The full rendering grammar is [Edit Spec v1](edit-spec.md). ## 3. Preview it Open Remotion Studio (`npm run studio`): every template is a composition, and the `custom` composition previews any spec you paste into its `defaultProps`. ## 4. Render it Every template is a composition in `remotion/index.ts`; the `custom` composition renders whatever spec you pass: ```bash npx remotion render remotion/index.ts hero-title out/hero.mp4 npx remotion render remotion/index.ts custom out/edit.mp4 \ --props='{"spec": , "theme": }' ``` `--props` accepts a file path too: `--props=./my-edit.json`. ## 5. Brand it Pass any `BrandTheme` as `theme`: colors, fonts, radius, and glass/solid surface. Same spec, another theme, rebranded video. Presets live in `src/theme/themes.tsx`; the site's theme picker shows them all. --- # Editing agent playbook This is the canonical behavior for an agent editing with OverlayMotion. It is designed for the common case where the user supplies footage and a weak prompt such as “make this good,” as well as for footage that contains spoken editing instructions. The agent produces two artifacts: 1. an **edit decision plan** recording evidence, assumptions, protected subjects, assets and QA checkpoints; 2. an **Edit Spec v1** describing the render. The plan explains judgment. The spec remains a deterministic rendering input. Machine-readable defaults and the plan schema live in `src/agent/`. Required and conditional tool contracts are in [Agent capability toolkit](agent-toolkit.md). ## Prime directive Improve comprehension without contradicting the speaker, hiding the important subject or fabricating facts. Follow this priority order: 1. explicit user request; 2. observable transcript and visual evidence; 3. subject, message, rights and platform safety; 4. established brand and platform defaults; 5. decorative style. When an explicit request conflicts with subject safety, satisfy both through layout. For example, split the screen and reframe the speaker into the open panel instead of placing secondary media over their face. ## What the agent may decide Make reversible aesthetic decisions without interrupting the job: caption style, conservative motion, exact safe placement, crop focus, phrase grouping, and the shortest faithful copy. Record these as assumptions. Verify or ask before asserting identity, changing meaning, making a factual claim, selecting unlicensed media, publishing a quote whose wording or attribution is uncertain, or making an ambiguous destructive cut. A low-risk draft may keep an unresolved item as a clearly labeled placeholder. ## Source analysis before effects Create one normalized source record before authoring the spec: - probe container, codec, dimensions, rotation, frame rate, duration, color range and audio streams; - transcribe speech with word start/end timestamps and language; - inspect a contact sheet across the full timeline; - inspect frames around each named person, object, gesture, scene change and requested edit beat; - note face, mouth, active-hand, demonstrated-object and platform-UI regions; - preserve the original; use a rotation-correct, editor-friendly proxy only when the renderer needs one. Treat spoken edit commands as commands only when the context supports it. Keep the original words in the transcript either way. ## Default edit for a weak prompt For a spoken social video, the useful default is deliberately modest: - trim only obvious dead head/tail space; do not rewrite the speaker; - add word-timed captions; - apply brand tokens and real supplied brand assets; - use restrained emphasis overlays only where the speech supplies a clear semantic beat; - preserve faces, hands and demonstrated objects; - keep source audio primary and avoid decorative sound under important speech; - deliver a debug preview or checkpoints before paying for a full render. Do not fill silence with effects. Do not turn every sentence into a card. ## Captions Spoken video gets captions by default unless the user opts out. - Prefer word timestamps from Whisper or another transcriber. Never infer exact cadence by evenly dividing a phrase when real timings exist. - Group about 3–8 words into a semantic phrase, normally no more than two lines and about 32 characters per line. Break at clauses, not mechanically. - Preserve pauses as empty time. A caption must disappear during a meaningful silence rather than stretching across it. - Keep casing and punctuation readable. Correct obvious transcription mistakes only when audio makes the intended word clear. - Put captions above platform controls and away from the face, mouth, active hands and any media carrying the current meaning. `caption-classic` accepts phrase `start`/`end` and word `start`/`end`, all relative to its overlay window. Text-only lines remain a compatibility fallback. ## Faces, split screens and secondary media Faces, mouths, eyes, active hands, a demonstrated object and the current gesture target are protected subjects. Add padding; do not merely avoid their exact bounding boxes. When showing an image or secondary video: 1. decide which subject is primary at that beat; 2. choose the secondary media region; 3. reframe the base source into the complementary region with `source.reframes`; 4. inspect the subject at the entrance, midpoint and exit, not only one frame; 5. place captions in the remaining safe region. Prefer a stable crop with modest easing. Use tracking only when a static crop cannot keep the subject safe; unnecessary tracking looks nervous and costs more to verify. ```json { "source": { "type": "video", "src": "speaker.mp4", "position": "center 28%", "reframes": [ { "time": { "start": "9s", "duration": "8s" }, "region": { "x": 4, "y": 8, "w": 44, "h": 84 }, "position": "58% center", "transitionSec": 0.3 } ] }, "overlays": [ { "template": "b-roll", "region": { "x": 52, "y": 8, "w": 44, "h": 84 }, "time": { "start": "9s", "duration": "8s" }, "props": { "src": "secondary.mp4", "frame": "flush", "divider": "left" } } ] } ``` ## Gesture-aware placement Words such as “here,” “there,” “this” and “right here” are spatial only when resolved with nearby frames. 1. Find the transcript time of the deictic word. 2. Inspect roughly 0.35 seconds before through 0.65 seconds after it. 3. Identify the active hand, fingertip and direction of the pointing ray. 4. Place the overlay just beyond the fingertip, usually with about 3% frame offset, while keeping hand, face and intended target visible. 5. Inspect the entire overlay window. Track only if the anchor moves enough to break the meaning. Use a custom percentage region for the resolved location. The region is the result of observation; a named corner is only a fallback. ## Quotes and factual inserts A quote is content, not decoration. - Select wording that directly supports the nearby speech and fits the safe region at a readable size. - Verify it against an authoritative source. Preserve exact wording, author or speaker, work/reference, and translation/edition when applicable. - Do not use an attractive paraphrase as a quotation. If only the idea is known, label it as a paraphrase or use original editorial copy without quote marks. - Prefer a shorter verified excerpt over shrinking a long quote until it is unreadable. - Record the source in the decision plan even when the on-screen design shows only the concise attribution. ## Assets and logos Use assets in this order: user-supplied, brand library, licensed source, then generated asset when generation is appropriate and disclosed. Record source URL, license and required credit. Keep visible credit concise; keep complete provenance in the decision plan. Never reconstruct or hallucinate a real company logo when the asset is unavailable. `logo-sting` accepts a real transparent `logo`; its monogram is an intentional fallback, not an imitation. ## Efficient QA Quality comes from targeted checks, not repeated full renders. 1. Validate the decision plan and Edit Spec. 2. Preview the timeline or render low-cost stills at: just before the edit, entrance, midpoint, exit and just after. 3. Add checkpoints for every spatial decision (face/split/gesture) and semantic decision (quote, number, name, attribution). 4. Check caption onset, a middle word, phrase disappearance and a pause. 5. Check source audio once, especially around wrappers and duplicated video layers. 6. Watch one complete preview at normal speed. Full-render once after these checks pass. For a stable static placement, three frames may be enough. For a moving hand, crop, face or tracked anchor, inspect more frames. Spend QA where uncertainty is. ## Completion report Report what was inferred, what was verified, asset provenance/credit, what the current template system could not express, and which QA checks ran. Never claim “perfect sync,” “face safe” or “verified quote” without the corresponding evidence. --- # Agent capability toolkit OverlayMotion specifies capabilities, not vendors. An agent may use local CLI tools, hosted services or a human-assisted step, provided the returned evidence meets these contracts. The canonical machine-readable list is `AGENT_CAPABILITIES` in `src/agent/policy.ts`. ## Required capabilities ### Media probe Input: source media. Output: duration, display dimensions, rotation, frame rate, codec, color/HDR information and audio streams. `ffprobe` is one suitable implementation. Never assume file extension describes the decoded video. ### Word transcription Input: speech audio. Output: language, segments, each word's start/end time and confidence where available. Whisper implementations are suitable. Segment-only transcription is a degraded fallback; evenly divided words are not “exact sync.” ### Frame sampling Input: media and timestamps. Output: a timeline contact sheet plus full-resolution frames at edit checkpoints. Sample the whole clip once, then densely around gestures, overlays, cuts, crops and semantic inserts. ### Visual inspection Input: frames plus transcript intent. Output: padded protected regions, crop focus, gesture anchor and confidence. Multimodal inspection is enough for stable placements. It must examine time, not one convenient still. ### Validation and preview QA Validate both the Edit Decision Plan and Edit Spec. Inspect low-cost checkpoints before full render; verify source audio, captions, spatial safety and semantic content. The MCP server exposes the guide, prompt, templates and validators. ## Conditional capabilities ### Face/hand landmarks or tracking Use a detector such as MediaPipe only when the face, hand or fingertip moves enough that sampled inspection cannot keep an anchor safe. Return time-series landmarks and confidence. Smooth noisy coordinates, keep padding, and fall back to a static safe region when confidence drops. Installing a large model for a stable three-second gesture is wasteful and creates extra failure modes. ### Asset acquisition Use when the requested image/video is absent. Search sources that expose usage rights, download an editor-friendly file, and return source URL, license and required credit. Reject a visually good asset when its rights cannot be established. Cache it locally so preview and render do not depend on a remote URL. ### Citation verification Use when displaying a quote, statistic, identity, date or other factual claim. Return verified copy, an authoritative source and any edition/translation that changes wording. Search snippets are discovery, not verification. ### Media normalization Use only when the source cannot be decoded predictably. Bake rotation and make an editor-friendly proxy while preserving the original reference. Do not silently alter speed, color or audio sample timing. ## Escalation ladder 1. metadata + transcript + sparse contact sheet; 2. targeted full-resolution frames; 3. denser sampling around uncertain motion; 4. landmarks/tracking when spatial meaning still cannot be preserved; 5. ask the user only when evidence cannot resolve a consequential choice. This ladder keeps tokens, model downloads and renders proportional to actual uncertainty. --- # Edit Spec v1 An edit spec is a JSON document that describes a finished edit: one base source, a stack of overlay templates, cameras, and sound. It is the only input an agent needs to produce a video. Zod schemas live in `src/spec/types.ts`; full validation (shape + template cross-checks) is `validateSpec` / `parseSpec` from `src/spec/validate.ts`. ```json { "version": 1, "format": "landscape", "fps": 60, "durationSec": 20, "source": { "type": "video", "src": "your-footage.mp4" }, "overlays": [ { "template": "speaker-card", "region": "lower-third", "time": { "start": "2s", "duration": "9s", "appear": 0.7 }, "props": { "name": "Ana", "role": "Founder", "photo": "ana.png" } } ] } ``` ## The two axes: region and time Natural language like "on the bottom, at 2/3 of the video, for 3s" compiles to `region: "lower-third"`, `time: { start: "66%", duration: "3s" }`. - **`region`** (space): a named region (`fullscreen`, `top-banner`, `lower-third`, `upper-third`, `caption-zone`, `left-panel`, `right-panel`, `center`, `corner-tl/tr/bl/br`) or a custom rect in percentages (`{ "x": 31, "y": 70, "w": 38, "h": 18 }`). Unset uses the template's preferred region. - **`time`** (timeline): values are seconds (`3` or `"3s"`), percentages of the owner timeline (`"66%"`), or negative seconds from the end (`"-2s"`). A window is `{ start, duration, appear, hold }`: `appear` is how long the entrance choreography takes, `hold` is how long the finished overlay stays after that. `duration` absent + `hold` present makes the window `appear + hold`. ## Source and the source contract `source` is the base layer: `{ type: "video", src }` (options: `muted`, `fit`, `position`, `flipHorizontal`, footage-only `camera`, time-windowed `reframes`), `{ type: "audio", src }`, or `{ type: "none" }`. ### Subject-safe source reframes `source.reframes` moves the base footage into a percentage rectangle during a time window while overlays stay locked to the composition. This is editorial layout, not camera motion. It is intended for split screens where simply covering half of the fullscreen source would hide a face or demonstrated object. Reframe windows may not overlap. ```json { "type": "video", "src": "speaker.mp4", "fit": "cover", "position": "center 30%", "reframes": [ { "time": { "start": "9s", "duration": "8s" }, "region": { "x": 4, "y": 8, "w": 44, "h": 84 }, "position": "60% center", "transitionSec": 0.3 } ] } ``` `fit` defaults to `cover`, `position` to `center`, and `transitionSec` to 0.3. Resolve the position by inspecting the subject across the full window. Every template declares one relationship with that base source, and validation enforces it: | Contract | Meaning | Requires | | --- | --- | --- | | `overlay` | Self-contained; draws over whatever the source is. Most templates. | nothing | | `annotates-video` | Only makes sense over footage (captions, ticker, recording chrome). | `source.type: "video"` | | `wraps-video` | Renders the base video inside its own layout (video-card). One per spec. | `source.type: "video"` | | `visualizes-audio` | Driven by a standalone audio file (audiogram); receives it as `sourceSrc`. | `source.type: "audio"` | A template that ships its own footage via props is still `overlay`: the contract describes the BASE source only. ## Camera motion vs object motion A **camera** moves the frame around finished content; **object motion** is the content moving, and it belongs to the template. A camera never choreographs entrances or exits. Cameras live at three scopes, location = scope: - `spec.camera`: the scene; source and every overlay move as one shot. - `source.camera`: footage only; overlays stay locked to their regions. - `overlay.camera`: one overlay's region viewport, on the overlay's own timeline. Presets: `push-in`, `push-in-out`, `push-in-fast-out`, `pull-out`, `pan-left/right/up/down`, `handheld` (fields `frequency`, `seed`), with `amount`, `focus {x,y}`, `time`, `easing`. Scene scope accepts an array of windows. Full grammar: `docs/camera-motion-spec.md`. ## The motion language Small on purpose, so mixed templates read as one design system. All three knobs are optional; unset keeps the template's native motion. - **`reveal`** (text entrance hint): `"fade-up"`, `"blur-in"`, `"typewriter"`. Text templates map it to their nearest native mode and may expose richer extensions via props (quote-card `animateIn` adds `"lines"`/`"words"`; tweet-card adds `"paragraphs"`/`"none"`). - **`enter`** (card entrance, renderer-provided, identical on every template): `"slide-left"`, `"slide-right"`, `"spring"`, `"mask"`. - **`exit`**: `"blur-out"` (blur + lift, the signature departure), `"fade-down"`, `"shrink"`. Templates with their own `exit` prop (speaker-card) treat the spec value as default, not override. ## Sound `sound` at the spec root sets defaults; each overlay may override with its own `sound` block. Cues resolve to built-in names, arbitrary audio paths, or `false` to silence one cue. The curated core palette (reach for these first): `click`, `pop`, `whoosh`, `ding`, `typewriter` (`CORE_SFX` in `src/sound/config.ts`). ## Brand theme Templates never hardcode style; they read tokens from the active `BrandTheme`: `colors` (primary, onPrimary, surface, onSurface, muted, background, optional secondary/accent), `fonts` (heading, body), `radius`, `logoText`, and optional `style` (surface `"solid" | "glass"`, `blur` up to 100, `opacity`, gradients, `borderColor`). Same spec + another theme = rebranded video. ## Formats `vertical` 1080×1920, `horizontal` 1920×1080, `landscape` 1620×1080, `square` 1080×1080. Templates scale through the shared `rem()` helper, so one spec renders correctly in all four. ## Validation `parseSpec(raw)` (throws) or `validateSpec(raw)` (safe result) from `src/spec/validate.ts`. Beyond shape, they enforce: known template slugs, source-contract requirements, at most one `wraps-video` overlay, and non-overlapping source-reframe windows. Overlay and source-reframe windows may not extend past the composition duration. `editSpec.parse` alone checks shape only; prefer the checked variants. --- # Camera motion in Edit Spec v1 Status: partially implemented. The preset subset (`preset`, `amount`, `time`, `easing`, optional `focus`) is live at three scopes: `overlay.camera` (src/player/OverlayCamera.tsx), `source.camera` for footage-only motion, and `spec.camera`, the scene scope where source and overlays move as one shot (src/player/SceneCamera.tsx). Beyond the directional presets there is a `handheld` preset (deterministic wobble; extra fields `frequency` in cycles per second and `seed` for phase offset, easing ignored). `push-in-out` provides a balanced focus move that restores framing before its window ends; `push-in-fast-out` spends most of its window approaching, then returns quickly. Keyframes, `rotationDeg`, and `crop` remain proposed. The examples page (`#examples`) layers separate focused presenter and quote push-in-out tracks, with handheld motion nested on the quote itself. ## Camera motion vs object motion The one distinction every author and agent must hold: a **camera** moves the frame around finished content; **object motion** is the content moving. They never share knobs. | | Camera motion | Object motion | | --- | --- | --- | | What moves | The viewport (scene, footage, or one overlay's region) | The template's own elements (card, text, items) | | Who owns it | The spec's `camera` blocks | The template | | Steered by | `spec.camera`, `source.camera`, `overlay.camera` | `time.appear`, `reveal`, `exit`, template props | | Examples | Push in on a face, handheld sway, settle after entrance | Card springs up, text types on, list items stagger | Rule of thumb: if the move belongs to a thing ("the card slides in"), it is object motion and lives in the template. If the move belongs to the viewer's eye ("we lean toward the speaker"), it is a camera. A camera never choreographs entrances or exits, and templates never implement camera motion internally. ## Three-scope authoring rule Templates never implement camera motion internally. Quote Card, Tweet Card, Bar Chart, Line Chart, and Stat Counter therefore use the same preset, amount, easing, timing, and normalized final scale behavior. - Put `camera` on an overlay to move only that card. - Put `camera` at the spec root to move the rendered footage and all cards as one complete composition. Both use `camera.time.start` for the requested start second and `camera.time.duration` for motion length. Root time is composition-relative; overlay time is relative to its own overlay window. ## Goal Make push-ins, pull-outs, pans, reframing, and small rotations available to every OverlayMotion composition without putting camera code inside templates. Camera motion is additive to Edit Spec v1. Existing specs remain valid. A camera block may live at three locations, and its location defines its scope: - `spec.camera`: source and all overlays move as one composed scene. - `source.camera`: footage moves; overlays stay fixed. - `overlay.camera`: one resolved overlay region moves; other layers stay fixed. Location-based scope avoids selectors and overlay IDs. Multiple scopes may be combined; their transforms nest in the order described below. ## Proposed schema ```ts type CameraEasing = | "linear" | "ease-in" | "ease-out" | "ease-in-out" | { bezier: [number, number, number, number] }; type CameraKeyframe = { at: TimeValue; focus?: { x: number; y: number }; // percent, defaults to {x: 50, y: 50} zoom?: number; // defaults to 1 rotationDeg?: number; // defaults to 0 easing?: CameraEasing; // interpolation leaving this keyframe }; type CameraBase = { time?: TimeWindow; easing?: CameraEasing; crop?: "cover" | "reveal"; }; type KeyframedCamera = CameraBase & { keyframes: CameraKeyframe[]; preset?: never; }; type PresetCamera = CameraBase & { preset: "push-in" | "push-in-out" | "push-in-fast-out" | "pull-out" | "pan-left" | "pan-right" | "pan-up" | "pan-down"; amount?: number; focus?: { x: number; y: number }; keyframes?: never; }; type Camera = KeyframedCamera | PresetCamera; ``` Add these optional fields to existing Zod objects: ```ts editSpec.camera?: Camera | Camera[]; editSpec.motion?: "full" | "reduced"; videoSource.camera?: Camera | Camera[]; overlay.camera?: Camera; ``` `camera` is invalid on `audio` and `none` sources. Initial runtime should allow one camera track per scope. A future spec version may add named or overlapping tracks if real use cases require them. ## Example ```json { "version": 1, "format": "vertical", "fps": 30, "durationSec": 12, "motion": "full", "camera": { "preset": "push-in", "amount": 0.06, "time": { "start": "0s", "duration": "12s" }, "easing": "ease-in-out" }, "source": { "type": "video", "src": "intro.mp4", "camera": { "time": { "start": "2s", "duration": "4s" }, "crop": "cover", "keyframes": [ { "at": "0%", "focus": { "x": 50, "y": 50 }, "zoom": 1 }, { "at": "100%", "focus": { "x": 72, "y": 42 }, "zoom": 1.25 } ] } }, "overlays": [ { "template": "stat-counter", "region": "corner-tl", "time": { "start": "3s", "duration": "4s" }, "camera": { "crop": "reveal", "keyframes": [ { "at": "0%", "zoom": 1.08, "rotationDeg": -1 }, { "at": "100%", "zoom": 1, "rotationDeg": 0 } ] }, "props": { "value": 150, "suffix": "K", "label": "subscribers" } } ] } ``` ## Time semantics - Root and source camera time uses the composition timeline. - Overlay camera time uses that overlay's `Sequence` timeline. `0s` means the first frame of the overlay, not the first frame of the composition. - `camera.time` selects a window inside the owner timeline. Without it, the camera window is the full owner timeline. - Outside that window, the camera transform is identity. A completed camera window never leaks its final zoom or pan into the following edit. - Keyframe `at` values are local to the camera window. Percentages use camera window duration. Negative seconds count from the end of that window. - Values before the first keyframe and after the last keyframe hold the nearest endpoint. Interpolation is clamped. - A keyframe's `easing` controls the segment from that keyframe to the next. The camera-level easing is the fallback. All resolution uses frames, `fps`, and pure interpolation. No CSS animation, wall clock, randomness, or runtime measurements may affect rendered frames. ## Coordinate and transform semantics `focus` is a point in the camera owner's untransformed local box: - `{x: 0, y: 0}` is top-left. - `{x: 100, y: 100}` is bottom-right. - Default is `{x: 50, y: 50}`. The focus point lands at viewport center. For viewport width `W`, height `H`, normalized focus `fx`, `fy`, zoom `z`, and zero rotation: ```text tx = W / 2 - z * fx * W ty = H / 2 - z * fy * H ``` General transform order is: ```text translate(viewport center) rotate(rotationDeg) scale(zoom) translate(-focus point) ``` Implement this as a matrix or as ordered transforms on a dedicated wrapper. Do not write the transform onto a template root: templates own their internal entrance, exit, and item transforms. Scope viewport: - Root: composition dimensions. - Source: actual footage slot, including a slot inside `wrapsVideo` templates. - Overlay: resolved region box. Render nesting: ```text composition clip root camera source camera source overlay region placement overlay camera template internal motion ``` When a template wraps video, its source camera stays immediately around the source passed to that template. Root and overlay nesting remains unchanged. ## Crop and edge policy `crop: "cover"` clips to the scope viewport and prevents empty edges. Runtime must clamp focus and, when required, raise effective minimum zoom. Rotation must use the rotated content bounds when computing the minimum cover scale. `crop: "reveal"` still clips to the viewport but allows background or transparent gaps inside it. This is useful when an overlay intentionally flies or drifts out of its region. Defaults: - Root and video source: `cover`. - Overlay: `reveal`. Validation should warn when `cover` materially changes requested focus or zoom. The exact adjusted values must remain deterministic at every frame. ## Presets Presets are authoring sugar, never separate rendering behavior. Compiler expands each preset into validated keyframes before evaluation. - `push-in`: zoom from `1` to `1 + amount`. - `pull-out`: zoom from `1 + amount` to `1`. - `pan-left/right/up/down`: move focus by `amount * 100` percentage points. Default `amount` is `0.1`. Preset direction describes camera attention: a `pan-right` raises focus `x`, so pixels move left. Preset focus defaults to center. ## Reduced motion `motion: "reduced"` freezes each camera at its first resolved keyframe. Server renders require the explicit spec value. Site player may offer a UI default from `prefers-reduced-motion`, but must compile it into this explicit input so player and CLI output remain reproducible. This first camera milestone does not silently alter templates' existing internal motion. A later contract can make templates consume the same preference. ## Validation - At least two keyframes for a keyframed camera. - Resolved keyframe times must be strictly increasing and inside camera window. - Focus values must be finite and between `0` and `100`. - Zoom must be finite and greater than `0`; recommended public range is `0.25` to `8`. - Rotation must be finite; recommended public range is `-45` to `45` degrees. - Custom Bezier values must be finite; x control points must be between `0` and `1`. - Reject a camera object containing both `preset` and `keyframes`. - Reject source camera on `audio` or `none`. Hard limits protect the public API. Warnings cover visually risky but valid requests such as strong rotation, extreme zoom, or heavy cover clamping. ## Implementation plan 1. **Schema and compiler** - Add camera Zod schemas in `src/spec/types.ts`. - Add pure preset expansion, time resolution, easing, and matrix evaluation in `src/spec/camera.ts`. 2. **Renderer** - Add a transform-only `CameraViewport` in `src/player/CameraViewport.tsx`. - Integrate it at root, source, and overlay boundaries in `SpecRenderer.tsx`. - Keep `OffthreadVideo` and template code unchanged. 3. **Authoring** - Show camera blocks in the site's copyable spec JSON. - Add preset examples and validation messages. 4. **Hardening** - Add rotation-aware cover math and visual fixtures for all formats. ## Acceptance tests - Existing Edit Spec v1 examples parse and render unchanged. - Keyframe evaluation matches exact expected matrices at start, middle, and end. - Equivalent 30 fps and 60 fps renders reach the same time-based values. - Root, source, and overlay scopes affect only their documented layers. - Nested transforms work when a template has `wrapsVideo`. - Cover never exposes an edge across vertical, horizontal, and square formats. - Reveal permits gaps while still clipping outside its viewport. - Template-owned transforms keep working inside an overlay camera. - Reduced motion freezes camera identically in Player, Studio, and CLI render. - Player and server render produce matching frames for the same spec. ## Open decisions - Ship rotation in the first runtime milestone or validate it as experimental. - Expose `reveal` publicly at launch or reserve it for overlay motion. - Add multiple named camera tracks only after a concrete overlap use case exists. --- # AI instructions You are an agent producing a video with OverlayMotion. Editorial behavior and weak-prompt defaults are normative in [Editing agent playbook](agent-playbook.md). This page is the concise compiler model. ## What you produce One JSON edit spec. You never write React, never touch template internals. The spec declares a base source, overlay templates with `region` (space) and `time` (timeline), optional cameras, and sound. `parseSpec` in `src/spec/validate.ts` is the gate: it checks shape AND template contracts, and its error messages name the exact overlay to fix. ## Rules that keep output correct 1. **Pick templates by source contract.** `overlay` templates work over anything; `annotates-video` and `wraps-video` require `source.type: "video"`; `visualizes-audio` requires `source.type: "audio"`. Only one `wraps-video` overlay per spec. The contract is in each template's registry entry. 2. **Camera moves the frame, templates move the content.** Want the eye to travel? Use `spec.camera` (scene), `source.camera` (footage) or `overlay.camera` (one card). Want a card to slide/spring/type? Use `enter`, `reveal`, `exit` and template props. Never both for the same idea. 3. **Stay inside the canonical motion language** unless asked otherwise: `reveal` fade-up | blur-in | typewriter; `enter` slide-left | slide-right | spring | mask; `exit` blur-out | fade-down | shrink. Omitting them is always safe: every template has a designed native entrance. 4. **Time grammar:** `"3s"` seconds, `"66%"` of the owner timeline, `"-2s"` from the end. `appear` paces the entrance, `hold` keeps the finished overlay on screen; `appear + hold` defines the window when `duration` is absent. 5. **Sound:** cues default on; silence one with `sounds: { swoop: false }`. Prefer the core palette: click, pop, whoosh, ding, typewriter. 6. **Brand comes from the theme, never from props.** No hex colors in props; the `BrandTheme` carries colors, fonts, radius, glass/solid. ## Workflow 1. Analyze the media and write an Edit Decision Plan from the playbook. 2. Read the template list (`src/templates/registry.ts`, the gallery, or MCP) and pick by contract + purpose. 3. Draft the spec. Sequence overlays with ~1s gaps between windows so handoffs read cleanly. 4. Validate with `parseSpec`. Fix what the error names. 5. Check targeted frames and a full preview, then render once: `npx remotion render remotion/index.ts custom out.mp4 --props=./edit.json` (props JSON is `{"spec": ..., "theme": ...}`). Editorial reference: [Editing agent playbook](agent-playbook.md). Technical reference: [Edit Spec v1](edit-spec.md). Camera grammar: [Camera motion](camera-motion-spec.md).