Add TikTok video skills (caption-animation, countdown-video, lower-thirds, short-form-video)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011XhxckAYjGMLrLECWuKBzg

.agents/skills/caption-animation/README.md

@@ -0,0 +1,24 @@
+# Caption Animation
+
+Turn a transcript or voiceover into word-timed, scroll-stopping captions: each word pops in on the syllable, the active word highlights, and the text stays glued to the narration. Build the kind of captions that drive watch-time on TikTok, Reels, and Shorts — readable, on-beat, and inside the safe area. For short-form creators burning open captions onto vertical video.
+
+## When it activates
+
+- "Add animated captions," "make TikTok captions," or "karaoke captions."
+- "Word-by-word subtitles," "auto subtitles," or "highlight the active word."
+- "Burn in captions to a video" or "sync captions to a voiceover/narration."
+- "Turn an SRT/transcript into animated text."
+
+## Example prompts
+
+- "Turn this Whisper transcript into karaoke captions with a per-word pop."
+- "Restyle these auto subtitles into a branded, Hormozi-style reveal."
+- "Burn word-level captions onto my voiceover and export a sidecar SRT."
+
+## What's inside
+
+- `SKILL.md` — the transcribe → normalize → page → animate → place → export pipeline.
+- `references/word-timed-captions.md` — deeper guidance on word-level timing and animation.
+
+---
+Part of **[Short-Form Video Skills](../../)** · Built by **[iart.ai](https://iart.ai/?utm_source=github&utm_medium=readme&utm_campaign=tiktok-video-skills&utm_content=skill_footer&utm_term=caption-animation)** — the AI motion agent for editable, on-brand motion graphics.

.agents/skills/caption-animation/SKILL.md

@@ -0,0 +1,177 @@
+---
+name: caption-animation
+description: This skill should be used when the user asks to "add animated captions", "make TikTok captions", "karaoke captions", "word-by-word subtitles", "auto subtitles", "highlight the active word", "burn in captions to a video", "sync captions to a voiceover/narration", or "turn an SRT/transcript into animated text". Covers word-level timing from Whisper, per-word pop/scale-in, active-word highlight, 9:16 safe-area placement, readable type, and burn-in vs sidecar SRT/VTT.
+version: 0.1.0
+---
+
+# Caption Animation
+
+Turn a transcript or voiceover into word-timed, scroll-stopping captions: each word pops in on the syllable, the active word highlights, and the text stays glued to the narration. Build the kind of captions that drive watch-time on TikTok, Reels, and Shorts — readable, on-beat, and inside the safe area.
+
+## When to use
+
+- Word-by-word or karaoke captions on short-form vertical video (9:16).
+- Burning open captions onto a voiceover/narration track from a transcript.
+- Turning a Whisper/SRT transcript into animated text in Remotion or on the web.
+- Restyling auto-generated subtitles into a branded, "Hormozi-style" reveal.
+
+## The pipeline
+
+| Stage | Job | Tool |
+|---|---|---|
+| Transcribe | Audio → word-level timestamps | Whisper (`@remotion/install-whisper-cpp`), AssemblyAI |
+| Normalize | → `{ text, startMs, endMs }[]` tokens | `@remotion/captions` `Caption` type |
+| Page | Group words into 1–4 word "pages" | `createTikTokStyleCaptions()` |
+| Animate | Per-word pop + active-word highlight | Remotion `spring()` / CSS |
+| Place | Safe-area, readable type | layout rules below |
+| Export | Burn-in (MP4) and/or sidecar SRT/VTT | Remotion render / file emit |
+
+## Two non-negotiable rules
+
+1. **Word-level timing, not line-level.** Karaoke reads as magic only when each word lands on the syllable. Always transcribe to word timestamps; never fake them by splitting a line evenly over its duration — drift is instantly visible.
+2. **Readability beats style.** Captions are read on a phone, in sunlight, muted. Bold sans-serif, heavy stroke or shadow, high contrast first; decoration second. A caption nobody can read is decoration, not a caption.
+
+## Word timing from a transcript
+
+Whisper emits word-level timestamps directly. Normalize every source into one flat token shape so the renderer never cares where the words came from:
+
+```ts
+// Caption token — the one shape everything downstream consumes
+type Token = { text: string; startMs: number; endMs: number };
+
+// Parse an SRT cue block "00:00:01,200 --> 00:00:01,640" into ms
+const toMs = (t: string) => {
+  const [h, m, rest] = t.split(":");
+  const [s, ms] = rest.split(",");
+  return ((+h * 60 + +m) * 60 + +s) * 1000 + +ms;
+};
+```
+
+If only sentence-level cues exist, re-transcribe for word timing — do not interpolate. Whisper's `medium.en` is ~2x faster than `large` and accurate enough for clean voice audio.
+
+## Per-word pop-in (Remotion)
+
+Drive each word's scale and opacity off a spring anchored to its own `startMs`. The micro-overshoot is what makes it feel "alive".
+
+```tsx
+import { useCurrentFrame, useVideoConfig, spring, interpolate } from "remotion";
+
+const Word: React.FC<{ token: Token; active: boolean }> = ({ token, active }) => {
+  const frame = useCurrentFrame();
+  const { fps } = useVideoConfig();
+  const enter = (token.startMs / 1000) * fps;            // word's own entrance frame
+  const p = spring({ frame: frame - enter, fps, config: { damping: 12, mass: 0.6 } });
+  const scale = interpolate(p, [0, 1], [0.6, 1]);        // pop from 60% → 100% (overshoots)
+  return (
+    <span style={{
+      display: "inline-block",
+      transform: `scale(${scale})`,
+      opacity: interpolate(p, [0, 1], [0, 1]),
+      color: active ? "#FFE45E" : "#FFFFFF",             // active-word highlight
+      transition: "color 80ms linear",
+    }}>{token.text}&nbsp;</span>
+  );
+};
+```
+
+Compute `active` by testing `frame` against each token's `[startMs, endMs]`. Keep the highlight a single accent color or a filled "box" behind the live word — never animate every word's color at once.
+
+## Pages, not walls of text
+
+Show 1–4 words at a time. Use Remotion's `createTikTokStyleCaptions()` (or group manually) and tune `combineTokensWithinMilliseconds`: ~200–500ms for true word-by-word, ~1000–1500ms for short readable phrases.
+
+```ts
+import { createTikTokStyleCaptions } from "@remotion/captions";
+const { pages } = createTikTokStyleCaptions({
+  captions, combineTokensWithinMilliseconds: 1200,       // ~2–3 words per page
+});
+```
+
+## Web/CSS pop (no Remotion)
+
+For a DOM/live player, give each word its own `animation-delay` equal to `startMs`:
+
+```css
+.word { display:inline-block; opacity:0; animation: pop .26s cubic-bezier(.34,1.56,.64,1) forwards; }
+@keyframes pop { from { opacity:0; transform:translateY(.18em) scale(.7) } to { opacity:1; transform:none } }
+.word.active { color:#FFE45E; }
+```
+```js
+words.forEach(w => { const el = mk(w.text); el.style.animationDelay = `${w.startMs}ms`; track.append(el); });
+```
+
+## Readable type and safe-area placement (9:16, 1080×1920)
+
+| Property | Value | Why |
+|---|---|---|
+| Font | Bold/ExtraBold sans (Montserrat, Inter, Helvetica) | Reads on small, busy screens |
+| Size | 56–80px (≈8% of frame height), min 45px | Legible muted on a phone |
+| Stroke | 2–6px solid black outline | Survives any background |
+| Shadow | Soft drop shadow as backup to stroke | Separation on bright frames |
+| Case | Uppercase or sentence; high contrast fill | Punch + scannability |
+| Vertical pos | Center band, ~62–70% down | Above the UI, below the action |
+| Bottom safe | Keep clear of bottom ~280px / 15% | Avoids caption/CTA/audio UI |
+| Side safe | Keep within center 80% width | Avoids right-rail icons |
+
+Place captions in the center band — not pinned to the very bottom, where the platform UI (username, audio, buttons) lives. Pad text to a max of ~2 lines.
+
+## Sync to a voiceover / narration
+
+The transcript already carries the timing of the exact audio. Lay that same audio under the composition and keep timestamps in the audio's timebase — captions and voice stay locked with zero manual nudging. If the VO is re-recorded, re-transcribe; never hand-shift offsets.
+
+## Burn-in vs sidecar SRT/VTT
+
+| | Burn-in (open) | Sidecar SRT/VTT (closed) |
+|---|---|---|
+| Where | Pixels in the MP4 | Separate `.srt`/`.vtt` file |
+| Social (TikTok/Reels) | Required — guaranteed, styleable | Often ignored by the platform |
+| Accessibility (ADA/WCAG) | Does not satisfy alone | Required (toggleable) |
+| Best practice | Burn animated captions for social | Also ship a sidecar for the web/SEO |
+
+Emit both: burn the animated captions for social, and write a plain SRT/VTT from the same tokens for accessible/SEO playback.
+
+## Output checklist
+
+- Word-level timestamps (real, from transcription) — no even-split fakery.
+- Per-word pop with micro-overshoot; one accent color for the active word.
+- 1–4 words per page; never a wall of text.
+- Bold sans, 56–80px, 2–6px stroke + shadow, high contrast.
+- Center band, clear of bottom ~280px and side rails.
+- Audio laid under composition; timestamps in the audio timebase.
+- Ship burn-in for social and a sidecar SRT/VTT for accessibility.
+
+## Deliver & verify (rendered stills → MP4)
+
+> **Packaged helper** (`scripts/`): tile your stills with `scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png`, then assert the encode with `scripts/probe-mp4.sh out.mp4 [WxH] [fps]`. See `scripts/README.md`.
+
+Captions ship as a Remotion composition (`<Composition>` + zod `schema` + `defaultProps`) — all word motion frame-driven off `useCurrentFrame()`, never `Date.now()` / `Math.random()` / timers. Deliverable = `out/*.mp4` (burned-in) + the project + the sidecar SRT/VTT. 9:16 vertical (1080×1920) is the default.
+
+**Verify loop — render stills → inspect → encode.** Word timing is the thing that breaks; check it at exact frames before you spend an encode.
+
+```bash
+# Stills at start / a sampled active-word frame / end — WITH SHIPPED PROPS (real tokens + audio)
+npx remotion still Captions out/f-start.png --frame=0   --props='{"captionsSrc":"vo.json"}'
+npx remotion still Captions out/f-mid.png   --frame=90  --props='{"captionsSrc":"vo.json"}'
+npx remotion still Captions out/f-end.png   --frame=N   --props='{"captionsSrc":"vo.json"}'  # N = durationInFrames-1
+
+# Inspect each PNG:
+#  - the word highlighted at frame 90 is the word whose [startMs,endMs] contains 90/fps (no drift)
+#  - burn-in legible: bold sans, stroke+shadow holds, no clipping
+#  - 9:16 safe area: caption sits in the center band, clear of top ~12% and bottom ~20-35% (captions/CTA/audio UI) and the right action rail
+
+npx remotion render Captions out/captions.mp4 --props='{"captionsSrc":"vo.json"}'   # encode once stills are right
+npx remotion render Captions out/demo.gif --codec=gif                                # README proof clip
+```
+
+Use `npx remotion compositions` to read `durationInFrames`/`fps` and pick the active-word + end frames.
+
+**Before you finish:**
+1. Stills render cleanly at frame 0, a mid active-word frame, and last — no missing font/audio.
+2. The correct word is highlighted at the sampled frame (frame/fps lands inside its token); no even-split fakery.
+3. Burn-in is legible (stroke+shadow) and the caption is fully inside the 9:16 safe area at every checked frame.
+4. Frame-driven only — no `Date.now()` / `Math.random()` / timers.
+5. Shipped props (real tokens, not just `defaultProps`) render correctly; MP4 + sidecar SRT/VTT emitted, GIF optional.
+
+## Reference files
+
+- `references/word-timed-captions.md` — end-to-end build: Whisper transcription and the `Caption` type, a full SRT parser, Remotion word-timed component with active-highlight, manual paging, an SRT/VTT emitter, per-platform safe-area maps, and a readable-type spec sheet.

.agents/skills/caption-animation/references/word-timed-captions.md

@@ -0,0 +1,254 @@
+# Word-Timed Captions — End-to-End Build
+
+A complete, runnable path from audio/transcript to animated burn-in captions plus a sidecar SRT/VTT. Code targets Remotion for burn-in and plain JS for the web; the data shapes are framework-agnostic.
+
+## 1. The one data shape
+
+Everything normalizes to a flat array of word tokens. Get to this shape and the renderer never cares about the source.
+
+```ts
+export type Token = { text: string; startMs: number; endMs: number };
+export type Page  = { startMs: number; endMs: number; tokens: Token[] };
+```
+
+Remotion's `@remotion/captions` uses a compatible `Caption` type:
+
+```ts
+type Caption = { text: string; startMs: number; endMs: number; timestampMs: number | null; confidence: number | null };
+```
+
+## 2. Transcribe to word-level timing (Whisper)
+
+Word timestamps are non-negotiable for karaoke. Whisper produces them natively.
+
+```bash
+npx @remotion/install-whisper-cpp   # one-time: installs whisper.cpp + a model
+```
+
+```ts
+import { transcribe, toCaptions } from "@remotion/install-whisper-cpp";
+
+const { transcription } = await transcribe({
+  inputPath: "voiceover.wav",      // 16kHz mono WAV is ideal for whisper.cpp
+  model: "medium.en",              // medium.en ≈ 2x faster than large, accurate on clean voice
+  tokenLevelTimestamps: true,      // <-- the key flag: per-word, not per-line
+  whisperPath: "./whisper.cpp",
+});
+const { captions } = toCaptions({ transcription }); // → Caption[]
+```
+
+Tips:
+- Feed clean voice audio (denoise/normalize first). Background music wrecks word boundaries.
+- `large` is most accurate but 2–3x real-time on a laptop. Use `medium.en` unless accuracy is critical.
+- Cloud alternatives (AssemblyAI, Deepgram, OpenAI Whisper API) also return word timing — map their `words[]` into `Token` the same way.
+
+Map any token shape into `Token`:
+
+```ts
+const toTokens = (caps: Caption[]): Token[] =>
+  caps.map(c => ({ text: c.text.trim(), startMs: c.startMs, endMs: c.endMs }));
+```
+
+## 3. Parse an existing SRT into tokens
+
+When handed an SRT, parse cues; if cues are sentence-level, re-transcribe for word timing rather than interpolating.
+
+```ts
+const toMs = (t: string) => {
+  const [h, m, rest] = t.trim().split(":");
+  const [s, ms] = rest.split(",");
+  return ((+h * 60 + +m) * 60 + +s) * 1000 + +ms;
+};
+
+export function parseSrt(src: string): Token[] {
+  return src.trim().split(/\n\s*\n/).map(block => {
+    const lines = block.split("\n");
+    const time = lines.find(l => l.includes("-->"))!;
+    const [a, b] = time.split("-->");
+    const text = lines.slice(lines.indexOf(time) + 1).join(" ").trim();
+    return { text, startMs: toMs(a), endMs: toMs(b) };
+  });
+}
+```
+
+## 4. Page the tokens (1–4 words at a time)
+
+Either use Remotion's helper or group manually. Manual grouping by gap + max-words keeps full control:
+
+```ts
+export function paginate(tokens: Token[], maxWords = 3, maxGapMs = 1200): Page[] {
+  const pages: Page[] = [];
+  let cur: Token[] = [];
+  for (const t of tokens) {
+    const span = cur.length ? t.endMs - cur[0].startMs : 0;
+    if (cur.length >= maxWords || span > maxGapMs) {
+      if (cur.length) pages.push(pageOf(cur)); cur = [];
+    }
+    cur.push(t);
+  }
+  if (cur.length) pages.push(pageOf(cur));
+  return pages;
+}
+const pageOf = (ts: Token[]): Page => ({ startMs: ts[0].startMs, endMs: ts.at(-1)!.endMs, tokens: ts });
+```
+
+Remotion equivalent:
+
+```ts
+import { createTikTokStyleCaptions } from "@remotion/captions";
+const { pages } = createTikTokStyleCaptions({ captions, combineTokensWithinMilliseconds: 1200 });
+// 200–500ms → word-by-word; 1000–1500ms → short readable phrases
+```
+
+## 5. Remotion component: word-timed with active highlight
+
+```tsx
+import { AbsoluteFill, Audio, Sequence, useCurrentFrame, useVideoConfig,
+         spring, interpolate, staticFile } from "remotion";
+
+const CAPTION_STYLE: React.CSSProperties = {
+  fontFamily: "Montserrat, Inter, sans-serif",
+  fontWeight: 800,
+  fontSize: 72,                         // ~8% of 1920px height
+  color: "#fff",
+  textAlign: "center",
+  textTransform: "uppercase",
+  WebkitTextStroke: "5px #000",         // heavy outline survives any background
+  paintOrder: "stroke fill",            // stroke behind fill, not clipping glyphs
+  textShadow: "0 4px 16px rgba(0,0,0,.55)",
+  lineHeight: 1.1,
+  padding: "0 80px",                    // keep inside center 80% width
+};
+
+const Word: React.FC<{ token: Token; active: boolean }> = ({ token, active }) => {
+  const frame = useCurrentFrame();
+  const { fps } = useVideoConfig();
+  const enter = (token.startMs / 1000) * fps;
+  const p = spring({ frame: frame - enter, fps, config: { damping: 12, mass: 0.6 } });
+  return (
+    <span style={{
+      display: "inline-block",
+      transform: `scale(${interpolate(p, [0, 1], [0.6, 1])})`,
+      opacity: interpolate(p, [0, 1], [0, 1]),
+      color: active ? "#FFE45E" : undefined,
+      transition: "color 80ms linear",
+    }}>{token.text}&nbsp;</span>
+  );
+};
+
+export const Captions: React.FC<{ pages: Page[]; audio: string }> = ({ pages, audio }) => {
+  const { fps } = useVideoConfig();
+  const f = (ms: number) => Math.round((ms / 1000) * fps);
+  const frame = useCurrentFrame();
+  return (
+    <AbsoluteFill>
+      <Audio src={staticFile(audio)} />        {/* same audio the transcript came from */}
+      {pages.map((pg, i) => (
+        <Sequence key={i} from={f(pg.startMs)} durationInFrames={Math.max(1, f(pg.endMs - pg.startMs))}>
+          <AbsoluteFill style={{ justifyContent: "flex-start", alignItems: "center", paddingTop: "62%" }}>
+            <div style={CAPTION_STYLE}>
+              {pg.tokens.map((t, j) => {
+                const now = (frame / fps) * 1000;
+                return <Word key={j} token={t} active={now >= t.startMs && now < t.endMs} />;
+              })}
+            </div>
+          </AbsoluteFill>
+        </Sequence>
+      ))}
+    </AbsoluteFill>
+  );
+};
+```
+
+Because the `<Audio>` and the timestamps share one timebase, voice and captions stay locked with no manual nudging. Re-record the VO → re-transcribe → done.
+
+## 6. Web/CSS path (live DOM player)
+
+```js
+function renderTrack(tokens, mountEl) {
+  tokens.forEach(t => {
+    const el = document.createElement("span");
+    el.className = "word";
+    el.textContent = t.text + " ";
+    el.style.animationDelay = `${t.startMs}ms`;   // each word pops on its own start
+    mountEl.append(el);
+  });
+}
+// drive .active off the media element's currentTime
+audio.addEventListener("timeupdate", () => {
+  const ms = audio.currentTime * 1000;
+  [...track.children].forEach((el, i) =>
+    el.classList.toggle("active", ms >= tokens[i].startMs && ms < tokens[i].endMs));
+});
+```
+
+```css
+.cap   { font: 800 72px/1.1 Montserrat, sans-serif; text-transform: uppercase;
+         color:#fff; -webkit-text-stroke:5px #000; paint-order:stroke fill;
+         text-shadow:0 4px 16px rgba(0,0,0,.55); text-align:center; }
+.word  { display:inline-block; opacity:0; animation: pop .26s cubic-bezier(.34,1.56,.64,1) forwards; }
+.word.active { color:#FFE45E; }
+@keyframes pop { from { opacity:0; transform:translateY(.18em) scale(.7) } to { opacity:1; transform:none } }
+```
+
+## 7. Emit a sidecar SRT / VTT from the same tokens
+
+Burn-in alone does not satisfy ADA/WCAG and many platforms ignore uploaded tracks — so ship both. Group tokens into readable cues and serialize:
+
+```ts
+const pad = (n: number, w = 2) => String(n).padStart(w, "0");
+const stamp = (ms: number, sep: string) => {
+  const h = Math.floor(ms / 3.6e6), m = Math.floor(ms % 3.6e6 / 6e4),
+        s = Math.floor(ms % 6e4 / 1000), x = ms % 1000;
+  return `${pad(h)}:${pad(m)}:${pad(s)}${sep}${pad(x, 3)}`;
+};
+
+export const toSrt = (pages: Page[]) => pages.map((p, i) =>
+  `${i + 1}\n${stamp(p.startMs, ",")} --> ${stamp(p.endMs, ",")}\n${p.tokens.map(t => t.text).join(" ")}`
+).join("\n\n") + "\n";
+
+export const toVtt = (pages: Page[]) => "WEBVTT\n\n" + pages.map(p =>
+  `${stamp(p.startMs, ".")} --> ${stamp(p.endMs, ".")}\n${p.tokens.map(t => t.text).join(" ")}`
+).join("\n\n") + "\n";
+```
+
+Use SRT for the broadest platform support; VTT when styling/positioning cues or serving on the web.
+
+## 8. Per-platform safe areas (1080×1920, 9:16)
+
+| Platform | Bottom keep-clear | Top keep-clear | Side keep-clear | Notes |
+|---|---|---|---|---|
+| TikTok | ~340px | ~120px | ~110px right rail | Captions, audio title, action buttons crowd the bottom-right |
+| Instagram Reels | ~280px | ~130px | ~60px | Username/CTA bar at bottom |
+| YouTube Shorts | ~240px | ~100px | ~120px right rail | Title + channel at bottom-left |
+| Facebook Reels | ~300px | ~130px | ~80px | Similar to Reels |
+
+Safe rule across all: place caption baseline at ~62–70% of frame height, keep text within the center 80% width, and never let it enter the bottom ~15%.
+
+## 9. Readable-type spec sheet
+
+| Property | Recommended | Floor |
+|---|---|---|
+| Weight | 800 (ExtraBold) | 700 |
+| Size @1080p | 56–80px | 45px |
+| Stroke | 4–6px solid #000 | 2px |
+| Shadow | `0 4px 16px rgba(0,0,0,.55)` | soft 1px |
+| Active accent | one color (e.g. #FFE45E) or filled box | — |
+| Lines on screen | 1–2 | 2 max |
+| Words per page | 1–4 | 5 max |
+
+Set `paint-order: stroke fill` so a thick stroke sits behind the glyph instead of eating it. Test on a bright frame and a busy frame; if it survives both, it ships.
+
+## 10. Render
+
+```bash
+npx remotion render src/index.ts Captions out/captions.mp4 \
+  --props='{"pages": ..., "audio": "voiceover.wav"}'
+```
+
+A 10-minute video with animated captions can take 30–60 min to render locally; render long-form in segments, or render on Remotion Lambda for parallel speed.
+
+---
+## Built by the team behind iart.ai
+
+This skill is part of an open motion-graphics collection from iart.ai — the AI motion agent that turns data, scripts, and designs into editable motion graphics (Remotion → MP4). If you'd rather not hand-build this, iart.ai can auto-time captions to a narration track and one-click burn them in from a template — change the text/audio and re-export. → [iart.ai](https://iart.ai/?utm_source=github&utm_medium=reference&utm_campaign=tiktok-video-skills&utm_content=ref_footer&utm_term=caption-animation)