Add HyperFrames skills (from heygen-com/hyperframes, Apache-2.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FXVsD1DgdzZLiPuwtKzMSY
2026-10-21 23:18:32 +00:00
1 parent 3e3f41ee5f
commit daac216bb9
@@ -0,0 +1,14 @@
+# Generated by setup.sh
+remotion-src/public/
+hf-src/assets/
+
+# Remotion / HF dependencies
+remotion-src/node_modules/
+remotion-src/package-lock.json
+
+# Render output
+remotion-src/out/
+hf-src/out/
+hf.mp4
+diff/
+strip/
@@ -0,0 +1,54 @@
+# Tier 2 — title-image-outro
+
+## What it tests
+
+Three-scene composition. Each scene exercises a different Remotion idiom:
+
+1. **Scene 1 (0–2 s)** — TitleScene with `spring({damping:12, stiffness:100, mass:1})`
+   driving a `transform: scale()` on text. Tests the lossy `spring → GSAP ease` translation.
+2. **Scene 2 (2–4 s)** — ImageScene that fades in a `staticFile`-loaded image and
+   linearly scales it from 0.8 → 1.0. Tests asset paths + linear `interpolate`.
+3. **Scene 3 (4–6 s)** — OutroScene with a 1-s linear fade-in. Sanity check after
+   the harder scenes.
+
+A silent 6-second WAV plays throughout at `volume={0.5}`. Tests `<Audio>` translation.
+
+If a translation passes T2, the skill correctly handles `<Sequence>` boundaries,
+`<Audio>` / `<Img>` / `staticFile`, and the Remotion `spring → GSAP ease` heuristic.
+
+## Translation walk-through
+
+| Remotion                                                            | HyperFrames                                                                                                         |
+| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
+| `<Sequence from={0} durationInFrames={60}>`                         | host `<div data-composition-src="compositions/scene-1.html" data-start="0" data-duration="2" data-track-index="0">` |
+| `spring({frame, fps, config: {damping:12, stiffness:100, mass:1}})` | `gsap.to(target, { scale: 1, duration: 0.7, ease: "back.out(1.4)" })`                                               |
+| `<Audio src={staticFile("music.wav")} volume={0.5} />`              | `<audio src="assets/music.wav" data-start="0" data-duration="6" data-volume="0.5" data-track-index="1">`            |
+| `<Img src={staticFile("square.png")} />`                            | `<img src="../assets/square.png">` inside `compositions/scene-2.html` (with setup.sh copying into both trees)       |
+| `interpolate(frame, [0, 15], [0, 1])` at 30 fps                     | `gsap.to(target, { opacity: 1, duration: 0.5, ease: "none" })`                                                      |
+
+The scene crossfading is a HyperFrames idiom, not a Remotion one: at scene boundaries
+each scene sub-composition sets its own `gsap.set(scene, { opacity: 0 })` at its local end so the scene disappears at the
+right time. Remotion does this implicitly by virtue of `<Sequence>`'s durationInFrames.
+
+## How to render and evaluate
+
+```bash
+# 1. Generate the binary assets (PNG + WAV) via ffmpeg
+./setup.sh
+
+# 2. Render Remotion baseline
+cd remotion-src && npm install && npm run render
+
+# 3. Render HyperFrames translation
+cd ../hf-src && npx hyperframes render --output ../hf.mp4
+
+# 4. Compare
+../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff
+```
+
+## Why threshold 0.95?
+
+Same threshold as T1 (`expected.json` codifies it for the orchestrator). Spring → `back.out(1.4)`
+came in cleaner than predicted during calibration — the validated mean is 0.985 against the
+0.95 gate. If the translation breaks anything else (spring overshoot wrong, stagger off,
+asset path drift), mean SSIM will fall well below 0.95 — that's the failure signal.
@@ -0,0 +1,29 @@
+{
+  "tier": 2,
+  "name": "title-image-outro",
+  "composition_id": "MultiScene",
+  "description": "Three-scene composition exercising Sequence, spring, interpolate, Audio, Img, and staticFile. Title scene uses Remotion's spring (translated to GSAP back.out as an approximation). Image scene scales an Img (from staticFile) with linear interpolate. Outro scene fades text in linearly. A silent WAV plays throughout at volume 0.5.",
+  "duration_seconds": 6,
+  "fps": 30,
+  "width": 1280,
+  "height": 720,
+  "ssim_threshold": 0.95,
+  "validation": {
+    "measured_mean_ssim": 0.986,
+    "measured_min_ssim": 0.964,
+    "measured_p05_ssim": 0.967,
+    "measured_p95_ssim": 1,
+    "measured_at": "2026-09-18",
+    "measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.7.71 (sub-composition fixtures)",
+    "notes": "Spring → back.out(1.4) translation came out cleaner than expected; mean 0.986 leaves substantial headroom over 0.95."
+  },
+  "translation_notes": [
+    "spring({damping:12, stiffness:100, mass:1}) → back.out(1.4) over 0.7s. Spring overshoot+settle and back.out's overshoot+settle have similar shape; budget ~0.02 SSIM for the late-tail curvature mismatch (validated lower than predicted in spec; original notes overestimated drift).",
+    "<Sequence from durationInFrames> → a sub-composition host with data-start/data-duration in seconds; the scene's own file uses local time and explicit gsap.set(opacity, 0/1) at its boundaries to crossfade in/out cleanly",
+    "<Audio src volume> → <audio data-start data-duration data-volume>",
+    "<Img src={staticFile('x')}> → <img src='assets/x'> (`../assets/x` inside the scene sub-composition file); setup.sh copies the asset into both fixture trees",
+    "interpolate with default linear easing → ease:'none' in GSAP",
+    "Fonts again rely on system Helvetica/Arial; ~0.015 SSIM cost from AA differences"
+  ],
+  "rationale": "Threshold 0.95 sits ~0.015 below measured p05 (0.966). T2 actually validated cleaner than T1 because the lower title fontWeight (140px vs T1's 160px) shows less of the system-font fallback divergence."
+}
@@ -0,0 +1,35 @@
+<template id="scene-1-template">
+  <div data-composition-id="scene-1" data-width="1280" data-height="720" data-duration="2">
+    <style>
+      .scene {
+        position: absolute;
+        inset: 0;
+        display: flex;
+        align-items: center;
+        justify-content: center;
+        opacity: 0;
+      }
+      .title {
+        font-size: 140px;
+        font-weight: 800;
+        color: #ffffff;
+        letter-spacing: 0.05em;
+        transform: scale(0);
+      }
+    </style>
+    <div class="scene">
+      <div class="title">Welcome</div>
+    </div>
+    <script>
+      // Scene 1 (0-2s): spring scale on "Welcome". Remotion:
+      //   spring({frame, fps, config: {damping:12, stiffness:100, mass:1}}) -> back.out(1.4) over ~0.7s.
+      const tl = gsap.timeline({ paused: true });
+      const scene = document.querySelector(".scene");
+      tl.set(scene, { opacity: 1 }, 0);
+      tl.to(scene.querySelector(".title"), { scale: 1, duration: 0.7, ease: "back.out(1.4)" }, 0);
+      tl.set(scene, { opacity: 0 }, 2);
+      window.__timelines = window.__timelines || {};
+      window.__timelines["scene-1"] = tl;
+    </script>
+  </div>
+</template>
@@ -0,0 +1,35 @@
+<template id="scene-2-template">
+  <div data-composition-id="scene-2" data-width="1280" data-height="720" data-duration="2">
+    <style>
+      .scene {
+        position: absolute;
+        inset: 0;
+        display: flex;
+        align-items: center;
+        justify-content: center;
+        opacity: 0;
+      }
+      .square {
+        width: 200px;
+        height: 200px;
+        opacity: 0;
+        transform: scale(0.8);
+      }
+    </style>
+    <div class="scene">
+      <img class="square" src="../assets/square.png" alt="" />
+    </div>
+    <script>
+      // Scene 2 (2-4s, local 0-2s): opacity 0->1 over 0.5s, scale 0.8->1.0 over 2s, both linear.
+      const tl = gsap.timeline({ paused: true });
+      const scene = document.querySelector(".scene");
+      const square = scene.querySelector(".square");
+      tl.set(scene, { opacity: 1 }, 0);
+      tl.to(square, { opacity: 1, duration: 0.5, ease: "none" }, 0);
+      tl.to(square, { scale: 1.0, duration: 2.0, ease: "none" }, 0);
+      tl.set(scene, { opacity: 0 }, 2);
+      window.__timelines = window.__timelines || {};
+      window.__timelines["scene-2"] = tl;
+    </script>
+  </div>
+</template>
@@ -0,0 +1,36 @@
+<template id="scene-3-template">
+  <div data-composition-id="scene-3" data-width="1280" data-height="720" data-duration="2">
+    <style>
+      .scene {
+        position: absolute;
+        inset: 0;
+        display: flex;
+        align-items: center;
+        justify-content: center;
+        opacity: 0;
+      }
+      .outro {
+        font-size: 100px;
+        font-weight: 600;
+        color: #ffffff;
+      }
+    </style>
+    <div class="scene">
+      <div class="outro">Goodbye</div>
+    </div>
+    <script>
+      // Scene 3 (4-6s, local 0-2s): opacity 0->1 over 1s, linear.
+      const tl = gsap.timeline({ paused: true });
+      const scene = document.querySelector(".scene");
+      tl.set(scene, { opacity: 1 }, 0);
+      tl.fromTo(
+        scene.querySelector(".outro"),
+        { opacity: 0 },
+        { opacity: 1, duration: 1.0, ease: "none" },
+        0,
+      );
+      window.__timelines = window.__timelines || {};
+      window.__timelines["scene-3"] = tl;
+    </script>
+  </div>
+</template>
@@ -0,0 +1,75 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <title>tier-2-multi-scene</title>
+    <script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.5/gsap.min.js"></script>
+    <style>
+      html,
+      body {
+        margin: 0;
+        padding: 0;
+        width: 1280px;
+        height: 720px;
+        overflow: hidden;
+        background: #0a0a0a;
+        font-family: Helvetica, Arial, sans-serif;
+      }
+    </style>
+  </head>
+  <body>
+    <div
+      id="stage"
+      data-composition-id="tier-2-multi-scene"
+      data-start="0"
+      data-width="1280"
+      data-height="720"
+      data-duration="6"
+      data-fps="30"
+    >
+      <div
+        id="scene-1"
+        class="clip"
+        data-composition-id="scene-1"
+        data-composition-src="compositions/scene-1.html"
+        data-start="0"
+        data-duration="2"
+        data-track-index="0"
+      ></div>
+
+      <div
+        id="scene-2"
+        class="clip"
+        data-composition-id="scene-2"
+        data-composition-src="compositions/scene-2.html"
+        data-start="2"
+        data-duration="2"
+        data-track-index="0"
+      ></div>
+
+      <div
+        id="scene-3"
+        class="clip"
+        data-composition-id="scene-3"
+        data-composition-src="compositions/scene-3.html"
+        data-start="4"
+        data-duration="2"
+        data-track-index="0"
+      ></div>
+
+      <audio
+        id="bg-music"
+        data-start="0"
+        data-duration="6"
+        data-track-index="1"
+        data-volume="0.5"
+        src="assets/music.wav"
+      ></audio>
+
+      <script>
+        window.__timelines = window.__timelines || {};
+        window.__timelines["tier-2-multi-scene"] = gsap.timeline({ paused: true });
+      </script>
+    </div>
+  </body>
+</html>
@@ -0,0 +1,14 @@
+{
+  "name": "tier-2-multi-scene-remotion",
+  "version": "0.0.0",
+  "private": true,
+  "scripts": {
+    "render": "remotion render MultiScene out/baseline.mp4"
+  },
+  "dependencies": {
+    "@remotion/cli": "^4.0.0",
+    "react": "^18.3.1",
+    "react-dom": "^18.3.1",
+    "remotion": "^4.0.0"
+  }
+}
@@ -0,0 +1,13 @@
+import { Config } from "@remotion/cli/config";
+
+// Match HyperFrames' default render so SSIM diffs measure translation
+// fidelity, not encoder differences.
+//
+//   setVideoImageFormat("png") avoids the JPEG limited-range/full-range
+//   colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM.
+//
+//   setColorSpace("bt709") matches HF's BT.709 SDR output.
+Config.setVideoImageFormat("png");
+Config.setColorSpace("bt709");
+Config.setOverwriteOutput(true);
+Config.setConcurrency(1);
@@ -0,0 +1,105 @@
+import {
+  AbsoluteFill,
+  Audio,
+  Img,
+  Sequence,
+  interpolate,
+  spring,
+  staticFile,
+  useCurrentFrame,
+  useVideoConfig,
+} from "remotion";
+
+const TitleScene = () => {
+  const frame = useCurrentFrame();
+  const { fps } = useVideoConfig();
+  const scale = spring({ frame, fps, config: { damping: 12, stiffness: 100, mass: 1 } });
+  return (
+    <AbsoluteFill
+      style={{
+        backgroundColor: "#0a0a0a",
+        justifyContent: "center",
+        alignItems: "center",
+        fontFamily: "Helvetica, Arial, sans-serif",
+      }}
+    >
+      <div
+        style={{
+          fontSize: 140,
+          fontWeight: 800,
+          color: "#ffffff",
+          transform: `scale(${scale})`,
+          letterSpacing: "0.05em",
+        }}
+      >
+        Welcome
+      </div>
+    </AbsoluteFill>
+  );
+};
+
+const ImageScene = () => {
+  const frame = useCurrentFrame();
+  const opacity = interpolate(frame, [0, 15], [0, 1], { extrapolateRight: "clamp" });
+  const scale = interpolate(frame, [0, 60], [0.8, 1.0], { extrapolateRight: "clamp" });
+  return (
+    <AbsoluteFill
+      style={{
+        backgroundColor: "#0a0a0a",
+        justifyContent: "center",
+        alignItems: "center",
+      }}
+    >
+      <Img
+        src={staticFile("square.png")}
+        style={{
+          width: 200,
+          height: 200,
+          opacity,
+          transform: `scale(${scale})`,
+        }}
+      />
+    </AbsoluteFill>
+  );
+};
+
+const OutroScene = () => {
+  const frame = useCurrentFrame();
+  const opacity = interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" });
+  return (
+    <AbsoluteFill
+      style={{
+        backgroundColor: "#0a0a0a",
+        justifyContent: "center",
+        alignItems: "center",
+        fontFamily: "Helvetica, Arial, sans-serif",
+      }}
+    >
+      <div
+        style={{
+          fontSize: 100,
+          fontWeight: 600,
+          color: "#ffffff",
+          opacity,
+        }}
+      >
+        Goodbye
+      </div>
+    </AbsoluteFill>
+  );
+};
+
+export const MultiScene = () => (
+  <AbsoluteFill>
+    <Sequence from={0} durationInFrames={60}>
+      <TitleScene />
+    </Sequence>
+    <Sequence from={60} durationInFrames={60}>
+      <ImageScene />
+    </Sequence>
+    <Sequence from={120} durationInFrames={60}>
+      <OutroScene />
+    </Sequence>
+    <Audio src={staticFile("music.wav")} volume={0.5} />
+  </AbsoluteFill>
+);
@@ -0,0 +1,13 @@
+import { Composition } from "remotion";
+import { MultiScene } from "./MultiScene";
+
+export const RemotionRoot = () => (
+  <Composition
+    id="MultiScene"
+    component={MultiScene}
+    durationInFrames={180}
+    fps={30}
+    width={1280}
+    height={720}
+  />
+);
@@ -0,0 +1,4 @@
+import { registerRoot } from "remotion";
+import { RemotionRoot } from "./Root";
+
+registerRoot(RemotionRoot);
@@ -0,0 +1,15 @@
+{
+  "compilerOptions": {
+    "target": "ES2018",
+    "module": "ESNext",
+    "jsx": "react-jsx",
+    "strict": true,
+    "esModuleInterop": true,
+    "moduleResolution": "node",
+    "skipLibCheck": true,
+    "forceConsistentCasingInFileNames": true,
+    "resolveJsonModule": true,
+    "isolatedModules": true
+  },
+  "include": ["src"]
+}
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# setup.sh — generate the binary assets this fixture needs.
+#
+# Both Remotion and HyperFrames variants need a 200x200 blue PNG and a
+# 6-second silent WAV. Generating them via ffmpeg keeps binaries out of
+# the repo while still letting the fixture render reproducibly.
+#
+# Run from the fixture root: ./setup.sh
+
+set -euo pipefail
+
+THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+if ! command -v ffmpeg >/dev/null 2>&1; then
+  echo "error: ffmpeg not on PATH" >&2
+  exit 2
+fi
+
+mkdir -p "$THIS_DIR/remotion-src/public" "$THIS_DIR/hf-src/assets"
+
+# 200x200 solid blue PNG, ~200 bytes.
+ffmpeg -y -hide_banner -loglevel error \
+  -f lavfi -i "color=color=#3066be:size=200x200" -frames:v 1 \
+  "$THIS_DIR/remotion-src/public/square.png"
+cp "$THIS_DIR/remotion-src/public/square.png" "$THIS_DIR/hf-src/assets/square.png"
+
+# 6-second silent WAV at 8 kHz mono. ~96 KB if checked in, but it is generated.
+ffmpeg -y -hide_banner -loglevel error \
+  -f lavfi -i "anullsrc=cl=mono:r=8000" -t 6 -acodec pcm_s16le \
+  "$THIS_DIR/remotion-src/public/music.wav"
+cp "$THIS_DIR/remotion-src/public/music.wav" "$THIS_DIR/hf-src/assets/music.wav"
+
+echo "generated:"
+ls -la "$THIS_DIR/remotion-src/public/" "$THIS_DIR/hf-src/assets/"