Add HyperFrames skills (from heygen-com/hyperframes, Apache-2.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FXVsD1DgdzZLiPuwtKzMSY
This commit is contained in:
Claude committed 2026-10-21 23:18:32 +00:00
1 parent 3e3f41ee5f
commit daac216bb9
300 files changed
+49175

No files matched your search

@@ -0,0 +1,10 @@
# Render output
remotion-src/out/
hf-src/out/
hf.mp4
diff/
strip/
# Remotion / HF dependencies
node_modules/
package-lock.json
@@ -0,0 +1,85 @@
# Tier 3 — stargazed-data-driven
## What it tests
A purpose-built data-driven fixture that exercises the realistic shape of a
production Remotion composition without using the runtime adapter from
PR #214. If a translation passes T3, the skill correctly handles:
- A `<Composition>` with a `z.object` schema and typed `defaultProps`
- Custom React subcomponents reused with different props across scenes
- A nested data structure (`stats[]`) materialized as repeated HTML with
per-instance attributes
- A frame-driven count-up animation (`AnimatedNumber` → GSAP `onUpdate`)
- Two different `spring` configs translated to two different `back.out` overshoots
- Per-instance delays via component props (`delayInFrames` → GSAP timeline offsets)
## Composition shape
```
Stargazed (10 s @ 30 fps, 1280×720)
├── Sequence 0–3 s TitleScene
│ ├── title ← spring scale
│ └── subtitle ← linear fade
├── Sequence 3–7 s StatsScene
│ ├── StatCard "Stars" 1247 #fbbf24 (delay 0 frames)
│ ├── StatCard "Forks" 312 #60a5fa (delay 12 frames)
│ └── StatCard "Issues" 48 #f87171 (delay 24 frames)
└── Sequence 7–10 s OutroScene
└── UnderlinedText "thanks for watching" ← scale-in underline
```
Each `StatCard` is a custom subcomponent that internally uses `AnimatedNumber`
to count from 0 to the target. `AnimatedNumber` itself derives the displayed
value from `useCurrentFrame()` + a manual `1 - (1 - t)^3` ease.
## The lossy parts (and why threshold = 0.90)
1. **`spring → back.out(N)`**: two different spring configs in this composition.
- `{ damping: 12, stiffness: 100, mass: 1 }` (title) → `back.out(1.4)`
- `{ damping: 14, stiffness: 90, mass: 1 }` (stat card) → `back.out(1.2)`
Overshoot ratio (1.4 vs 1.2) approximates the damping difference. The
late-tail curve of GSAP's back ease and Remotion's spring don't match
exactly — costs ~0.03 mean SSIM per spring instance.
2. **Count-up easing**: `AnimatedNumber` uses `1 - (1 - t)^3` (cubic ease-out)
manually computed in the component. GSAP's `power3.out` is the same curve
shape — should match closely. The displayed integer is rounded each frame
in both renderers; minor mismatches occur when the rounded value flips
between two numbers on a sub-frame timing difference.
3. **Font rendering**: same caveat as T1/T2. System Helvetica/Arial fallback
produces minor anti-aliasing differences between renderers. Affects the
stat card numbers (large weight 800) most.
A mean SSIM below 0.90 in T3 indicates a _structural_ mismatch (wrong scene
durations, wrong stagger timing, missing prop wiring), not approximation
drift. That's the failure signal we care about. The calibrated mean against
Remotion @ 4.0 with PNG/BT.709 output is 0.953.
## Translation walk-through (skill cheat sheet)
| Remotion | HyperFrames |
| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| `<Composition schema={z.object({...})} defaultProps={...} />` | data-\* attributes on root `#stage` div |
| nested array prop (`stats[]`) | repeated HTML markup with per-instance `data-*` attrs |
| custom React subcomponent | inline repeated HTML using the component's prop interface as the template |
| `<AnimatedNumber from={0} to={value} dur={45} />` (cubic ease-out count-up) | tween on `{ v: 0 }` object with `onUpdate` rewriting `textContent`, ease `power3.out` |
| `spring({damping:12, stiffness:100})` | `back.out(1.4)` over ~0.7 s |
| `spring({damping:14, stiffness:90})` | `back.out(1.2)` over ~0.7 s |
| `delayInFrames={i * 12}` (per-instance) | GSAP timeline offset `(i * 0.4)` s |
| `useVideoConfig()` to get `fps` | dropped — composition fps is in `data-fps` on `#stage` |
## How to render and evaluate
```bash
# Render Remotion baseline (no setup.sh — no binary assets in this fixture)
cd remotion-src && npm install && npm run render
# Render HyperFrames translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# Compare
../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff
```
@@ -0,0 +1,41 @@
{
"tier": 3,
"name": "stargazed-data-driven",
"composition_id": "Stargazed",
"description": "Data-driven 10-second composition with three scenes, custom React subcomponents reused across scenes, a Zod schema with defaultProps, and a count-up number animation. Translates the realistic shape of a production Remotion composition into HF — without using the runtime adapter from PR #214.",
"duration_seconds": 10,
"fps": 30,
"width": 1280,
"height": 720,
"ssim_threshold": 0.9,
"validation": {
"measured_mean_ssim": 0.953,
"measured_min_ssim": 0.927,
"measured_p05_ssim": 0.938,
"measured_p95_ssim": 0.978,
"measured_at": "2026-09-18",
"measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.7.71 (sub-composition fixtures)",
"notes": "Count-up timing in StatsScene shows a few-frame offset between Remotion's manual 1-(1-t)^3 and GSAP's power3.out — both formulas are identical, so the offset comes from sub-frame timing of when the seek + onUpdate fire. Final values converge correctly. Visible only as transient digit mismatches mid-animation; no SSIM impact above the noise floor."
},
"remotion_apis_exercised": [
"Composition with z.object schema and typed defaultProps",
"Sequence (3 nested with computed offsets)",
"AbsoluteFill",
"useCurrentFrame, useVideoConfig",
"interpolate (single-segment, multi-segment, with extrapolation)",
"spring (two configs: damping:12 and damping:14)",
"Custom React subcomponents reused with different props (StatCard ×3)",
"Custom React utility component (AnimatedNumber for count-up)",
"Custom React utility component (UnderlinedText)",
"Per-instance delay via prop (delayInFrames)"
],
"translation_notes": [
"Zod schema + defaultProps → data-* attributes on the root #stage div. The skill emits one data attribute per scalar prop; nested arrays (stats[]) get materialized as repeated HTML markup with per-instance data attributes (data-stat-index, data-stat-value, --card-color).",
"Custom React subcomponents inline as repeated HTML divs. The component prop interface becomes the repeated markup template. This is lossy for components with internal state — fine here because StatCard, AnimatedNumber, UnderlinedText all derive from props alone.",
"AnimatedNumber's frame-driven count-up → a GSAP tween on a { v: 0 } counter object with onUpdate rewriting textContent. GSAP's power3.out is cubic easeOut, matching the Remotion 1-(1-t)^3 manual ease.",
"Two different springs in this composition: damping:12 → back.out(1.4) (snappy), damping:14 → back.out(1.2) (calmer). The 1.4 vs 1.2 overshoot ratio approximates the damping difference.",
"Per-instance stagger via delayInFrames prop translates to a GSAP timeline offset of (i * 0.4)s.",
"Threshold 0.90 reflects: spring → back.out approximation (×2 different configs), the count-up easing curve match (very close but not identical due to sub-frame seek timing), font/AA differences on body text. SSIM well below 0.90 indicates a structural mismatch, not approximation drift."
],
"rationale": "Threshold 0.90 sits ~0.04 below measured p05 (0.938). The wider gap vs T1/T2 reflects T3's bigger approximation budget (2 spring instances + count-up timing + font fallback on multiple text sizes). Mean SSIM below 0.90 = structural mismatch (wrong durations, wrong stagger, missing prop wiring), not approximation drift."
}
@@ -0,0 +1,52 @@
<template id="scene-1-template">
<div data-composition-id="scene-1" data-width="1280" data-height="720" data-duration="3">
<style>
.scene {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
opacity: 0;
}
.scene-1 {
flex-direction: column;
}
.scene-1 .title {
font-size: 160px;
font-weight: 900;
letter-spacing: 0.05em;
transform: scale(0);
}
.scene-1 .subtitle {
font-size: 36px;
font-weight: 400;
color: #9ca3af;
margin-top: 24px;
opacity: 0;
}
</style>
<div class="scene scene-1">
<div class="title">STARGAZED</div>
<div class="subtitle">by HeyGen</div>
</div>
<script>
// Remotion: <Sequence 0..90> TitleScene. Frame -> time at 30 fps.
const tl = gsap.timeline({ paused: true });
const scene = document.querySelector(".scene-1");
tl.set(scene, { opacity: 1 }, 0);
// spring({damping:12, stiffness:100, mass:1}) -> back.out(1.4) ~0.7s
tl.to(scene.querySelector(".title"), { scale: 1, duration: 0.7, ease: "back.out(1.4)" }, 0);
// interpolate(frame, [20,40], [0,1]) -> 0.667s..1.333s linear
tl.fromTo(
scene.querySelector(".subtitle"),
{ opacity: 0 },
{ opacity: 1, duration: 0.667, ease: "none" },
0.667,
);
tl.set(scene, { opacity: 0 }, 3);
window.__timelines = window.__timelines || {};
window.__timelines["scene-1"] = tl;
</script>
</div>
</template>
@@ -0,0 +1,99 @@
<template id="scene-2-template">
<div data-composition-id="scene-2" data-width="1280" data-height="720" data-duration="4">
<style>
.scene {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
opacity: 0;
}
.scene-2 {
gap: 48px;
}
.stat-card {
width: 280px;
height: 220px;
background: #1a1a1a;
border-radius: 16px;
border: 2px solid var(--card-color);
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
opacity: 0;
transform: scale(0);
}
.stat-card .number {
font-size: 72px;
font-weight: 800;
color: var(--card-color);
line-height: 1;
}
.stat-card .label {
font-size: 24px;
font-weight: 500;
color: #9ca3af;
margin-top: 16px;
text-transform: uppercase;
letter-spacing: 0.1em;
}
</style>
<div class="scene scene-2">
<div
class="stat-card"
data-stat-index="0"
data-stat-value="1247"
style="--card-color: #fbbf24"
>
<div class="number">0</div>
<div class="label">Stars</div>
</div>
<div
class="stat-card"
data-stat-index="1"
data-stat-value="312"
style="--card-color: #60a5fa"
>
<div class="number">0</div>
<div class="label">Forks</div>
</div>
<div class="stat-card" data-stat-index="2" data-stat-value="48" style="--card-color: #f87171">
<div class="number">0</div>
<div class="label">Issues</div>
</div>
</div>
<script>
// Remotion: <Sequence 90..210> StatsScene; StatCard stagger i*12 frames = i*0.4s.
const tl = gsap.timeline({ paused: true });
const scene = document.querySelector(".scene-2");
tl.set(scene, { opacity: 1 }, 0);
scene.querySelectorAll(".stat-card").forEach((card, i) => {
const start = i * 0.4;
const value = Number(card.dataset.statValue);
const numberEl = card.querySelector(".number");
// spring({damping:14, stiffness:90, mass:1}) -> back.out(1.2) ~0.7s; opacity 0..0.4s linear
tl.to(card, { scale: 1, duration: 0.7, ease: "back.out(1.2)" }, start);
tl.to(card, { opacity: 1, duration: 0.4, ease: "none" }, start);
// AnimatedNumber: easeOutCubic over 45 frames (1.5s) is GSAP power3.out on a counter.
const counter = { v: 0 };
tl.to(
counter,
{
v: value,
duration: 1.5,
ease: "power3.out",
onUpdate: () => {
numberEl.textContent = Math.round(counter.v).toLocaleString();
},
},
start,
);
});
tl.set(scene, { opacity: 0 }, 4);
window.__timelines = window.__timelines || {};
window.__timelines["scene-2"] = tl;
</script>
</div>
</template>
@@ -0,0 +1,55 @@
<template id="scene-3-template">
<div data-composition-id="scene-3" data-width="1280" data-height="720" data-duration="3">
<style>
.scene {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
opacity: 0;
}
.scene-3 .outro-wrap {
position: relative;
display: inline-block;
opacity: 0;
}
.scene-3 .outro-text {
font-size: 80px;
font-weight: 600;
}
.scene-3 .outro-underline {
position: absolute;
left: 0;
bottom: -8px;
width: 100%;
height: 6px;
background: #fbbf24;
border-radius: 3px;
transform: scaleX(0);
transform-origin: left center;
}
</style>
<div class="scene scene-3">
<div class="outro-wrap">
<div class="outro-text">thanks for watching</div>
<div class="outro-underline"></div>
</div>
</div>
<script>
// Remotion: <Sequence 210..300> OutroScene.
const tl = gsap.timeline({ paused: true });
const scene = document.querySelector(".scene-3");
tl.set(scene, { opacity: 1 }, 0);
tl.to(scene.querySelector(".outro-wrap"), { opacity: 1, duration: 0.4, ease: "none" }, 0);
// interpolate(frame, [10,40], [0,1]) for the underline -> starts 0.333s, 1s long
tl.to(
scene.querySelector(".outro-underline"),
{ scaleX: 1, duration: 1.0, ease: "none" },
0.333,
);
window.__timelines = window.__timelines || {};
window.__timelines["scene-3"] = tl;
</script>
</div>
</template>
@@ -0,0 +1,70 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>tier-3-data-driven</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.5/gsap.min.js"></script>
<style>
html,
body {
margin: 0;
padding: 0;
width: 1280px;
height: 720px;
overflow: hidden;
background: #0a0a0a;
font-family: Helvetica, Arial, sans-serif;
color: #ffffff;
}
</style>
</head>
<body>
<div
id="stage"
data-composition-id="tier-3-data-driven"
data-start="0"
data-width="1280"
data-height="720"
data-duration="10"
data-fps="30"
data-title="STARGAZED"
data-subtitle="by HeyGen"
data-outro="thanks for watching"
>
<div
id="scene-1"
class="clip"
data-composition-id="scene-1"
data-composition-src="compositions/scene-1.html"
data-start="0"
data-duration="3"
data-track-index="0"
></div>
<div
id="scene-2"
class="clip"
data-composition-id="scene-2"
data-composition-src="compositions/scene-2.html"
data-start="3"
data-duration="4"
data-track-index="0"
></div>
<div
id="scene-3"
class="clip"
data-composition-id="scene-3"
data-composition-src="compositions/scene-3.html"
data-start="7"
data-duration="3"
data-track-index="0"
></div>
<script>
window.__timelines = window.__timelines || {};
window.__timelines["tier-3-data-driven"] = gsap.timeline({ paused: true });
</script>
</div>
</body>
</html>
@@ -0,0 +1,15 @@
{
"name": "tier-3-data-driven-remotion",
"version": "0.0.0",
"private": true,
"scripts": {
"render": "remotion render Stargazed out/baseline.mp4"
},
"dependencies": {
"@remotion/cli": "^4.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"remotion": "^4.0.0",
"zod": "^3.22.0"
}
}
@@ -0,0 +1,13 @@
import { Config } from "@remotion/cli/config";
// Match HyperFrames' default render so SSIM diffs measure translation
// fidelity, not encoder differences.
//
// setVideoImageFormat("png") avoids the JPEG limited-range/full-range
// colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM.
//
// setColorSpace("bt709") matches HF's BT.709 SDR output.
Config.setVideoImageFormat("png");
Config.setColorSpace("bt709");
Config.setOverwriteOutput(true);
Config.setConcurrency(1);
@@ -0,0 +1,27 @@
import { Composition } from "remotion";
import { z } from "zod";
import { Stargazed, stargazedSchema } from "./Stargazed";
const defaultProps: z.infer<typeof stargazedSchema> = {
title: "STARGAZED",
subtitle: "by HeyGen",
stats: [
{ label: "Stars", value: 1247, color: "#fbbf24" },
{ label: "Forks", value: 312, color: "#60a5fa" },
{ label: "Issues", value: 48, color: "#f87171" },
],
outro: "thanks for watching",
};
export const RemotionRoot = () => (
<Composition
id="Stargazed"
component={Stargazed}
schema={stargazedSchema}
durationInFrames={300}
fps={30}
width={1280}
height={720}
defaultProps={defaultProps}
/>
);
@@ -0,0 +1,37 @@
import { AbsoluteFill, Sequence } from "remotion";
import { z } from "zod";
import { TitleScene } from "./scenes/TitleScene";
import { StatsScene } from "./scenes/StatsScene";
import { OutroScene } from "./scenes/OutroScene";
export const stargazedSchema = z.object({
title: z.string(),
subtitle: z.string(),
stats: z.array(
z.object({
label: z.string(),
value: z.number(),
color: z.string(),
}),
),
outro: z.string(),
});
export const Stargazed: React.FC<z.infer<typeof stargazedSchema>> = ({
title,
subtitle,
stats,
outro,
}) => (
<AbsoluteFill style={{ backgroundColor: "#0a0a0a" }}>
<Sequence from={0} durationInFrames={90}>
<TitleScene title={title} subtitle={subtitle} />
</Sequence>
<Sequence from={90} durationInFrames={120}>
<StatsScene stats={stats} />
</Sequence>
<Sequence from={210} durationInFrames={90}>
<OutroScene text={outro} />
</Sequence>
</AbsoluteFill>
);
@@ -0,0 +1,23 @@
import { interpolate, useCurrentFrame } from "remotion";
interface Props {
from: number;
to: number;
durationInFrames: number;
}
/**
* Counts from `from` to `to` over `durationInFrames` with easeOut.
* Driven entirely by useCurrentFrame — deterministic.
*/
export const AnimatedNumber: React.FC<Props> = ({ from, to, durationInFrames }) => {
const frame = useCurrentFrame();
const t = interpolate(frame, [0, durationInFrames], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
// Ease-out cubic — fast start, slow finish, matches the ramp on data dashboards.
const eased = 1 - (1 - t) ** 3;
const value = Math.round(from + (to - from) * eased);
return <>{value.toLocaleString()}</>;
};
@@ -0,0 +1,59 @@
import { interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
import { AnimatedNumber } from "./AnimatedNumber";
interface Props {
label: string;
value: number;
color: string;
delayInFrames: number;
}
export const StatCard: React.FC<Props> = ({ label, value, color, delayInFrames }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const local = frame - delayInFrames;
const scale = spring({
frame: local,
fps,
config: { damping: 14, stiffness: 90, mass: 1 },
});
const opacity = interpolate(local, [0, 12], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<div
style={{
width: 280,
height: 220,
background: "#1a1a1a",
borderRadius: 16,
border: `2px solid ${color}`,
display: "flex",
flexDirection: "column",
justifyContent: "center",
alignItems: "center",
opacity,
transform: `scale(${scale})`,
}}
>
<div style={{ fontSize: 72, fontWeight: 800, color, lineHeight: 1 }}>
{local >= 0 ? <AnimatedNumber from={0} to={value} durationInFrames={45} /> : 0}
</div>
<div
style={{
fontSize: 24,
fontWeight: 500,
color: "#9ca3af",
marginTop: 16,
textTransform: "uppercase",
letterSpacing: "0.1em",
}}
>
{label}
</div>
</div>
);
};
@@ -0,0 +1,47 @@
import { interpolate, useCurrentFrame } from "remotion";
interface Props {
text: string;
color: string;
}
export const UnderlinedText: React.FC<Props> = ({ text, color }) => {
const frame = useCurrentFrame();
// Underline scales from left over 0-30 frames.
const underlineScaleX = interpolate(frame, [10, 40], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
const opacity = interpolate(frame, [0, 12], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<div style={{ position: "relative", display: "inline-block", opacity }}>
<div
style={{
fontSize: 80,
fontWeight: 600,
color: "#ffffff",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
{text}
</div>
<div
style={{
position: "absolute",
left: 0,
bottom: -8,
width: "100%",
height: 6,
background: color,
transform: `scaleX(${underlineScaleX})`,
transformOrigin: "left center",
borderRadius: 3,
}}
/>
</div>
);
};
@@ -0,0 +1,4 @@
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
@@ -0,0 +1,18 @@
import { AbsoluteFill } from "remotion";
import { UnderlinedText } from "../components/UnderlinedText";
interface Props {
text: string;
}
export const OutroScene: React.FC<Props> = ({ text }) => (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<UnderlinedText text={text} color="#fbbf24" />
</AbsoluteFill>
);
@@ -0,0 +1,34 @@
import { AbsoluteFill } from "remotion";
import { StatCard } from "../components/StatCard";
interface Stat {
label: string;
value: number;
color: string;
}
interface Props {
stats: Stat[];
}
export const StatsScene: React.FC<Props> = ({ stats }) => (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
gap: 48,
flexDirection: "row",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
{stats.map((stat, i) => (
<StatCard
key={stat.label}
label={stat.label}
value={stat.value}
color={stat.color}
delayInFrames={i * 12}
/>
))}
</AbsoluteFill>
);
@@ -0,0 +1,55 @@
import { AbsoluteFill, interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
interface Props {
title: string;
subtitle: string;
}
export const TitleScene: React.FC<Props> = ({ title, subtitle }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const titleScale = spring({
frame,
fps,
config: { damping: 12, stiffness: 100, mass: 1 },
});
const subtitleOpacity = interpolate(frame, [20, 40], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
flexDirection: "column",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<div
style={{
fontSize: 160,
fontWeight: 900,
color: "#ffffff",
letterSpacing: "0.05em",
transform: `scale(${titleScale})`,
}}
>
{title}
</div>
<div
style={{
fontSize: 36,
fontWeight: 400,
color: "#9ca3af",
marginTop: 24,
opacity: subtitleOpacity,
}}
>
{subtitle}
</div>
</AbsoluteFill>
);
};
@@ -0,0 +1,15 @@
{
"compilerOptions": {
"target": "ES2018",
"module": "ESNext",
"jsx": "react-jsx",
"strict": true,
"esModuleInterop": true,
"moduleResolution": "node",
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true
},
"include": ["src"]
}