API guide

Everything the studio UI does, it does through this HTTP API — there are no private endpoints. Post a prompt, poll a job, download an mp4.

The shape of a job

A render is asynchronous. POST /api/jobs returns immediately with a job id; the work happens in a background worker. You poll GET /api/jobs/{id} until status is terminal, then fetch the video.

A 90-second 1080p render takes roughly 4–6 minutes. Draft quality is about four times faster and is the right choice while you are integrating.

Quickstart

Three ways to do the same thing: create a render and wait for the file.

# 1. create the job
JOB=$(curl -sX POST http://127.0.0.1:8000/api/jobs \
  -H 'Content-Type: application/json' \
  -d '{
        "prompt": "Explain how vaccines train the immune system",
        "target_seconds": 90,
        "orientation": "horizontal",
        "quality": "standard"
      }' | jq -r .id)

# 2. poll until it finishes
while :; do
  read -r STATUS PROGRESS STAGE <<<"$(curl -s http://127.0.0.1:8000/api/jobs/$JOB \
      | jq -r '"\(.status) \(.progress) \(.stage)"')"
  echo "$STATUS  $PROGRESS  $STAGE"
  case "$STATUS" in done|failed|cancelled) break;; esac
  sleep 3
done

# 3. download it
curl -s -o video.mp4 "http://127.0.0.1:8000/api/jobs/$JOB/video?download=true"

import time
import httpx

BASE = "http://127.0.0.1:8000"


def render(prompt: str, **options) -> bytes:
    """Create a render, wait for it, and return the mp4 bytes."""
    with httpx.Client(base_url=BASE, timeout=60) as client:
        job = client.post("/api/jobs", json={
            "prompt": prompt,
            "target_seconds": 90,
            "orientation": "horizontal",
            "quality": "standard",
            **options,
        }).raise_for_status().json()

        job_id = job["id"]
        while True:
            job = client.get(f"/api/jobs/{job_id}").raise_for_status().json()
            print(f"{job['status']:10} {job['progress']:6.1%}  {job['stage']}")

            if job["status"] == "done":
                break
            if job["status"] in ("failed", "cancelled"):
                raise RuntimeError(job.get("error") or job["stage"])
            time.sleep(3)

        # `warnings` is where soft problems land: a repaired anchor, a missing
        # icon, an image lookup that fell back. Worth logging, never fatal.
        for warning in job["warnings"]:
            print("warning:", warning)

        return client.get(f"/api/jobs/{job_id}/video", timeout=300).content


open("video.mp4", "wb").write(render("Explain how vaccines train the immune system"))

import { writeFile } from 'node:fs/promises';

const BASE = 'http://127.0.0.1:8000';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function render(prompt, options = {}) {
  const res = await fetch(`${BASE}/api/jobs`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      prompt,
      target_seconds: 90,
      orientation: 'horizontal',
      quality: 'standard',
      ...options,
    }),
  });
  if (!res.ok) throw new Error(JSON.stringify(await res.json()));

  let job = await res.json();
  while (!['done', 'failed', 'cancelled'].includes(job.status)) {
    await sleep(3000);
    job = await (await fetch(`${BASE}/api/jobs/${job.id}`)).json();
    console.log(job.status, (job.progress * 100).toFixed(1) + '%', job.stage);
  }
  if (job.status !== 'done') throw new Error(job.error ?? job.stage);

  const video = await fetch(`${BASE}/api/jobs/${job.id}/video`);
  return Buffer.from(await video.arrayBuffer());
}

await writeFile('video.mp4', await render('Explain how vaccines train the immune system'));

Job lifecycle

status moves forward through these values. Treat anything that is not terminal as "still working" rather than switching on each one — new stages may be added.

Creating a render

POST /api/jobs — JSON body, or multipart when attaching a document.

The response is the full job object with status: "queued". Unknown fields are rejected with 422 rather than ignored, so a typo in an option never silently produces the wrong video.

Screen formats

Aspect ratio is not just an encoder setting — the layout engine composes differently for each shape. Columns collapse into rows on a phone, flow diagrams run downwards instead of across, and the type grid changes so text is sized by whichever edge runs out first. The planner is also told the exact canvas dimensions, so it places elements correctly for that shape.

orientation is the simple control and overrides format. Use format when you need an exact ratio — widescreen (1920×1080), shorts (1080×1920), square, portrait_45, tablet. Explicit width/height override both.

Live from GET /api/assets/formats. The draft quality preset halves each dimension; standard and high render at the base size.

Style & hand

Everything under style is optional; anything omitted falls back to your saved defaults.

{
  "prompt": "How does a heat pump work?",
  "format": "mobile",
  "style": {
    "ink":        "#1b1b1b",
    "accent":     "#e5644d",
    "background": "#fbfbf7",
    "font":       "kalam",          // id from GET /api/assets/fonts
    "hand":       "hand:7628d3fd",  // or "default", or "none"
    "hand_scale": 1.2,
    "stroke_width": 3.0,
    "paper_texture": true
  }
}

Where drawings come from

An svg element resolves through a chain: the 51-icon built-in library on a confident match, then a keyword search across roughly 200,000 open line icons that need no account, then SVG the model authored itself, then a neutral placeholder with a job warning. The search is ranked toward true stroke collections — tracing a filled glyph only gives you its outline.

GET /api/assets/sources lists them and reports whether the lookup is on; DELETE /api/assets/sources/cache drops everything fetched so far. Only the concept word is sent — never the prompt or the source document — and web_assets: false in settings keeps every request local.

Named styles

GET /api/assets/styles returns eleven complete looks — chalkboard_white, chalkboard_color, chalkboard_black, whiteboard, modern_minimal, technical, sharpie, playful, editorial, illustrations, pencil — plus the two axes they are built from, which you can set independently:

Choosing the hand

Uploading a hand

The image must have a transparent background; an opaque one is rejected rather than accepted and then rendered as a rectangle covering the board. The one thing the file cannot tell us is where the pen tip is, so it is stored per asset as a fraction of the image and can be corrected afterwards.

curl -X POST http://127.0.0.1:8000/api/assets/hands \
  -F 'file=@hand.png' -F 'name=My hand' \
  -F 'tip_x=0.05' -F 'tip_y=0.03'

# move the tip later (the studio UI does this when you click the picture)
curl -X PATCH http://127.0.0.1:8000/api/assets/hands/7628d3fd \
  -H 'Content-Type: application/json' \
  -d '{"tip_x": 0.07, "tip_y": 0.04, "scale": 1.1}'

tip_x / tip_y are 0–1 fractions measured from the top-left of the image. That point is placed exactly on the ink.

Rendering from a document

To attach a PDF, Markdown or text file, send multipart instead of JSON. The options go in a payload field as a JSON string, and the file in file. A prompt alongside the document steers how it is used.

curl -X POST http://127.0.0.1:8000/api/jobs \
  -F 'payload={"prompt":"Summarise this for a general audience","target_seconds":120,"format":"square"}' \
  -F 'file=@paper.pdf'

Accepted: .pdf, .txt, .md, .rst — up to 25 MB. Text is extracted, de-hyphenated and truncated to 60,000 characters. A scanned PDF with no text layer fails at once with a clear message rather than producing an empty video.

Images

The language model never calls your image API. It writes a short visual query as an element's content — “honeybee on a lavender flower” — and the engine resolves that query before rendering. Results are cached by query, a failed lookup degrades to an icon instead of failing the render, and the model never holds your key.

Pointing it at your own API

curl -X PUT http://127.0.0.1:8000/api/settings \
  -H 'Content-Type: application/json' \
  -d '{
        "image_provider":      "custom",
        "image_custom_url":    "https://api.example.com/search?q={query}&limit=1",
        "image_custom_path":   "results[0].url",
        "image_custom_header": "X-Api-Key",
        "image_api_key":       "sk-..."
      }'

# verify it end to end — fetches one real image
curl -X POST 'http://127.0.0.1:8000/api/assets/images/test?query=a%20red%20apple'

Built-in alternatives: unsplash, pexels (search) and openai (generation). Or skip the API entirely and upload your own via POST /api/assets/media — those are matched by name and keyword.

Narration & music

Four ways to give a video a voice. All of them re-time the boards to the audio.

# attach a voice recording, with a transcript for exact timing
curl -X POST http://127.0.0.1:8000/api/jobs/$JOB/audio \
  -F 'file=@narration.m4a' -F 'transcript=@narration.srt'

# or a webcam take, shown as a circle in the bottom-right corner
curl -X POST http://127.0.0.1:8000/api/jobs/$JOB/presenter \
  -F 'file=@take.webm' -F 'shape=circle' -F 'position=bottom_right' -F 'size=0.30'

# then re-render to build it in
curl -X POST http://127.0.0.1:8000/api/jobs/$JOB/rerender \
  -H 'Content-Type: application/json' -d '{"revoice": true}'

Without a transcript the recording is split across scenes in proportion to how much each one says; with one, each scene starts when its words are actually spoken. Either way the response reports the duration so you can check it.

Music

Pass music_track (an id from GET /api/assets/music), music_gain in dB and music_duck. When there is narration the bed is side-chain compressed against it, so it drops out of the way of speech. Upload your own with POST /api/assets/music.

Editing a render

A finished render is not final. The timeline is a document, and every change is a small validated operation applied to it. Editing never renders — turning edits into a new mp4 is an explicit /rerender, so a burst of tweaks costs one render rather than one each.

# move an element, recolour it, and pin its timing — in one request
curl -X POST http://127.0.0.1:8000/api/jobs/$JOB/ops \
  -H 'Content-Type: application/json' \
  -d '{"ops": [
        {"op": "set_box",    "scene": "s02", "element": "s02-e03",
         "x": 0.12, "y": 0.30, "w": 0.50, "h": 0.14},
        {"op": "set_style",  "scene": "s02", "element": "s02-e03", "color": "accent"},
        {"op": "set_timing", "scene": "s02", "element": "s02-e03",
         "start": 1.2, "duration": 1.8}
      ]}'

The response lists what was applied and what was rejected, each in plain English, plus the new revision and the full timeline. A bad operation is reported and skipped, never fatal — the rest of the batch still applies.

Editing by chat

POST /api/jobs/{id}/chat with a message. The model is shown a compact summary of the timeline — ids, boxes, times, narration — and returns operations, which are validated and applied exactly like the ones above. It never rewrites the timeline directly.

curl -X POST http://127.0.0.1:8000/api/jobs/$JOB/chat \
  -H 'Content-Type: application/json' \
  -d '{"message": "make the heading in scene 2 bigger and cut the last scene"}'

# { "reply": "…", "applied": ["s02-e01: size ×1.6", "removed scene s05 …"],
#   "rejected": [], "revision": 3, "timeline": { … } }

POST /api/jobs/{id}/chat/image uploads an image into the conversation; pass scene to drop it straight onto a board. GET /api/jobs/{id}/chat reads the history, DELETE clears it.

Live frames

GET /api/jobs/{id}/frame?t=12.5 renders one frame of the current timeline. It is served from a browser kept warm per job, so scrubbing costs about 80 ms a frame rather than the two seconds a cold launch would. Any edit invalidates it automatically.

PUT /api/jobs/{id}/timeline replaces the whole document if you would rather generate it yourself.

Polling & progress

There are no webhooks. Poll GET /api/jobs/{id} every 2–4 seconds; the studio UI uses 1.4s. Each response carries a live stage string that is safe to show a user directly — during rendering it includes a frame counter and an ETA.

{
  "id": "6617b867295749c8b473dfb26a2ccc6d",
  "status": "rendering",
  "progress": 0.612,
  "stage": "rendering frame 457/909 (11.8 fps, ~38s left)",
  "title": "How Vaccines Work",
  "duration": 30.3,
  "scene_count": 4,
  "element_count": 13,
  "frames": 909,
  "preview_files": ["previews/s01.jpg", "previews/s02.jpg"],
  "warnings": [],
  "logs": [{ "t": 1788555056.38, "level": "info", "msg": "plan: 4 scenes via groq" }]
}

logs is the same stream the studio shows, capped at the last 400 lines. warnings collects soft problems — a repaired anchor, an icon that fell back to a placeholder, an image lookup that failed. None of them stop a render, and all of them are worth surfacing.

To stop a job: POST /api/jobs/{id}/cancel. Cancellation is cooperative — the pipeline checks between stages and between frames, so it takes effect within a second or two rather than instantly.

Fetching the video

Artefacts live under storage/jobs/{id}/ and are deleted with the job. DELETE /api/jobs/{id} refuses while a job is still running — cancel first.

Streaming scenes

A whiteboard video is watchable long before it is finished. The board for scene one is settled the moment its narration has been spoken — nothing in scene four can change it — so POST /api/v2/videos/stream hands each scene over as it is made instead of making the viewer wait for an encode. First scene in a few seconds, rather than the length of the whole render.

It takes the same body as /generate and returns text/event-stream. What travels is the timeline, not pixels: a few kilobytes of JSON per scene against megabytes of H.264, and the receiver draws it at its own resolution with the same renderer this app uses. There is a working player at /stream — its source is web/stream.js, written to be copied.

Three things a client has to get right

1. It is a POST, so EventSource cannot be used. Parse the events off fetch's body reader — the format is a name line, a data line, and a blank line.
2. Scene times are final when sent. Each scene is placed after the ones before it and nothing later moves it, so append rather than reconcile.
3. Drawings are sent once, with the first scene that uses them. Accumulate assets across events; do not replace them.

const res = await fetch('/api/v2/videos/stream', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', 'x-api-key': KEY },
  body: JSON.stringify({ prompt: 'Solve 2x + 6 = 20, showing every step' }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '', meta = null, scenes = [], assets = {};

for (;;) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  let cut;
  while ((cut = buffer.indexOf('\n\n')) !== -1) {
    const block = buffer.slice(0, cut);
    buffer = buffer.slice(cut + 2);

    const name = block.match(/^event: (.+)$/m)?.[1];
    const data = block.match(/^data: (.+)$/m)?.[1];
    if (!data) continue;
    const payload = JSON.parse(data);

    if (name === 'meta') meta = payload;
    if (name === 'scene') {
      Object.assign(assets, payload.assets);      // accumulate, never replace
      scenes.push(payload.scene);
      // hand it to the renderer and start playing — the rest is still coming
      await engine.load({ timeline: { ...meta, scenes }, assets,
                          hand: meta.hand, fonts: meta.fonts });
      new Audio(payload.audio.url).play();
    }
  }
}

Drawing it yourself

The renderer is served at /renderer/src/engine.js. It is a pure function of time — engine.seek(t) draws the board exactly as it stands at t, with no playback state of its own — which is what lets the encoder screenshot frames out of order, and equally lets you drive it from requestAnimationFrame against your own clock. Fonts, pictures and voice clips are all served from the same origin as the stream, so nothing needs CORS.

Voice clips are at GET /api/v2/videos/{id}/scenes/{scene_id}/audio and need no API key — an <audio> element cannot send a header, and requiring one would mean no client could ever play the sound. The URL carries the random video id, the same protection the finished mp4 has.

Turning a stream into a file

A streamed job is a finished render that was never encoded: it keeps its timeline, its per-scene clips and a mixed voice track, so it appears in the library badged streamed, opens in the editor, and can be encoded whenever you want one — POST /api/jobs/{id}/rerender with {"revoice": false}. The narration already exists, so that only draws and encodes; it does not pay for the voice a second time. Reach for /generate instead when you knew you wanted a file from the start.

The timeline

The timeline is the contract between the planner and the renderer, and it is a plain JSON document. You can fetch it, edit it, and it fully describes the video — every position, colour and millisecond. Coordinates are normalised 0–1, so the same timeline renders identically at 720p or 4K.

{
  "version": "1.0",
  "title": "How Vaccines Work",
  "width": 1920, "height": 1080, "fps": 30,
  "duration": 30.3,
  "style": { "ink": "#1b1b1b", "accent": "#e5644d", "hand": "default" },
  "scenes": [{
    "id": "s02",
    "title": "The Setup",
    "narration": "A vaccine shows the body a harmless piece of a pathogen.",
    "start": 4.07, "duration": 11.55,
    "narration_start": 1.28,        // the heading is written before anyone speaks
    "camera": { "zoom": 1.03, "center": { "x": 0.5, "y": 0.47 } },
    "transition": "erase",
    "elements": [{
      "id": "s02-e03",
      "type": "text",               // text | svg | shape | arrow | highlight | image
      "content": "harmless piece",
      "box": { "x": 0.077, "y": 0.42, "w": 0.868, "h": 0.126 },
      "draw": "write",              // write | fade | pop | wipe | instant
      "anchor": "harmless piece",   // verbatim narration phrase this lands on
      "start": 5.79, "duration": 1.25,
      "align": "left", "font_scale": 1.0, "emphasis": false
    }]
  }]
}

anchor is the interesting field. The model marks each drawing with a verbatim phrase from its own narration; the aligner converts that into an absolute time, and starts the stroke slightly before the word so the ink lands as it is spoken. Anchors that do not appear in the narration are repaired or dropped, and every repair shows up in warnings.

Because an anchor is tied to the audio, it also decides the order things are drawn in. order only sequences elements that have no anchor and breaks ties between elements anchored to the same phrase — an order that contradicts the anchors is ignored, and said so in warnings. The hand draws one thing at a time, so a scene with more ink in it than there are words to cover reports that too, naming the stroke that falls furthest behind.

Errors

Errors are always JSON with a detail field — a string, or a list of validation problems.

Endpoint reference

Live from the running server's OpenAPI schema. The interactive version is at /docs, and the raw schema at /openapi.json.

v2 REST API

Dynamo.sh exposes a second API surface at /api/v2/videos that follows the widely-used whiteboard-video REST contract, so an existing integration can point at a self-hosted instance by changing one base URL. Same paths, same parameter names, same defaults, same validation — including which combinations return 422.

curl -X POST http://127.0.0.1:8000/api/v2/videos/generate \
  -H 'x-api-key: YOUR_KEY' -H 'Content-Type: application/json' \
  -d '{
        "prompt": "Explain how a solar panel turns light into electricity",
        "timing": "2",
        "video_orientation": "vertical",
        "dynamo_video_engine": "dynamo_canvas",
        "canvas_style_variant": "technical",
        "background_track": "documentary",
        "pen_animation_style": "stylus",
        "scene_pacing": "fast",
        "watermark": true,
        "logo_position": "br"
      }'

# { "job_id": "...", "video_id": "...", "status": "queued",
#   "unsupported_parameters": [] }

What is implemented

All of it, with two exceptions. Every documented parameter is validated and acted on: engine and style variants, orientation, timing (including "auto"), pacing, pen style, colour, music, watermark and custom logo with corner placement, reference documents, custom images with their description/keep-original/no-animation flags, inserted video clips with use_ai_audio_at, reference images, own-narration audio or video with round-overlay and side-by-side layouts, split size, fit and framing focus, script-only mode, podcast mode, narration and on-screen-text languages, language variants, visibility and caller-supplied ids.

The two exceptions are reported back in unsupported_parameters on every response rather than silently ignored: own_narration_webcam_crop_x, which the contract itself deprecates in favour of own_narration_focus_position, and variant_visuals_mode, because every language variant here is a full independent render — there is no cheaper shared-visuals path to choose.

Differences worth knowing

• upload-file — the contract describes a presigned PUT. This instance has no object store, so it takes the bytes at step one and returns an upload_url that accepts the same object. A client following the three-step flow works unchanged; one that skips step two also works.
• Plan gating — there are no plans or tiers here, so nothing returns 403 for timing: "15" or for uploads.
• Status vocabulary — v2 reports queued / processing / completed / failed; the native API's finer stages stay available on /api/jobs/{id}.

Going to production

There is no authentication

The server binds to 127.0.0.1 and has no auth, because it is built to run locally. Anyone who can reach it can read your saved API keys' usage (not the keys — those are only ever returned masked), spend your model credits, and delete your renders. If you expose it, put it behind a reverse proxy that handles authentication, and keep WB_HOST on loopback so the proxy is the only way in.

Concurrency

Each job drives its own browser instance, which is the memory-hungry part. WB_MAX_CONCURRENT_JOBS (default 2) caps how many run at once; extra jobs queue. Raise it only if you have the RAM — budget roughly 500 MB per concurrent render.

Swapping the infrastructure

Retention

Nothing is cleaned up automatically. A 90-second 1080p render is roughly 15 MB, plus previews and a timeline. Delete jobs you no longer need — that removes the row and the whole job directory.