repo/services/multistreaming/docs/SCENES.md
Ezequiel C. 187379de4e multistreaming: scenes/composition, installer build support, service updates
- multistreaming (new): RTMP ingest + multi-platform fan-out with pluggable providers (Twitch/YouTube/Kick/custom), zero-knowledge key vaults, Authelia OIDC auth, shared rooms with editor/streamer roles, single-use invites, per-account streaming grants, and scenes & composition (grid/PiP layouts, text/image overlays, per-output audio routing).
- installer: support Dockerfile build in metadata (not just image) and RSA key generation for the Authelia OIDC JWKS.
- authelia: add OIDC provider with portainer + multistreaming clients (public + PKCE).
- services: remove allprox; add nginx-proxy-manager and portainer; update lldap; regenerate catalog.
2026-09-02 21:18:25 +02:00

28 KiB
Raw Permalink Blame History

SCENES — Scene & Composition Design Contract

Status: authoritative spec for Phase 3. Backend (t2) and frontend (t3) implementation tasks consume this document and must not make design decisions that contradict it.

  • Scope: services/multistreaming (CommonJS backend in src/, React/TS/shadcn frontend in web/src/).
  • Constraint: no runtime tests. Verification is node --check (backend) and npm run build (web = tsc -b && vite build).

1. Overview

Today a room is a passthrough router: each published feed is remuxed (-c copy) to every enabled output that routes that feed. Scenes add a second programming mode for a room:

  1. A scene is a named, persisted composition for a room: a layout (grid or picture-in-picture) that places live feeds, plus text/image overlays.
  2. A scene has scene outputs (destinations) — one per platform account the composed program is pushed to — each with its own audio routing.
  3. Activating a scene switches the room from passthrough mode to composed mode. In composed mode the room pushes the composed program (re-encoded) instead of raw per-feed passthrough. Deactivating the scene restores passthrough.
  4. Passthrough mode (default) is unchanged: with no composed scene active, fan-out stays -c copy exactly as today.

Security invariants are unchanged and extended in §8.


2. Terminology

Term Meaning
feed A single OBS ingest publishing to rtmp://<host>:1935/live/<streamKey>.
live feed A feed currently publishing (tracked from RTMP publish events).
output Existing passthrough routing record (feedId → accountId). Unchanged.
scene A named composition definition for a room.
scene output A destination for the composed program (sceneId → accountId) with an audio mode.
slot A position in the layout, referenced by index; holds a feedId or null.
overlay A text or image element drawn on top of the composed video.
program audio The mixed audio of all live feeds participating in the scene.
canvas The composed video frame size (default 1920×1080).

3. Persisted data model

Store gains three collections, exactly like the existing ones, normalized in _normalize and written atomically by _save:

// store.js state shape additions
{
  scenes:       [ Scene ],
  sceneOutputs: [ SceneOutput ],
  images:       [ RoomImage ],
}

3.1 Scene

{
  "id": "uuid",
  "roomId": "uuid",
  "ownerId": "uuid",          // creator; informational only
  "name": "Main show",        // trimmed, 1..64 chars
  "active": false,            // at most one true per room
  "layout": {
    "type": "grid",           // "grid" | "pip"
    "columns": 2,             // grid ONLY: integer 2..4
    "slots": ["feedId", null] // Array<string|null>; see slot rules
  },
  "overlays": [
    // text overlay
    { "id": "uuid", "kind": "text", "text": "LIVE", "x": 32, "y": 32,
      "fontSize": 48, "color": "#ffffff", "bold": true },
    // image overlay
    { "id": "uuid", "kind": "image", "imageId": "uuid", "x": 16, "y": 844,
      "width": 200, "height": 112, "opacity": 1.0 }
  ],
  "createdAt": 1700000000000,
  "updatedAt": 1700000000000
}

Slot rules

  • slots is an array indexed by slot number. Each element is a feedId string (must be a feed in the same room) or null (empty).
  • grid: slots.length ∈ 1..6 (the cell order, row-major). columns ∈ 2..4.
  • pip: slots.length == 2 — index 0 is the main (fullscreen) feed, index 1 is the PiP window feed.
  • Unknown feedId, duplicate feedId, or out-of-range slot length is rejected with HTTP 400 (see §6.2 validation).

Layout is always valid JSON with no secrets. A scene never stores stream keys.

3.2 SceneOutput

{
  "id": "uuid",
  "roomId": "uuid",
  "sceneId": "uuid",          // must be a scene in the same room
  "accountId": "uuid",        // must be an account in the room (accountsInRoom)
  "enabled": true,
  "audio": { "mode": "program" }            // | { mode: "silent" }
                                              // | { mode: "feed", "feedId": "uuid" }
}
  • audio.mode:
    • "program" — the mixed program audio (§5.4).
    • "silent" — no audio stream (-an).
    • "feed" — one specific feed's audio (feedId required, feed in the same room).
  • Enabled scene outputs are the push destinations when the scene is active.

3.3 RoomImage

{
  "id": "uuid",
  "roomId": "uuid",
  "name": "logo.png",       // sanitized, 1..64 chars
  "mime": "image/png",      // sniffed: image/png | image/jpeg | image/webp
  "size": 48231,            // decoded bytes
  "path": "uploads/<uuid>.<ext>", // server-generated; never user input
  "createdAt": 1700000000000
}
  • Files live under <dataDir>/uploads/. The stored basename matches /^[a-f0-9-]{36}\.(png|jpg|webp)$/ (UUID + sniffed extension). The client never chooses the path.

3.4 Validation (server-side, authoritative)

  • Scene name: trimmed string, 1..64 chars.
  • layout.type: "grid" or "pip" (else 400).
  • Grid: columns integer 2..4; slots array length 1..6.
  • PiP: slots array length exactly 2.
  • Every non-null slot feedId must exist in store.listFeeds(roomId).
  • overlays array length ≤ 8.
    • text: text trimmed non-empty, ≤ 256 chars; x ∈ [0,1920) integer, y ∈ [0,1080) integer; fontSize ∈ 8..144 integer; color matches /^#[0-9a-fA-F]{6}$/; bold optional boolean.
    • image: imageId must be a RoomImage in the same room; x,y,width, height integers with width,height ∈ 16..1920/1080 respectively; opacity optional ∈ [0,1].
  • sceneOutput: sceneId in room; accountId in store.accountsInRoom(roomId); audio.mode one of the three; mode === "feed" requires a valid room feedId.
  • Image upload: decoded size ≤ SCENE_MAX_IMAGE_BYTES (default 1 MiB); type sniffed from magic bytes (PNG 89 50 4E 47, JPEG FF D8 FF, WebP RIFF....WEBP). Client-declared MIME is not trusted.

3.5 Pruning (deletion hooks)

  • removeFeed(id) additionally: in every scene of the same room set each slot element equal to id to null; for every sceneOutput whose audio.mode === "feed" and audio.feedId === id, set audio = { mode: "silent" }.
  • removeAccount(id) additionally: delete sceneOutputs with accountId === id (mirrors existing output pruning).
  • removeScene(id): delete sceneOutputs with sceneId === id. If it was the active scene, no scene is active; reconcile stops the scene fanout.
  • removeImage(id): delete the file, the record, and every overlay item (kind === "image") across the room's scenes that references imageId.
  • _normalize: add the three arrays; defensively strip scene slots referencing unknown feeds, sceneOutputs referencing unknown scenes/accounts, and overlay image items referencing unknown images; if more than one scene is active per room, keep the first and clear the rest.

4. Composition engine — src/compose.js

New module. It contains all FFmpeg knowledge and produces argument arrays; fanout.js stays lifecycle-only. Exports:

  • CANVAS{ width, height, fps, videoBitrate, audioBitrate } resolved from config (§9).
  • escapeDrawtext(text) and escapeFilterPath(path) — escaping helpers (§5.5).
  • buildScenePlan({ scene, liveFeeds, getInputUrl, destinations, config }) → a plan object or null (null when zero live feeds, i.e. nothing to compose).
// buildScenePlan output (passthrough to fanout, never persisted)
{
  inputs:  [ { feedId, url } ],               // live feeds in slot order
  outputs: [ { outputId, url, args, safeArgs } ] // one ffmpeg argv per destination
}

fanout.js spawns one ffmpeg child per outputs[] entry, each reading all inputs[] and applying its own audio -map (see §5.4). This mirrors the existing per-output child model and keeps restart/reconcile reuseable. It means video is re-encoded once per destination — an accepted, documented trade-off for a small self-hosted tool (per-output audio routing cannot be shared through a single tee muxer without identical audio).

4.1 Inputs

For a scene with slots, the live inputs are the slots' feeds that are currently live, in slot order (null/non-live slots skipped). Input index i (0-based) corresponds to inputs[i].

Source URL for feed with streamKey: rtmp://127.0.0.1:<rtmpPort>/live/<streamKey>. Destination URLs come from destinations (resolved by index.js from grants, exactly like today) — never stored.

4.2 Layouts

Canvas = SCENE_WIDTH × SCENE_HEIGHT (default 1920×1080), fps = SCENE_FPS (default 30). n = number of live inputs.

Normalization chain for input i to a target cell Wi×Hi:

[i:v] scale=Wi:Hi:force_original_aspect_ratio=decrease,
      pad=Wi:Hi:(ow-iw)/2:(oh-ih)/2,
      settb=AVTB, fps=30, format=yuv420p [v_i]

(fps uses the canvas fps constant.)

Grid, n > 1c = min(columns, n), rows = ceil(n / c), cellW = 1920 / c, cellH = 1080 / rows. For input i: x_i = (i % c) * cellW, y_i = floor(i / c) * cellH.

[v_0][v_1]...[v_{n-1}] xstack=inputs=n:layout=x_0_y_0|x_1_y_1|...|x_{n-1}_y_{n-1}:fill=black [comp]

Grid, n == 1 — single fullscreen tile (no xstack): the normalization chain targets 1920x1080 and its label is [comp].

PiP, both slots live (m = main input index, p = pip input index):

[m:v] scale=1920:1080:force_original_aspect_ratio=decrease, pad=1920:1080:(ow-iw)/2:(oh-ih)/2, settb=AVTB, fps=30, format=yuv420p [main]
[p:v] scale=480:270:force_original_aspect_ratio=decrease,  pad=480:270:(ow-iw)/2:(oh-ih)/2,  settb=AVTB, fps=30, format=yuv420p [pip]
[main][pip] overlay=x=main_w-overlay_w-16:y=16:format=auto [comp]

PiP, only one slot live — promote that feed to fullscreen (same chain as grid n==1) and label it [comp].

PiP, no slots livebuildScenePlan returns null.

4.3 Overlays

Overlays are applied to [comp] in array order, chaining labels [comp] → [ov_1] → [ov_2] → … → [vout]. If there are no overlays, [comp] is [vout].

Text overlay k (prev = previous label, fontpath = SCENE_FONT or SCENE_FONT_BOLD when bold):

[prev] drawtext=text='<escapeDrawtext(text)>':x=<x>:y=<y>:fontsize=<fs>:
       fontcolor=<color>:fontfile='<escapeFilterPath(fontpath)>':expansion=none [ov_k]

Image overlay k:

movie=filename='<escapeFilterPath(absImagePath)>', scale=<w>:<h> [img_k];
[prev][img_k] overlay=x=<x>:y=<y>:format=auto [ov_k]
  • absImagePath = path.join(config.dataDir, image.path).
  • Image scale is exact scale=w:h (no aspect preservation — the editor controls both dimensions).

4.4 Audio routing

Program audio (built only if at least one destination uses mode: "program" and n ≥ 1):

n == 1: [0:a] aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]
n >= 2: [0:a][1:a]...[n-1:a] amix=inputs=n:duration=first:dropout_transition=2,
        aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]

Assumption (documented): every participating feed publishes H.264 video + AAC audio (OBS defaults). Video-only or audio-only feeds are out of scope for composition; the -map <j>:a? form still tolerates a missing audio stream for mode: "feed".

Per-destination audio args:

mode live? args
program [aout] exists -map [vout] -map [aout] -c:a aac -b:a <AB> -ar 48000
program [aout] absent -map [vout] -an
silent -map [vout] -an
feed (feed live at input index j) yes -map [vout] -map j:a? -c:a aac -b:a <AB> -ar 48000
feed (feed not live) no -map [vout] -an

4.5 Full ffmpeg argv (per destination)

ffmpeg -hide_banner -loglevel warning
       -i <input0> -i <input1> ... -i <input_{n-1}>
       -filter_complex <graph>
       -map [vout] <audio args from §4.4>
       -c:v libx264 -preset veryfast -tune zerolatency -pix_fmt yuv420p
       -g 60 -sc_threshold 0 -b:v <VB> -maxrate <VB> -bufsize <2*VB>
       < -c:a aac -b:a <AB> -ar 48000 | -an >
       -f flv <dest.url>
  • VB = SCENE_VIDEO_BITRATE (default 4500k), AB = SCENE_AUDIO_BITRATE (default 160k). All values are strings passed via a spawn array (no shell).
  • safeArgs = args with the final dest.url replaced by redactUrl(dest.url) (reuse the existing redaction). Only safeArgs is logged. The filter graph contains no destination keys; image/font paths are server-generated and not secret.

4.6 Escaping rules (exact)

escapeDrawtext(s) — FFmpeg filtergraph second-level quoting. In order:

  1. Strip \n, \r, and NUL (replace with a single space).
  2. Replace \\\.
  3. Replace '\'.
  4. The caller wraps the result in single quotes and always passes :expansion=none, so %, :, , cannot expand or terminate the value.

escapeFilterPath(p) — same two-character escaping (steps 23) for paths, then wrap in single quotes.

Because spawn uses an argv array (never a shell) and drawtext text is single-quoted + \/'-escaped + expansion=none + length-capped, overlay text cannot inject FFmpeg arguments.


5. Fanout changes — src/fanout.js + src/index.js

5.1 Generic child engine

fanout.js becomes a generic unit-of-work engine keyed by a string key:

  • apply(key, { children }) — create the unit if absent, then reconcile children by childKey + a stable signature of the argv. A child whose signature changed is killed and respawned; removed children are killed; new children are spawned.
    • Signature for a child = safeArgs.join(' ') (already redacted; deterministic).
  • stop(key), stopAll(), status(), liveFeedIds() unchanged in shape.
  • status() still maps childKey{ running, pid, restarts, startedAt, log }.

Child lifecycle (spawn/kill/restart/backoff/log-capture/log-redaction) is identical to the current implementation; only the argv comes from a caller.

5.2 Two kinds of units

  • Passthrough unit: key = feedId, childKey = outputId, argv = today's -c copy argv. Built by index.js from destinationsFor(feed).
  • Scene unit: key = "scene:" + sceneId, childKey = sceneOutputId, argv = buildScenePlan(...).outputs[i].args with safeArgs.

5.3 Live-feed tracking

index.js keeps its own liveFeeds Set (populated on postPublish, removed on donePublish), independent of fanout state, because in composed mode the passthrough unit for a feed is stopped but the feed is still live.

5.4 Reconciliation (single entry point reconcileAll)

Called on: feed publish/unpublish, scene create/update/delete/activate, scene output add/update/delete, grant start/stop, output add/update/delete. Replaces the current ad-hoc refreshLive.

for each room:
  scene = store.findActiveScene(room.id)
  if scene:
    for feed in room.feeds: fanout.stop(feed.id)      // suspend passthrough
    liveInSlots = scene.layout.slots.filter(isLive)    // slot order
    if liveInSlots.length == 0:
      fanout.stop("scene:" + scene.id)
    else:
      destinations = enabledSceneOutputs(scene.id)      // account granted? → pushUrl
      plan = compose.buildScenePlan({ scene, liveFeeds: liveInSlots, ...destinations })
      fanout.apply("scene:" + scene.id, { children: plan.outputs })
  else:
    fanout.stop("scene:" + scene.id)                    // ensure scene fanout gone
    for feed in room.feeds where live:
      fanout.apply(feed.id, { children: destinationsFor(feed) })

destinationsFor(feed) is the existing resolver (enabled outputs routing the feed whose account has a live grant). enabledSceneOutputs(scene.id) is the scene analogue (enabled scene outputs whose account has a live grant).

Semantics (documented): while a scene is active, only that scene's outputs push; the room's per-feed passthrough outputs are suspended and resume when the scene is deactivated. This is a deliberate simplification of the OBS "one program at a time" model.

Live scene switches are safe: activating/deactivating/editing a scene, or a feed coming/going, simply re-runs reconcileAll, which reconciles each unit by argv signature — changed children restart, unchanged children are left running.


6. REST API

All routes are behind authed. Errors use the existing shape { error: string }. Non-members get 404; insufficient role gets 403; invalid body gets 400 (mirrors the existing feeds/outputs endpoints).

6.1 Scenes

Method Path Role Body → 201/200 response
GET /api/rooms/:roomId/scenes any member Scene[]
POST /api/rooms/:roomId/scenes owner/editor { name, layout }Scene (201)
GET /api/scenes/:id any member Scene
PUT /api/scenes/:id owner/editor partial { name?, layout?, overlays?, active? }Scene
POST /api/scenes/:id/activate owner/editor — → Scene (sets active, deactivates others)
DELETE /api/scenes/:id owner/editor — → { ok: true } (prunes sceneOutputs)
  • PUT with active: true deactivates the room's other scenes.
  • POST activate is a convenience; it must also refreshLive()/reconcileAll().

6.2 Scene outputs (destinations)

Method Path Role Body → response
GET /api/rooms/:roomId/scene-outputs any member SceneOutput[] (with masked account)
POST /api/rooms/:roomId/scene-outputs owner/editor { sceneId, accountId, enabled?, audio }SceneOutput (201)
PUT /api/scene-outputs/:id owner/editor partial { accountId?, enabled?, audio? }SceneOutput
DELETE /api/scene-outputs/:id owner/editor — → { ok: true }
  • GET responses embed account: maskAccount(account, viewerId) exactly like the existing outputs list, so editor/streamer never see another owner's secretCiphertext.

6.3 Images

Method Path Role Body → response
GET /api/rooms/:roomId/images any member { id, name, mime, size, url }[]
POST /api/rooms/:roomId/images owner/editor { name, data } (base64) → image meta (201)
GET /api/images/:id/file any member file bytes (correct Content-Type)
DELETE /api/images/:id owner/editor — → { ok: true } (prunes overlay refs)
  • POST decodes base64, enforces SCENE_MAX_IMAGE_BYTES, sniffs magic bytes, writes <dataDir>/uploads/<uuid>.<ext>, returns meta with url = "/api/images/<id>/file".
  • GET /file streams with res.sendFile after the room-membership check.

6.4 /api/state additions

Each room object in the existing /api/state response gains:

{
  "...existing room keys...": "...",
  "activeSceneId": "uuid|null",
  "scenes": [ /* Scene[] */ ],
  "sceneOutputs": [
    { "...SceneOutput...": "...", "account": { /* masked Account */ },
      "runtime": { "running": false, "restarts": 0, "log": [] } }
  ],
  "images": [ { "id", "name", "mime", "size", "url" } ]
}
  • runtime for scene outputs comes from fanout.status() keyed by the scene output id (mirrors existing output runtime).

7. Security invariants (summary)

Unchanged:

  1. Server never persists plaintext stream keys — only secretCiphertext + serverWrapped; destination URLs are resolved in memory from grants at spawn time. Scenes/sceneOutputs/overlays contain no keys.
  2. FFmpeg logs redact destination URLs (redactUrl). Scene logs use safeArgs.
  3. maskAccount still hides secretCiphertext from every viewer except the owning user (editors/streamers never see others' keys).

New / extended:

  1. Overlay text cannot inject FFmpeg args: spawn argv array (no shell) + escapeDrawtext + expansion=none + 256-char cap + control-char strip.
  2. Image overlays: ≤ 1 MiB decoded, magic-byte type sniffing, server-generated UUID filenames under dataDir/uploads, strict basename regex, path passed through escapeFilterPath (no user input reaches movie).
  3. Scene routes are role-gated (owner/editor write, streamer read-only, 404 for non-members), matching the existing feeds/outputs model.

8. Frontend contract

8.1 lib/types.ts additions

export type LayoutType = "grid" | "pip"
export interface SceneLayout {
  type: LayoutType
  columns?: number            // grid only, 2..4
  slots: Array<string | null> // index = slot number; grid 1..6, pip length 2
}
export type OverlayItem =
  | { id: string; kind: "text"; text: string; x: number; y: number; fontSize: number; color: string; bold?: boolean }
  | { id: string; kind: "image"; imageId: string; x: number; y: number; width: number; height: number; opacity?: number }
export type AudioRouting =
  | { mode: "program" }
  | { mode: "silent" }
  | { mode: "feed"; feedId: string }

export interface Scene {
  id: string
  roomId: string
  ownerId: string
  name: string
  active: boolean
  layout: SceneLayout
  overlays: OverlayItem[]
  createdAt: number
  updatedAt: number
}

export interface SceneOutput {
  id: string
  roomId: string
  sceneId: string
  accountId: string
  enabled: boolean
  audio: AudioRouting
  account?: Account            // masked by server
  runtime?: OutputRuntime
}

export interface RoomImage {
  id: string
  name: string
  mime: string
  size: number
  url: string
}

// Room gains:
export interface Room {
  // ...existing keys...
  activeSceneId: string | null
  scenes: Scene[]
  sceneOutputs: SceneOutput[]
  images: RoomImage[]
}

8.2 lib/api.ts additions

createScene: (roomId: string, body: { name: string; layout: SceneLayout }) => request<Scene>(`/api/rooms/${roomId}/scenes`, { method: "POST", body }),
updateScene: (id: string, body: Partial<Pick<Scene, "name" | "layout" | "overlays" | "active">>) => request<Scene>(`/api/scenes/${id}`, { method: "PUT", body }),
deleteScene: (id: string) => request<{ ok: boolean }>(`/api/scenes/${id}`, { method: "DELETE" }),
activateScene: (id: string) => request<Scene>(`/api/scenes/${id}/activate`, { method: "POST" }),

createSceneOutput: (roomId: string, body: { sceneId: string; accountId: string; audio: AudioRouting; enabled?: boolean }) => request<SceneOutput>(`/api/rooms/${roomId}/scene-outputs`, { method: "POST", body }),
updateSceneOutput: (id: string, body: Partial<Pick<SceneOutput, "accountId" | "enabled" | "audio">>) => request<SceneOutput>(`/api/scene-outputs/${id}`, { method: "PUT", body }),
deleteSceneOutput: (id: string) => request<{ ok: boolean }>(`/api/scene-outputs/${id}`, { method: "DELETE" }),

uploadImage: (roomId: string, body: { name: string; data: string }) => request<RoomImage>(`/api/rooms/${roomId}/images`, { method: "POST", body }),
deleteImage: (id: string) => request<{ ok: boolean }>(`/api/images/${id}`, { method: "DELETE" }),

8.3 Component tree

New files under web/src/components/:

  • scene-panel.tsxScenePanel({ room, canEdit, onRefresh }). A Card in the room center view: lists scenes (active badge), Activate, Edit, Delete (destructive confirms via AlertDialog), New scene button. Also lists the active scene's destinations (scene outputs) with their audio mode.
  • scene-editor.tsxSceneEditorDialog({ open, onOpenChange, scene, room, onSaved }). Form built from Field/FieldGroup:
    • name Field (Input),
    • layout picker (Select grid/pip; grid columns Select; slot assignment: one feed Select per slot, options = room.feeds + "empty"),
    • overlay editor: list + add text/image; text form (Input text, x/y, fontSize, color Input type="color", bold toggle); image form (image Select, x/y/width/height) — client-side validation matching §3.4,
    • audio routing editor: per scene output, Select mode (program/silent/feed; feed Select when mode=feed).
  • image-upload.tsxImageUploader({ room, canEdit, onUploaded }). File input; client-side pre-check (size ≤ 1 MiB, type ∈ png/jpeg/webp) before base64 encoding and api.uploadImage.
  • scene-sidebar.tsx (or inline in room-sidebar.tsx) — active scene name + badge under the Outputs group.

Modified files:

  • room-view.tsx — render ScenePanel above the existing Outputs card (or as a first tab); keep the passthrough Outputs card unchanged.
  • room-sidebar.tsx — show the active scene.
  • App.tsx — wire SceneEditorDialog/ImageUploader open-state like the existing dialogs; pass canEdit = activeRoom.role !== "streamer".

8.4 State & data flow

  • Reuse the existing 3-second useApp poll: /api/state now carries scenes/sceneOutputs/images, so no new polling is required.
  • Every mutation calls api.* then onRefresh() (the existing pattern).
  • Scene editor local state (name/layout/overlays/audio) is held in the dialog; it is assembled into the PUT/POST body on submit, then the editor closes and onRefresh() repaints.

8.5 Conventions (shadcn)

Use the existing primitives and patterns: cn() from @/lib/utils, lucide-react icons, Field/FieldGroup/FieldLabel for forms, Dialog for editors, AlertDialog for destructive confirms (delete scene/output/image, and overwrite warnings), Select for pickers, Switch for enabled, Badge for the active/ live state, toast for success/error. No raw confirm() in new code (the existing output delete uses it, but new UI must use AlertDialog).


9. Config & image additions

src/config.js gains (all intEnv/string env with defaults):

key env default
sceneWidth SCENE_WIDTH 1920
sceneHeight SCENE_HEIGHT 1080
sceneFps SCENE_FPS 30
sceneVideoBitrate SCENE_VIDEO_BITRATE 4500k
sceneAudioBitrate SCENE_AUDIO_BITRATE 160k
sceneFont SCENE_FONT /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf
sceneFontBold SCENE_FONT_BOLD /usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf
sceneMaxImageBytes SCENE_MAX_IMAGE_BYTES 1048576

Dockerfile must additionally apk add --no-cache font-dejavu so drawtext has a font (Composition requires a font; this is a hard runtime dependency of the text overlay feature). ffmpeg is already installed.


10. Contract coverage checklist

Downstream tasks must satisfy these; the spec is complete when each is implementable with no further design decisions.

t2 (backend) — data model (§3) persisted and exposed via /api/state (§6.4) without plaintext keys; routes (§6) with the role matrix (§6) and 404-on-non-member; compose.js filter graphs (§4) + per-output audio -map (§4.4) + -c copy passthrough otherwise (§5.4); live switch reconciliation (§5.4); URL redaction + text/image injection defenses (§4.5, §4.6, §7). Verify: node --check on src/{index,fanout,server,store,compose,config,providers,grants,auth,oidc}.js.

t3 (frontend) — types (§8.1), API client (§8.2), scene list/activate/edit UI with grid/PiP slot assignment (§8.3), text/image overlay editor with client-side size/type pre-check (§8.3), per-output audio routing UI (§8.3), shadcn conventions (§8.5), no key exposure (masked account in §6.2/§6.4). Verify: cd services/multistreaming/web && npm run build.

11. Decisions & trade-offs (locked)

  1. Per-destination re-encode: one ffmpeg child per scene output, each re-encoding the composed video, so per-output audio -map can differ. Chosen over a shared tee muxer (which forces identical audio). Accepted cost.
  2. One program at a time: an active scene suspends the room's per-feed passthrough outputs; they resume on deactivation.
  3. Live-slot layouts: only live feeds are laid out; empty/non-live slots are skipped, and n == 1 is promoted to fullscreen. Grid wraps min(columns, n) columns.
  4. Audio: composition assumes H.264 + AAC inputs (OBS defaults); feed mode uses -map <j>:a? for resilience; program mode requires ≥ 1 audio stream.
  5. Images are uploaded as base64 JSON (no new npm deps, no multer) and stored under dataDir/uploads with server-generated names.