- multistreaming (new): RTMP ingest + multi-platform fan-out with pluggable providers (Twitch/YouTube/Kick/custom), zero-knowledge key vaults, Authelia OIDC auth, shared rooms with editor/streamer roles, single-use invites, per-account streaming grants, and scenes & composition (grid/PiP layouts, text/image overlays, per-output audio routing). - installer: support Dockerfile build in metadata (not just image) and RSA key generation for the Authelia OIDC JWKS. - authelia: add OIDC provider with portainer + multistreaming clients (public + PKCE). - services: remove allprox; add nginx-proxy-manager and portainer; update lldap; regenerate catalog.
28 KiB
SCENES — Scene & Composition Design Contract
Status: authoritative spec for Phase 3. Backend (t2) and frontend (t3)
implementation tasks consume this document and must not make design decisions
that contradict it.
- Scope:
services/multistreaming(CommonJS backend insrc/, React/TS/shadcn frontend inweb/src/). - Constraint: no runtime tests. Verification is
node --check(backend) andnpm run build(web =tsc -b && vite build).
1. Overview
Today a room is a passthrough router: each published feed is remuxed
(-c copy) to every enabled output that routes that feed. Scenes add a second
programming mode for a room:
- A scene is a named, persisted composition for a room: a layout (grid or picture-in-picture) that places live feeds, plus text/image overlays.
- A scene has scene outputs (destinations) — one per platform account the composed program is pushed to — each with its own audio routing.
- Activating a scene switches the room from passthrough mode to composed mode. In composed mode the room pushes the composed program (re-encoded) instead of raw per-feed passthrough. Deactivating the scene restores passthrough.
- Passthrough mode (default) is unchanged: with no composed scene active,
fan-out stays
-c copyexactly as today.
Security invariants are unchanged and extended in §8.
2. Terminology
| Term | Meaning |
|---|---|
| feed | A single OBS ingest publishing to rtmp://<host>:1935/live/<streamKey>. |
| live feed | A feed currently publishing (tracked from RTMP publish events). |
| output | Existing passthrough routing record (feedId → accountId). Unchanged. |
| scene | A named composition definition for a room. |
| scene output | A destination for the composed program (sceneId → accountId) with an audio mode. |
| slot | A position in the layout, referenced by index; holds a feedId or null. |
| overlay | A text or image element drawn on top of the composed video. |
| program audio | The mixed audio of all live feeds participating in the scene. |
| canvas | The composed video frame size (default 1920×1080). |
3. Persisted data model
Store gains three collections, exactly like the existing ones, normalized in
_normalize and written atomically by _save:
// store.js state shape additions
{
scenes: [ Scene ],
sceneOutputs: [ SceneOutput ],
images: [ RoomImage ],
}
3.1 Scene
{
"id": "uuid",
"roomId": "uuid",
"ownerId": "uuid", // creator; informational only
"name": "Main show", // trimmed, 1..64 chars
"active": false, // at most one true per room
"layout": {
"type": "grid", // "grid" | "pip"
"columns": 2, // grid ONLY: integer 2..4
"slots": ["feedId", null] // Array<string|null>; see slot rules
},
"overlays": [
// text overlay
{ "id": "uuid", "kind": "text", "text": "LIVE", "x": 32, "y": 32,
"fontSize": 48, "color": "#ffffff", "bold": true },
// image overlay
{ "id": "uuid", "kind": "image", "imageId": "uuid", "x": 16, "y": 844,
"width": 200, "height": 112, "opacity": 1.0 }
],
"createdAt": 1700000000000,
"updatedAt": 1700000000000
}
Slot rules
slotsis an array indexed by slot number. Each element is afeedIdstring (must be a feed in the same room) ornull(empty).grid:slots.length∈ 1..6 (the cell order, row-major).columns∈ 2..4.pip:slots.length== 2 — index 0 is the main (fullscreen) feed, index 1 is the PiP window feed.- Unknown
feedId, duplicatefeedId, or out-of-range slot length is rejected with HTTP 400 (see §6.2 validation).
Layout is always valid JSON with no secrets. A scene never stores stream keys.
3.2 SceneOutput
{
"id": "uuid",
"roomId": "uuid",
"sceneId": "uuid", // must be a scene in the same room
"accountId": "uuid", // must be an account in the room (accountsInRoom)
"enabled": true,
"audio": { "mode": "program" } // | { mode: "silent" }
// | { mode: "feed", "feedId": "uuid" }
}
audio.mode:"program"— the mixed program audio (§5.4)."silent"— no audio stream (-an)."feed"— one specific feed's audio (feedIdrequired, feed in the same room).
- Enabled scene outputs are the push destinations when the scene is active.
3.3 RoomImage
{
"id": "uuid",
"roomId": "uuid",
"name": "logo.png", // sanitized, 1..64 chars
"mime": "image/png", // sniffed: image/png | image/jpeg | image/webp
"size": 48231, // decoded bytes
"path": "uploads/<uuid>.<ext>", // server-generated; never user input
"createdAt": 1700000000000
}
- Files live under
<dataDir>/uploads/. The stored basename matches/^[a-f0-9-]{36}\.(png|jpg|webp)$/(UUID + sniffed extension). The client never chooses the path.
3.4 Validation (server-side, authoritative)
- Scene
name: trimmed string, 1..64 chars. layout.type:"grid"or"pip"(else 400).- Grid:
columnsinteger 2..4;slotsarray length 1..6. - PiP:
slotsarray length exactly 2. - Every non-null slot
feedIdmust exist instore.listFeeds(roomId). overlaysarray length ≤ 8.- text:
texttrimmed non-empty, ≤ 256 chars;x∈ [0,1920) integer,y∈ [0,1080) integer;fontSize∈ 8..144 integer;colormatches/^#[0-9a-fA-F]{6}$/;boldoptional boolean. - image:
imageIdmust be aRoomImagein the same room;x,y,width,heightintegers withwidth,height∈ 16..1920/1080 respectively;opacityoptional ∈ [0,1].
- text:
sceneOutput:sceneIdin room;accountIdinstore.accountsInRoom(roomId);audio.modeone of the three;mode === "feed"requires a valid roomfeedId.- Image upload: decoded size ≤
SCENE_MAX_IMAGE_BYTES(default 1 MiB); type sniffed from magic bytes (PNG89 50 4E 47, JPEGFF D8 FF, WebPRIFF....WEBP). Client-declared MIME is not trusted.
3.5 Pruning (deletion hooks)
removeFeed(id)additionally: in every scene of the same room set each slot element equal toidtonull; for every sceneOutput whoseaudio.mode === "feed"andaudio.feedId === id, setaudio = { mode: "silent" }.removeAccount(id)additionally: delete sceneOutputs withaccountId === id(mirrors existing output pruning).removeScene(id): delete sceneOutputs withsceneId === id. If it was the active scene, no scene is active; reconcile stops the scene fanout.removeImage(id): delete the file, the record, and every overlay item (kind === "image") across the room's scenes that referencesimageId._normalize: add the three arrays; defensively strip scene slots referencing unknown feeds, sceneOutputs referencing unknown scenes/accounts, and overlay image items referencing unknown images; if more than one scene isactiveper room, keep the first and clear the rest.
4. Composition engine — src/compose.js
New module. It contains all FFmpeg knowledge and produces argument arrays;
fanout.js stays lifecycle-only. Exports:
CANVAS—{ width, height, fps, videoBitrate, audioBitrate }resolved from config (§9).escapeDrawtext(text)andescapeFilterPath(path)— escaping helpers (§5.5).buildScenePlan({ scene, liveFeeds, getInputUrl, destinations, config })→ a plan object ornull(null when zero live feeds, i.e. nothing to compose).
// buildScenePlan output (passthrough to fanout, never persisted)
{
inputs: [ { feedId, url } ], // live feeds in slot order
outputs: [ { outputId, url, args, safeArgs } ] // one ffmpeg argv per destination
}
fanout.js spawns one ffmpeg child per outputs[] entry, each reading all
inputs[] and applying its own audio -map (see §5.4). This mirrors the
existing per-output child model and keeps restart/reconcile reuseable. It means
video is re-encoded once per destination — an accepted, documented trade-off for
a small self-hosted tool (per-output audio routing cannot be shared through a
single tee muxer without identical audio).
4.1 Inputs
For a scene with slots, the live inputs are the slots' feeds that are
currently live, in slot order (null/non-live slots skipped). Input index i
(0-based) corresponds to inputs[i].
Source URL for feed with streamKey: rtmp://127.0.0.1:<rtmpPort>/live/<streamKey>.
Destination URLs come from destinations (resolved by index.js from grants,
exactly like today) — never stored.
4.2 Layouts
Canvas = SCENE_WIDTH × SCENE_HEIGHT (default 1920×1080), fps = SCENE_FPS
(default 30). n = number of live inputs.
Normalization chain for input i to a target cell Wi×Hi:
[i:v] scale=Wi:Hi:force_original_aspect_ratio=decrease,
pad=Wi:Hi:(ow-iw)/2:(oh-ih)/2,
settb=AVTB, fps=30, format=yuv420p [v_i]
(fps uses the canvas fps constant.)
Grid, n > 1 — c = min(columns, n), rows = ceil(n / c),
cellW = 1920 / c, cellH = 1080 / rows. For input i:
x_i = (i % c) * cellW, y_i = floor(i / c) * cellH.
[v_0][v_1]...[v_{n-1}] xstack=inputs=n:layout=x_0_y_0|x_1_y_1|...|x_{n-1}_y_{n-1}:fill=black [comp]
Grid, n == 1 — single fullscreen tile (no xstack): the normalization chain
targets 1920x1080 and its label is [comp].
PiP, both slots live (m = main input index, p = pip input index):
[m:v] scale=1920:1080:force_original_aspect_ratio=decrease, pad=1920:1080:(ow-iw)/2:(oh-ih)/2, settb=AVTB, fps=30, format=yuv420p [main]
[p:v] scale=480:270:force_original_aspect_ratio=decrease, pad=480:270:(ow-iw)/2:(oh-ih)/2, settb=AVTB, fps=30, format=yuv420p [pip]
[main][pip] overlay=x=main_w-overlay_w-16:y=16:format=auto [comp]
PiP, only one slot live — promote that feed to fullscreen (same chain as
grid n==1) and label it [comp].
PiP, no slots live — buildScenePlan returns null.
4.3 Overlays
Overlays are applied to [comp] in array order, chaining labels
[comp] → [ov_1] → [ov_2] → … → [vout]. If there are no overlays,
[comp] is [vout].
Text overlay k (prev = previous label, fontpath = SCENE_FONT or
SCENE_FONT_BOLD when bold):
[prev] drawtext=text='<escapeDrawtext(text)>':x=<x>:y=<y>:fontsize=<fs>:
fontcolor=<color>:fontfile='<escapeFilterPath(fontpath)>':expansion=none [ov_k]
Image overlay k:
movie=filename='<escapeFilterPath(absImagePath)>', scale=<w>:<h> [img_k];
[prev][img_k] overlay=x=<x>:y=<y>:format=auto [ov_k]
absImagePath=path.join(config.dataDir, image.path).- Image scale is exact
scale=w:h(no aspect preservation — the editor controls both dimensions).
4.4 Audio routing
Program audio (built only if at least one destination uses mode: "program"
and n ≥ 1):
n == 1: [0:a] aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]
n >= 2: [0:a][1:a]...[n-1:a] amix=inputs=n:duration=first:dropout_transition=2,
aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]
Assumption (documented): every participating feed publishes H.264 video +
AAC audio (OBS defaults). Video-only or audio-only feeds are out of scope for
composition; the -map <j>:a? form still tolerates a missing audio stream for
mode: "feed".
Per-destination audio args:
| mode | live? | args |
|---|---|---|
program |
[aout] exists |
-map [vout] -map [aout] -c:a aac -b:a <AB> -ar 48000 |
program |
[aout] absent |
-map [vout] -an |
silent |
— | -map [vout] -an |
feed (feed live at input index j) |
yes | -map [vout] -map j:a? -c:a aac -b:a <AB> -ar 48000 |
feed (feed not live) |
no | -map [vout] -an |
4.5 Full ffmpeg argv (per destination)
ffmpeg -hide_banner -loglevel warning
-i <input0> -i <input1> ... -i <input_{n-1}>
-filter_complex <graph>
-map [vout] <audio args from §4.4>
-c:v libx264 -preset veryfast -tune zerolatency -pix_fmt yuv420p
-g 60 -sc_threshold 0 -b:v <VB> -maxrate <VB> -bufsize <2*VB>
< -c:a aac -b:a <AB> -ar 48000 | -an >
-f flv <dest.url>
VB=SCENE_VIDEO_BITRATE(default4500k),AB=SCENE_AUDIO_BITRATE(default160k). All values are strings passed via a spawn array (no shell).safeArgs=argswith the finaldest.urlreplaced byredactUrl(dest.url)(reuse the existing redaction). OnlysafeArgsis logged. The filter graph contains no destination keys; image/font paths are server-generated and not secret.
4.6 Escaping rules (exact)
escapeDrawtext(s) — FFmpeg filtergraph second-level quoting. In order:
- Strip
\n,\r, and NUL (replace with a single space). - Replace
\→\\. - Replace
'→\'. - The caller wraps the result in single quotes and always passes
:expansion=none, so%,:,,cannot expand or terminate the value.
escapeFilterPath(p) — same two-character escaping (steps 2–3) for paths, then
wrap in single quotes.
Because spawn uses an argv array (never a shell) and drawtext text is
single-quoted + \/'-escaped + expansion=none + length-capped, overlay text
cannot inject FFmpeg arguments.
5. Fanout changes — src/fanout.js + src/index.js
5.1 Generic child engine
fanout.js becomes a generic unit-of-work engine keyed by a string key:
apply(key, { children })— create the unit if absent, then reconcile children bychildKey+ a stable signature of the argv. A child whose signature changed is killed and respawned; removed children are killed; new children are spawned.- Signature for a child =
safeArgs.join(' ')(already redacted; deterministic).
- Signature for a child =
stop(key),stopAll(),status(),liveFeedIds()unchanged in shape.status()still mapschildKey→{ running, pid, restarts, startedAt, log }.
Child lifecycle (spawn/kill/restart/backoff/log-capture/log-redaction) is identical to the current implementation; only the argv comes from a caller.
5.2 Two kinds of units
- Passthrough unit:
key = feedId,childKey = outputId, argv = today's-c copyargv. Built byindex.jsfromdestinationsFor(feed). - Scene unit:
key = "scene:" + sceneId,childKey = sceneOutputId, argv =buildScenePlan(...).outputs[i].argswithsafeArgs.
5.3 Live-feed tracking
index.js keeps its own liveFeeds Set (populated on postPublish,
removed on donePublish), independent of fanout state, because in composed mode
the passthrough unit for a feed is stopped but the feed is still live.
5.4 Reconciliation (single entry point reconcileAll)
Called on: feed publish/unpublish, scene create/update/delete/activate, scene
output add/update/delete, grant start/stop, output add/update/delete. Replaces
the current ad-hoc refreshLive.
for each room:
scene = store.findActiveScene(room.id)
if scene:
for feed in room.feeds: fanout.stop(feed.id) // suspend passthrough
liveInSlots = scene.layout.slots.filter(isLive) // slot order
if liveInSlots.length == 0:
fanout.stop("scene:" + scene.id)
else:
destinations = enabledSceneOutputs(scene.id) // account granted? → pushUrl
plan = compose.buildScenePlan({ scene, liveFeeds: liveInSlots, ...destinations })
fanout.apply("scene:" + scene.id, { children: plan.outputs })
else:
fanout.stop("scene:" + scene.id) // ensure scene fanout gone
for feed in room.feeds where live:
fanout.apply(feed.id, { children: destinationsFor(feed) })
destinationsFor(feed) is the existing resolver (enabled outputs routing the
feed whose account has a live grant). enabledSceneOutputs(scene.id) is the
scene analogue (enabled scene outputs whose account has a live grant).
Semantics (documented): while a scene is active, only that scene's outputs push; the room's per-feed passthrough outputs are suspended and resume when the scene is deactivated. This is a deliberate simplification of the OBS "one program at a time" model.
Live scene switches are safe: activating/deactivating/editing a scene, or a feed
coming/going, simply re-runs reconcileAll, which reconciles each unit by argv
signature — changed children restart, unchanged children are left running.
6. REST API
All routes are behind authed. Errors use the existing shape
{ error: string }. Non-members get 404; insufficient role gets 403; invalid
body gets 400 (mirrors the existing feeds/outputs endpoints).
6.1 Scenes
| Method | Path | Role | Body → 201/200 response |
|---|---|---|---|
| GET | /api/rooms/:roomId/scenes |
any member | Scene[] |
| POST | /api/rooms/:roomId/scenes |
owner/editor | { name, layout } → Scene (201) |
| GET | /api/scenes/:id |
any member | Scene |
| PUT | /api/scenes/:id |
owner/editor | partial { name?, layout?, overlays?, active? } → Scene |
| POST | /api/scenes/:id/activate |
owner/editor | — → Scene (sets active, deactivates others) |
| DELETE | /api/scenes/:id |
owner/editor | — → { ok: true } (prunes sceneOutputs) |
PUTwithactive: truedeactivates the room's other scenes.POST activateis a convenience; it must alsorefreshLive()/reconcileAll().
6.2 Scene outputs (destinations)
| Method | Path | Role | Body → response |
|---|---|---|---|
| GET | /api/rooms/:roomId/scene-outputs |
any member | SceneOutput[] (with masked account) |
| POST | /api/rooms/:roomId/scene-outputs |
owner/editor | { sceneId, accountId, enabled?, audio } → SceneOutput (201) |
| PUT | /api/scene-outputs/:id |
owner/editor | partial { accountId?, enabled?, audio? } → SceneOutput |
| DELETE | /api/scene-outputs/:id |
owner/editor | — → { ok: true } |
- GET responses embed
account: maskAccount(account, viewerId)exactly like the existing outputs list, so editor/streamer never see another owner'ssecretCiphertext.
6.3 Images
| Method | Path | Role | Body → response |
|---|---|---|---|
| GET | /api/rooms/:roomId/images |
any member | { id, name, mime, size, url }[] |
| POST | /api/rooms/:roomId/images |
owner/editor | { name, data } (base64) → image meta (201) |
| GET | /api/images/:id/file |
any member | file bytes (correct Content-Type) |
| DELETE | /api/images/:id |
owner/editor | — → { ok: true } (prunes overlay refs) |
- POST decodes base64, enforces
SCENE_MAX_IMAGE_BYTES, sniffs magic bytes, writes<dataDir>/uploads/<uuid>.<ext>, returns meta withurl = "/api/images/<id>/file". GET /filestreams withres.sendFileafter the room-membership check.
6.4 /api/state additions
Each room object in the existing /api/state response gains:
{
"...existing room keys...": "...",
"activeSceneId": "uuid|null",
"scenes": [ /* Scene[] */ ],
"sceneOutputs": [
{ "...SceneOutput...": "...", "account": { /* masked Account */ },
"runtime": { "running": false, "restarts": 0, "log": [] } }
],
"images": [ { "id", "name", "mime", "size", "url" } ]
}
runtimefor scene outputs comes fromfanout.status()keyed by the scene output id (mirrors existing outputruntime).
7. Security invariants (summary)
Unchanged:
- Server never persists plaintext stream keys — only
secretCiphertext+serverWrapped; destination URLs are resolved in memory fromgrantsat spawn time. Scenes/sceneOutputs/overlays contain no keys. - FFmpeg logs redact destination URLs (
redactUrl). Scene logs usesafeArgs. maskAccountstill hidessecretCiphertextfrom every viewer except the owning user (editors/streamers never see others' keys).
New / extended:
- Overlay text cannot inject FFmpeg args: spawn argv array (no shell) +
escapeDrawtext+expansion=none+ 256-char cap + control-char strip. - Image overlays: ≤ 1 MiB decoded, magic-byte type sniffing, server-generated
UUID filenames under
dataDir/uploads, strict basename regex, path passed throughescapeFilterPath(no user input reachesmovie). - Scene routes are role-gated (owner/editor write, streamer read-only, 404 for non-members), matching the existing feeds/outputs model.
8. Frontend contract
8.1 lib/types.ts additions
export type LayoutType = "grid" | "pip"
export interface SceneLayout {
type: LayoutType
columns?: number // grid only, 2..4
slots: Array<string | null> // index = slot number; grid 1..6, pip length 2
}
export type OverlayItem =
| { id: string; kind: "text"; text: string; x: number; y: number; fontSize: number; color: string; bold?: boolean }
| { id: string; kind: "image"; imageId: string; x: number; y: number; width: number; height: number; opacity?: number }
export type AudioRouting =
| { mode: "program" }
| { mode: "silent" }
| { mode: "feed"; feedId: string }
export interface Scene {
id: string
roomId: string
ownerId: string
name: string
active: boolean
layout: SceneLayout
overlays: OverlayItem[]
createdAt: number
updatedAt: number
}
export interface SceneOutput {
id: string
roomId: string
sceneId: string
accountId: string
enabled: boolean
audio: AudioRouting
account?: Account // masked by server
runtime?: OutputRuntime
}
export interface RoomImage {
id: string
name: string
mime: string
size: number
url: string
}
// Room gains:
export interface Room {
// ...existing keys...
activeSceneId: string | null
scenes: Scene[]
sceneOutputs: SceneOutput[]
images: RoomImage[]
}
8.2 lib/api.ts additions
createScene: (roomId: string, body: { name: string; layout: SceneLayout }) => request<Scene>(`/api/rooms/${roomId}/scenes`, { method: "POST", body }),
updateScene: (id: string, body: Partial<Pick<Scene, "name" | "layout" | "overlays" | "active">>) => request<Scene>(`/api/scenes/${id}`, { method: "PUT", body }),
deleteScene: (id: string) => request<{ ok: boolean }>(`/api/scenes/${id}`, { method: "DELETE" }),
activateScene: (id: string) => request<Scene>(`/api/scenes/${id}/activate`, { method: "POST" }),
createSceneOutput: (roomId: string, body: { sceneId: string; accountId: string; audio: AudioRouting; enabled?: boolean }) => request<SceneOutput>(`/api/rooms/${roomId}/scene-outputs`, { method: "POST", body }),
updateSceneOutput: (id: string, body: Partial<Pick<SceneOutput, "accountId" | "enabled" | "audio">>) => request<SceneOutput>(`/api/scene-outputs/${id}`, { method: "PUT", body }),
deleteSceneOutput: (id: string) => request<{ ok: boolean }>(`/api/scene-outputs/${id}`, { method: "DELETE" }),
uploadImage: (roomId: string, body: { name: string; data: string }) => request<RoomImage>(`/api/rooms/${roomId}/images`, { method: "POST", body }),
deleteImage: (id: string) => request<{ ok: boolean }>(`/api/images/${id}`, { method: "DELETE" }),
8.3 Component tree
New files under web/src/components/:
scene-panel.tsx—ScenePanel({ room, canEdit, onRefresh }). A Card in the room center view: lists scenes (active badge), Activate, Edit, Delete (destructive confirms viaAlertDialog), New scene button. Also lists the active scene's destinations (scene outputs) with their audio mode.scene-editor.tsx—SceneEditorDialog({ open, onOpenChange, scene, room, onSaved }). Form built fromField/FieldGroup:- name
Field(Input), - layout picker (
Selectgrid/pip; grid columnsSelect; slot assignment: one feedSelectper slot, options = room.feeds + "empty"), - overlay editor: list + add text/image; text form (
Inputtext, x/y, fontSize, colorInput type="color", bold toggle); image form (imageSelect, x/y/width/height) — client-side validation matching §3.4, - audio routing editor: per scene output,
Selectmode (program/silent/feed; feedSelectwhen mode=feed).
- name
image-upload.tsx—ImageUploader({ room, canEdit, onUploaded }). File input; client-side pre-check (size ≤ 1 MiB, type ∈ png/jpeg/webp) before base64 encoding andapi.uploadImage.scene-sidebar.tsx(or inline inroom-sidebar.tsx) — active scene name + badge under the Outputs group.
Modified files:
room-view.tsx— renderScenePanelabove the existing Outputs card (or as a first tab); keep the passthrough Outputs card unchanged.room-sidebar.tsx— show the active scene.App.tsx— wireSceneEditorDialog/ImageUploaderopen-state like the existing dialogs; passcanEdit = activeRoom.role !== "streamer".
8.4 State & data flow
- Reuse the existing 3-second
useApppoll:/api/statenow carriesscenes/sceneOutputs/images, so no new polling is required. - Every mutation calls
api.*thenonRefresh()(the existing pattern). - Scene editor local state (name/layout/overlays/audio) is held in the dialog;
it is assembled into the
PUT/POSTbody on submit, then the editor closes andonRefresh()repaints.
8.5 Conventions (shadcn)
Use the existing primitives and patterns: cn() from @/lib/utils, lucide-react
icons, Field/FieldGroup/FieldLabel for forms, Dialog for editors,
AlertDialog for destructive confirms (delete scene/output/image, and overwrite
warnings), Select for pickers, Switch for enabled, Badge for the active/
live state, toast for success/error. No raw confirm() in new code (the
existing output delete uses it, but new UI must use AlertDialog).
9. Config & image additions
src/config.js gains (all intEnv/string env with defaults):
| key | env | default |
|---|---|---|
sceneWidth |
SCENE_WIDTH |
1920 |
sceneHeight |
SCENE_HEIGHT |
1080 |
sceneFps |
SCENE_FPS |
30 |
sceneVideoBitrate |
SCENE_VIDEO_BITRATE |
4500k |
sceneAudioBitrate |
SCENE_AUDIO_BITRATE |
160k |
sceneFont |
SCENE_FONT |
/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf |
sceneFontBold |
SCENE_FONT_BOLD |
/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf |
sceneMaxImageBytes |
SCENE_MAX_IMAGE_BYTES |
1048576 |
Dockerfile must additionally apk add --no-cache font-dejavu so drawtext has
a font (Composition requires a font; this is a hard runtime dependency of the
text overlay feature). ffmpeg is already installed.
10. Contract coverage checklist
Downstream tasks must satisfy these; the spec is complete when each is implementable with no further design decisions.
t2 (backend) — data model (§3) persisted and exposed via /api/state (§6.4)
without plaintext keys; routes (§6) with the role matrix (§6) and 404-on-non-member;
compose.js filter graphs (§4) + per-output audio -map (§4.4) + -c copy
passthrough otherwise (§5.4); live switch reconciliation (§5.4); URL redaction +
text/image injection defenses (§4.5, §4.6, §7). Verify: node --check on
src/{index,fanout,server,store,compose,config,providers,grants,auth,oidc}.js.
t3 (frontend) — types (§8.1), API client (§8.2), scene list/activate/edit UI
with grid/PiP slot assignment (§8.3), text/image overlay editor with client-side
size/type pre-check (§8.3), per-output audio routing UI (§8.3), shadcn
conventions (§8.5), no key exposure (masked account in §6.2/§6.4). Verify:
cd services/multistreaming/web && npm run build.
11. Decisions & trade-offs (locked)
- Per-destination re-encode: one ffmpeg child per scene output, each
re-encoding the composed video, so per-output audio
-mapcan differ. Chosen over a sharedteemuxer (which forces identical audio). Accepted cost. - One program at a time: an active scene suspends the room's per-feed passthrough outputs; they resume on deactivation.
- Live-slot layouts: only live feeds are laid out; empty/non-live slots are
skipped, and
n == 1is promoted to fullscreen. Grid wrapsmin(columns, n)columns. - Audio: composition assumes H.264 + AAC inputs (OBS defaults);
feedmode uses-map <j>:a?for resilience;programmode requires ≥ 1 audio stream. - Images are uploaded as base64 JSON (no new npm deps, no
multer) and stored underdataDir/uploadswith server-generated names.