multistreaming: scenes/composition, installer build support, service updates
- multistreaming (new): RTMP ingest + multi-platform fan-out with pluggable providers (Twitch/YouTube/Kick/custom), zero-knowledge key vaults, Authelia OIDC auth, shared rooms with editor/streamer roles, single-use invites, per-account streaming grants, and scenes & composition (grid/PiP layouts, text/image overlays, per-output audio routing). - installer: support Dockerfile build in metadata (not just image) and RSA key generation for the Authelia OIDC JWKS. - authelia: add OIDC provider with portainer + multistreaming clients (public + PKCE). - services: remove allprox; add nginx-proxy-manager and portainer; update lldap; regenerate catalog.
This commit is contained in:
Ezequiel C.
parent
bb754cdd8c
commit
187379de4e
106 changed files with 21391 additions and 286 deletions
684
services/multistreaming/docs/SCENES.md
Normal file
684
services/multistreaming/docs/SCENES.md
Normal file
|
|
@ -0,0 +1,684 @@
|
|||
# SCENES — Scene & Composition Design Contract
|
||||
|
||||
Status: **authoritative spec** for Phase 3. Backend (`t2`) and frontend (`t3`)
|
||||
implementation tasks consume this document and must not make design decisions
|
||||
that contradict it.
|
||||
|
||||
- Scope: `services/multistreaming` (CommonJS backend in `src/`, React/TS/shadcn
|
||||
frontend in `web/src/`).
|
||||
- Constraint: no runtime tests. Verification is `node --check` (backend) and
|
||||
`npm run build` (web = `tsc -b && vite build`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Today a room is a passthrough router: each published feed is remuxed
|
||||
(`-c copy`) to every enabled output that routes that feed. Scenes add a second
|
||||
programming mode for a room:
|
||||
|
||||
1. A **scene** is a named, persisted composition for a room: a layout (grid or
|
||||
picture-in-picture) that places live feeds, plus text/image overlays.
|
||||
2. A scene has **scene outputs** (destinations) — one per platform account the
|
||||
composed program is pushed to — each with its own **audio routing**.
|
||||
3. Activating a scene switches the room from passthrough mode to composed mode.
|
||||
In composed mode the room pushes the composed program (re-encoded) instead of
|
||||
raw per-feed passthrough. Deactivating the scene restores passthrough.
|
||||
4. **Passthrough mode (default) is unchanged**: with no composed scene active,
|
||||
fan-out stays `-c copy` exactly as today.
|
||||
|
||||
Security invariants are unchanged and extended in §8.
|
||||
|
||||
---
|
||||
|
||||
## 2. Terminology
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| feed | A single OBS ingest publishing to `rtmp://<host>:1935/live/<streamKey>`. |
|
||||
| live feed | A feed currently publishing (tracked from RTMP publish events). |
|
||||
| output | Existing passthrough routing record (`feedId → accountId`). Unchanged. |
|
||||
| scene | A named composition definition for a room. |
|
||||
| scene output | A destination for the composed program (`sceneId → accountId`) with an audio mode. |
|
||||
| slot | A position in the layout, referenced by index; holds a `feedId` or `null`. |
|
||||
| overlay | A text or image element drawn on top of the composed video. |
|
||||
| program audio | The mixed audio of all live feeds participating in the scene. |
|
||||
| canvas | The composed video frame size (default 1920×1080). |
|
||||
|
||||
---
|
||||
|
||||
## 3. Persisted data model
|
||||
|
||||
`Store` gains three collections, exactly like the existing ones, normalized in
|
||||
`_normalize` and written atomically by `_save`:
|
||||
|
||||
```js
|
||||
// store.js state shape additions
|
||||
{
|
||||
scenes: [ Scene ],
|
||||
sceneOutputs: [ SceneOutput ],
|
||||
images: [ RoomImage ],
|
||||
}
|
||||
```
|
||||
|
||||
### 3.1 `Scene`
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "uuid",
|
||||
"roomId": "uuid",
|
||||
"ownerId": "uuid", // creator; informational only
|
||||
"name": "Main show", // trimmed, 1..64 chars
|
||||
"active": false, // at most one true per room
|
||||
"layout": {
|
||||
"type": "grid", // "grid" | "pip"
|
||||
"columns": 2, // grid ONLY: integer 2..4
|
||||
"slots": ["feedId", null] // Array<string|null>; see slot rules
|
||||
},
|
||||
"overlays": [
|
||||
// text overlay
|
||||
{ "id": "uuid", "kind": "text", "text": "LIVE", "x": 32, "y": 32,
|
||||
"fontSize": 48, "color": "#ffffff", "bold": true },
|
||||
// image overlay
|
||||
{ "id": "uuid", "kind": "image", "imageId": "uuid", "x": 16, "y": 844,
|
||||
"width": 200, "height": 112, "opacity": 1.0 }
|
||||
],
|
||||
"createdAt": 1700000000000,
|
||||
"updatedAt": 1700000000000
|
||||
}
|
||||
```
|
||||
|
||||
**Slot rules**
|
||||
|
||||
- `slots` is an array indexed by slot number. Each element is a `feedId` string
|
||||
(must be a feed in the same room) or `null` (empty).
|
||||
- `grid`: `slots.length` ∈ 1..6 (the cell order, row-major). `columns` ∈ 2..4.
|
||||
- `pip`: `slots.length` == 2 — index 0 is the **main** (fullscreen) feed, index 1
|
||||
is the **PiP** window feed.
|
||||
- Unknown `feedId`, duplicate `feedId`, or out-of-range slot length is rejected
|
||||
with HTTP 400 (see §6.2 validation).
|
||||
|
||||
**Layout is always valid JSON with no secrets.** A scene never stores stream keys.
|
||||
|
||||
### 3.2 `SceneOutput`
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "uuid",
|
||||
"roomId": "uuid",
|
||||
"sceneId": "uuid", // must be a scene in the same room
|
||||
"accountId": "uuid", // must be an account in the room (accountsInRoom)
|
||||
"enabled": true,
|
||||
"audio": { "mode": "program" } // | { mode: "silent" }
|
||||
// | { mode: "feed", "feedId": "uuid" }
|
||||
}
|
||||
```
|
||||
|
||||
- `audio.mode`:
|
||||
- `"program"` — the mixed program audio (§5.4).
|
||||
- `"silent"` — no audio stream (`-an`).
|
||||
- `"feed"` — one specific feed's audio (`feedId` required, feed in the same room).
|
||||
- Enabled scene outputs are the push destinations when the scene is active.
|
||||
|
||||
### 3.3 `RoomImage`
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "uuid",
|
||||
"roomId": "uuid",
|
||||
"name": "logo.png", // sanitized, 1..64 chars
|
||||
"mime": "image/png", // sniffed: image/png | image/jpeg | image/webp
|
||||
"size": 48231, // decoded bytes
|
||||
"path": "uploads/<uuid>.<ext>", // server-generated; never user input
|
||||
"createdAt": 1700000000000
|
||||
}
|
||||
```
|
||||
|
||||
- Files live under `<dataDir>/uploads/`. The stored basename matches
|
||||
`/^[a-f0-9-]{36}\.(png|jpg|webp)$/` (UUID + sniffed extension). The client
|
||||
never chooses the path.
|
||||
|
||||
### 3.4 Validation (server-side, authoritative)
|
||||
|
||||
- Scene `name`: trimmed string, 1..64 chars.
|
||||
- `layout.type`: `"grid"` or `"pip"` (else 400).
|
||||
- Grid: `columns` integer 2..4; `slots` array length 1..6.
|
||||
- PiP: `slots` array length exactly 2.
|
||||
- Every non-null slot `feedId` must exist in `store.listFeeds(roomId)`.
|
||||
- `overlays` array length ≤ 8.
|
||||
- text: `text` trimmed non-empty, ≤ 256 chars; `x` ∈ [0,1920) integer,
|
||||
`y` ∈ [0,1080) integer; `fontSize` ∈ 8..144 integer; `color` matches
|
||||
`/^#[0-9a-fA-F]{6}$/`; `bold` optional boolean.
|
||||
- image: `imageId` must be a `RoomImage` in the same room; `x`,`y`,`width`,
|
||||
`height` integers with `width`,`height` ∈ 16..1920/1080 respectively;
|
||||
`opacity` optional ∈ [0,1].
|
||||
- `sceneOutput`: `sceneId` in room; `accountId` in `store.accountsInRoom(roomId)`;
|
||||
`audio.mode` one of the three; `mode === "feed"` requires a valid room `feedId`.
|
||||
- Image upload: decoded size ≤ `SCENE_MAX_IMAGE_BYTES` (default 1 MiB); type
|
||||
sniffed from magic bytes (PNG `89 50 4E 47`, JPEG `FF D8 FF`, WebP
|
||||
`RIFF....WEBP`). Client-declared MIME is **not** trusted.
|
||||
|
||||
### 3.5 Pruning (deletion hooks)
|
||||
|
||||
- `removeFeed(id)` additionally: in every scene of the same room set each slot
|
||||
element equal to `id` to `null`; for every sceneOutput whose
|
||||
`audio.mode === "feed"` and `audio.feedId === id`, set `audio = { mode: "silent" }`.
|
||||
- `removeAccount(id)` additionally: delete sceneOutputs with `accountId === id`
|
||||
(mirrors existing output pruning).
|
||||
- `removeScene(id)`: delete sceneOutputs with `sceneId === id`. If it was the
|
||||
active scene, no scene is active; reconcile stops the scene fanout.
|
||||
- `removeImage(id)`: delete the file, the record, and every overlay item
|
||||
(`kind === "image"`) across the room's scenes that references `imageId`.
|
||||
- `_normalize`: add the three arrays; defensively strip scene slots referencing
|
||||
unknown feeds, sceneOutputs referencing unknown scenes/accounts, and overlay
|
||||
image items referencing unknown images; if more than one scene is `active`
|
||||
per room, keep the first and clear the rest.
|
||||
|
||||
---
|
||||
|
||||
## 4. Composition engine — `src/compose.js`
|
||||
|
||||
New module. It contains **all** FFmpeg knowledge and produces argument arrays;
|
||||
`fanout.js` stays lifecycle-only. Exports:
|
||||
|
||||
- `CANVAS` — `{ width, height, fps, videoBitrate, audioBitrate }` resolved from
|
||||
config (§9).
|
||||
- `escapeDrawtext(text)` and `escapeFilterPath(path)` — escaping helpers (§5.5).
|
||||
- `buildScenePlan({ scene, liveFeeds, getInputUrl, destinations, config })` →
|
||||
a plan object or `null` (null when zero live feeds, i.e. nothing to compose).
|
||||
|
||||
```js
|
||||
// buildScenePlan output (passthrough to fanout, never persisted)
|
||||
{
|
||||
inputs: [ { feedId, url } ], // live feeds in slot order
|
||||
outputs: [ { outputId, url, args, safeArgs } ] // one ffmpeg argv per destination
|
||||
}
|
||||
```
|
||||
|
||||
`fanout.js` spawns **one ffmpeg child per `outputs[]` entry**, each reading all
|
||||
`inputs[]` and applying its own audio `-map` (see §5.4). This mirrors the
|
||||
existing per-output child model and keeps restart/reconcile reuseable. It means
|
||||
video is re-encoded once per destination — an accepted, documented trade-off for
|
||||
a small self-hosted tool (per-output audio routing cannot be shared through a
|
||||
single `tee` muxer without identical audio).
|
||||
|
||||
### 4.1 Inputs
|
||||
|
||||
For a scene with `slots`, the live inputs are the slots' feeds that are
|
||||
currently live, **in slot order** (null/non-live slots skipped). Input index `i`
|
||||
(0-based) corresponds to `inputs[i]`.
|
||||
|
||||
Source URL for feed with `streamKey`: `rtmp://127.0.0.1:<rtmpPort>/live/<streamKey>`.
|
||||
Destination URLs come from `destinations` (resolved by `index.js` from grants,
|
||||
exactly like today) — **never stored**.
|
||||
|
||||
### 4.2 Layouts
|
||||
|
||||
Canvas = `SCENE_WIDTH` × `SCENE_HEIGHT` (default 1920×1080), fps = `SCENE_FPS`
|
||||
(default 30). `n` = number of live inputs.
|
||||
|
||||
Normalization chain for input `i` to a target cell `Wi×Hi`:
|
||||
|
||||
```
|
||||
[i:v] scale=Wi:Hi:force_original_aspect_ratio=decrease,
|
||||
pad=Wi:Hi:(ow-iw)/2:(oh-ih)/2,
|
||||
settb=AVTB, fps=30, format=yuv420p [v_i]
|
||||
```
|
||||
|
||||
(`fps` uses the canvas fps constant.)
|
||||
|
||||
**Grid, n > 1** — `c = min(columns, n)`, `rows = ceil(n / c)`,
|
||||
`cellW = 1920 / c`, `cellH = 1080 / rows`. For input `i`:
|
||||
`x_i = (i % c) * cellW`, `y_i = floor(i / c) * cellH`.
|
||||
|
||||
```
|
||||
[v_0][v_1]...[v_{n-1}] xstack=inputs=n:layout=x_0_y_0|x_1_y_1|...|x_{n-1}_y_{n-1}:fill=black [comp]
|
||||
```
|
||||
|
||||
**Grid, n == 1** — single fullscreen tile (no xstack): the normalization chain
|
||||
targets `1920x1080` and its label is `[comp]`.
|
||||
|
||||
**PiP, both slots live** (`m` = main input index, `p` = pip input index):
|
||||
|
||||
```
|
||||
[m:v] scale=1920:1080:force_original_aspect_ratio=decrease, pad=1920:1080:(ow-iw)/2:(oh-ih)/2, settb=AVTB, fps=30, format=yuv420p [main]
|
||||
[p:v] scale=480:270:force_original_aspect_ratio=decrease, pad=480:270:(ow-iw)/2:(oh-ih)/2, settb=AVTB, fps=30, format=yuv420p [pip]
|
||||
[main][pip] overlay=x=main_w-overlay_w-16:y=16:format=auto [comp]
|
||||
```
|
||||
|
||||
**PiP, only one slot live** — promote that feed to fullscreen (same chain as
|
||||
grid n==1) and label it `[comp]`.
|
||||
|
||||
**PiP, no slots live** — `buildScenePlan` returns `null`.
|
||||
|
||||
### 4.3 Overlays
|
||||
|
||||
Overlays are applied to `[comp]` in array order, chaining labels
|
||||
`[comp] → [ov_1] → [ov_2] → … → [vout]`. If there are no overlays,
|
||||
`[comp]` is `[vout]`.
|
||||
|
||||
**Text overlay k** (`prev` = previous label, `fontpath` = `SCENE_FONT` or
|
||||
`SCENE_FONT_BOLD` when `bold`):
|
||||
|
||||
```
|
||||
[prev] drawtext=text='<escapeDrawtext(text)>':x=<x>:y=<y>:fontsize=<fs>:
|
||||
fontcolor=<color>:fontfile='<escapeFilterPath(fontpath)>':expansion=none [ov_k]
|
||||
```
|
||||
|
||||
**Image overlay k**:
|
||||
|
||||
```
|
||||
movie=filename='<escapeFilterPath(absImagePath)>', scale=<w>:<h> [img_k];
|
||||
[prev][img_k] overlay=x=<x>:y=<y>:format=auto [ov_k]
|
||||
```
|
||||
|
||||
- `absImagePath` = `path.join(config.dataDir, image.path)`.
|
||||
- Image scale is exact `scale=w:h` (no aspect preservation — the editor controls
|
||||
both dimensions).
|
||||
|
||||
### 4.4 Audio routing
|
||||
|
||||
Program audio (built **only if** at least one destination uses `mode: "program"`
|
||||
and `n ≥ 1`):
|
||||
|
||||
```
|
||||
n == 1: [0:a] aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]
|
||||
n >= 2: [0:a][1:a]...[n-1:a] amix=inputs=n:duration=first:dropout_transition=2,
|
||||
aresample=48000, aformat=sample_fmts=fltp:channel_layouts=stereo [aout]
|
||||
```
|
||||
|
||||
**Assumption (documented):** every participating feed publishes H.264 video +
|
||||
AAC audio (OBS defaults). Video-only or audio-only feeds are out of scope for
|
||||
composition; the `-map <j>:a?` form still tolerates a missing audio stream for
|
||||
`mode: "feed"`.
|
||||
|
||||
Per-destination audio args:
|
||||
|
||||
| mode | live? | args |
|
||||
|------|-------|------|
|
||||
| `program` | `[aout]` exists | `-map [vout] -map [aout] -c:a aac -b:a <AB> -ar 48000` |
|
||||
| `program` | `[aout]` absent | `-map [vout] -an` |
|
||||
| `silent` | — | `-map [vout] -an` |
|
||||
| `feed` (feed live at input index `j`) | yes | `-map [vout] -map j:a? -c:a aac -b:a <AB> -ar 48000` |
|
||||
| `feed` (feed not live) | no | `-map [vout] -an` |
|
||||
|
||||
### 4.5 Full ffmpeg argv (per destination)
|
||||
|
||||
```
|
||||
ffmpeg -hide_banner -loglevel warning
|
||||
-i <input0> -i <input1> ... -i <input_{n-1}>
|
||||
-filter_complex <graph>
|
||||
-map [vout] <audio args from §4.4>
|
||||
-c:v libx264 -preset veryfast -tune zerolatency -pix_fmt yuv420p
|
||||
-g 60 -sc_threshold 0 -b:v <VB> -maxrate <VB> -bufsize <2*VB>
|
||||
< -c:a aac -b:a <AB> -ar 48000 | -an >
|
||||
-f flv <dest.url>
|
||||
```
|
||||
|
||||
- `VB` = `SCENE_VIDEO_BITRATE` (default `4500k`), `AB` = `SCENE_AUDIO_BITRATE`
|
||||
(default `160k`). All values are strings passed via a spawn **array** (no shell).
|
||||
- `safeArgs` = `args` with the final `dest.url` replaced by `redactUrl(dest.url)`
|
||||
(reuse the existing redaction). Only `safeArgs` is logged. The filter graph
|
||||
contains no destination keys; image/font paths are server-generated and not secret.
|
||||
|
||||
### 4.6 Escaping rules (exact)
|
||||
|
||||
`escapeDrawtext(s)` — FFmpeg filtergraph second-level quoting. In order:
|
||||
|
||||
1. Strip `\n`, `\r`, and NUL (replace with a single space).
|
||||
2. Replace `\` → `\\`.
|
||||
3. Replace `'` → `\'`.
|
||||
4. The caller wraps the result in single quotes and always passes
|
||||
`:expansion=none`, so `%`, `:`, `,` cannot expand or terminate the value.
|
||||
|
||||
`escapeFilterPath(p)` — same two-character escaping (steps 2–3) for paths, then
|
||||
wrap in single quotes.
|
||||
|
||||
Because spawn uses an argv array (never a shell) and drawtext text is
|
||||
single-quoted + `\`/`'`-escaped + `expansion=none` + length-capped, overlay text
|
||||
cannot inject FFmpeg arguments.
|
||||
|
||||
---
|
||||
|
||||
## 5. Fanout changes — `src/fanout.js` + `src/index.js`
|
||||
|
||||
### 5.1 Generic child engine
|
||||
|
||||
`fanout.js` becomes a generic unit-of-work engine keyed by a string `key`:
|
||||
|
||||
- `apply(key, { children })` — create the unit if absent, then reconcile
|
||||
children by `childKey` + a stable **signature** of the argv. A child whose
|
||||
signature changed is killed and respawned; removed children are killed; new
|
||||
children are spawned.
|
||||
- Signature for a child = `safeArgs.join(' ')` (already redacted; deterministic).
|
||||
- `stop(key)`, `stopAll()`, `status()`, `liveFeedIds()` unchanged in shape.
|
||||
- `status()` still maps `childKey` → `{ running, pid, restarts, startedAt, log }`.
|
||||
|
||||
Child lifecycle (spawn/kill/restart/backoff/log-capture/log-redaction) is
|
||||
identical to the current implementation; only the argv comes from a caller.
|
||||
|
||||
### 5.2 Two kinds of units
|
||||
|
||||
- **Passthrough unit**: `key = feedId`, `childKey = outputId`, argv = today's
|
||||
`-c copy` argv. Built by `index.js` from `destinationsFor(feed)`.
|
||||
- **Scene unit**: `key = "scene:" + sceneId`, `childKey = sceneOutputId`, argv =
|
||||
`buildScenePlan(...).outputs[i].args` with `safeArgs`.
|
||||
|
||||
### 5.3 Live-feed tracking
|
||||
|
||||
`index.js` keeps its own `liveFeeds` **Set<feedId>** (populated on `postPublish`,
|
||||
removed on `donePublish`), independent of fanout state, because in composed mode
|
||||
the passthrough unit for a feed is stopped but the feed is still live.
|
||||
|
||||
### 5.4 Reconciliation (single entry point `reconcileAll`)
|
||||
|
||||
Called on: feed publish/unpublish, scene create/update/delete/activate, scene
|
||||
output add/update/delete, grant start/stop, output add/update/delete. Replaces
|
||||
the current ad-hoc `refreshLive`.
|
||||
|
||||
```
|
||||
for each room:
|
||||
scene = store.findActiveScene(room.id)
|
||||
if scene:
|
||||
for feed in room.feeds: fanout.stop(feed.id) // suspend passthrough
|
||||
liveInSlots = scene.layout.slots.filter(isLive) // slot order
|
||||
if liveInSlots.length == 0:
|
||||
fanout.stop("scene:" + scene.id)
|
||||
else:
|
||||
destinations = enabledSceneOutputs(scene.id) // account granted? → pushUrl
|
||||
plan = compose.buildScenePlan({ scene, liveFeeds: liveInSlots, ...destinations })
|
||||
fanout.apply("scene:" + scene.id, { children: plan.outputs })
|
||||
else:
|
||||
fanout.stop("scene:" + scene.id) // ensure scene fanout gone
|
||||
for feed in room.feeds where live:
|
||||
fanout.apply(feed.id, { children: destinationsFor(feed) })
|
||||
```
|
||||
|
||||
`destinationsFor(feed)` is the existing resolver (enabled outputs routing the
|
||||
feed whose account has a live grant). `enabledSceneOutputs(scene.id)` is the
|
||||
scene analogue (enabled scene outputs whose account has a live grant).
|
||||
|
||||
**Semantics (documented):** while a scene is active, only that scene's outputs
|
||||
push; the room's per-feed passthrough outputs are suspended and resume when the
|
||||
scene is deactivated. This is a deliberate simplification of the OBS "one
|
||||
program at a time" model.
|
||||
|
||||
Live scene switches are safe: activating/deactivating/editing a scene, or a feed
|
||||
coming/going, simply re-runs `reconcileAll`, which reconciles each unit by argv
|
||||
signature — changed children restart, unchanged children are left running.
|
||||
|
||||
---
|
||||
|
||||
## 6. REST API
|
||||
|
||||
All routes are behind `authed`. Errors use the existing shape
|
||||
`{ error: string }`. Non-members get 404; insufficient role gets 403; invalid
|
||||
body gets 400 (mirrors the existing feeds/outputs endpoints).
|
||||
|
||||
### 6.1 Scenes
|
||||
|
||||
| Method | Path | Role | Body → 201/200 response |
|
||||
|--------|------|------|--------------------------|
|
||||
| GET | `/api/rooms/:roomId/scenes` | any member | `Scene[]` |
|
||||
| POST | `/api/rooms/:roomId/scenes` | owner/editor | `{ name, layout }` → `Scene` (201) |
|
||||
| GET | `/api/scenes/:id` | any member | `Scene` |
|
||||
| PUT | `/api/scenes/:id` | owner/editor | partial `{ name?, layout?, overlays?, active? }` → `Scene` |
|
||||
| POST | `/api/scenes/:id/activate` | owner/editor | — → `Scene` (sets active, deactivates others) |
|
||||
| DELETE | `/api/scenes/:id` | owner/editor | — → `{ ok: true }` (prunes sceneOutputs) |
|
||||
|
||||
- `PUT` with `active: true` deactivates the room's other scenes.
|
||||
- `POST activate` is a convenience; it must also `refreshLive()`/`reconcileAll()`.
|
||||
|
||||
### 6.2 Scene outputs (destinations)
|
||||
|
||||
| Method | Path | Role | Body → response |
|
||||
|--------|------|------|------------------|
|
||||
| GET | `/api/rooms/:roomId/scene-outputs` | any member | `SceneOutput[]` (with masked `account`) |
|
||||
| POST | `/api/rooms/:roomId/scene-outputs` | owner/editor | `{ sceneId, accountId, enabled?, audio }` → `SceneOutput` (201) |
|
||||
| PUT | `/api/scene-outputs/:id` | owner/editor | partial `{ accountId?, enabled?, audio? }` → `SceneOutput` |
|
||||
| DELETE | `/api/scene-outputs/:id` | owner/editor | — → `{ ok: true }` |
|
||||
|
||||
- GET responses embed `account: maskAccount(account, viewerId)` exactly like the
|
||||
existing outputs list, so editor/streamer never see another owner's
|
||||
`secretCiphertext`.
|
||||
|
||||
### 6.3 Images
|
||||
|
||||
| Method | Path | Role | Body → response |
|
||||
|--------|------|------|------------------|
|
||||
| GET | `/api/rooms/:roomId/images` | any member | `{ id, name, mime, size, url }[]` |
|
||||
| POST | `/api/rooms/:roomId/images` | owner/editor | `{ name, data }` (base64) → image meta (201) |
|
||||
| GET | `/api/images/:id/file` | any member | file bytes (correct Content-Type) |
|
||||
| DELETE | `/api/images/:id` | owner/editor | — → `{ ok: true }` (prunes overlay refs) |
|
||||
|
||||
- POST decodes base64, enforces `SCENE_MAX_IMAGE_BYTES`, sniffs magic bytes,
|
||||
writes `<dataDir>/uploads/<uuid>.<ext>`, returns meta with
|
||||
`url = "/api/images/<id>/file"`.
|
||||
- `GET /file` streams with `res.sendFile` after the room-membership check.
|
||||
|
||||
### 6.4 `/api/state` additions
|
||||
|
||||
Each room object in the existing `/api/state` response gains:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"...existing room keys...": "...",
|
||||
"activeSceneId": "uuid|null",
|
||||
"scenes": [ /* Scene[] */ ],
|
||||
"sceneOutputs": [
|
||||
{ "...SceneOutput...": "...", "account": { /* masked Account */ },
|
||||
"runtime": { "running": false, "restarts": 0, "log": [] } }
|
||||
],
|
||||
"images": [ { "id", "name", "mime", "size", "url" } ]
|
||||
}
|
||||
```
|
||||
|
||||
- `runtime` for scene outputs comes from `fanout.status()` keyed by the scene
|
||||
output id (mirrors existing output `runtime`).
|
||||
|
||||
---
|
||||
|
||||
## 7. Security invariants (summary)
|
||||
|
||||
Unchanged:
|
||||
|
||||
1. Server never persists plaintext stream keys — only `secretCiphertext` +
|
||||
`serverWrapped`; destination URLs are resolved in memory from `grants` at
|
||||
spawn time. Scenes/sceneOutputs/overlays contain no keys.
|
||||
2. FFmpeg logs redact destination URLs (`redactUrl`). Scene logs use `safeArgs`.
|
||||
3. `maskAccount` still hides `secretCiphertext` from every viewer except the
|
||||
owning user (editors/streamers never see others' keys).
|
||||
|
||||
New / extended:
|
||||
|
||||
4. Overlay text cannot inject FFmpeg args: spawn argv array (no shell) +
|
||||
`escapeDrawtext` + `expansion=none` + 256-char cap + control-char strip.
|
||||
5. Image overlays: ≤ 1 MiB decoded, magic-byte type sniffing, server-generated
|
||||
UUID filenames under `dataDir/uploads`, strict basename regex, path passed
|
||||
through `escapeFilterPath` (no user input reaches `movie`).
|
||||
6. Scene routes are role-gated (owner/editor write, streamer read-only, 404 for
|
||||
non-members), matching the existing feeds/outputs model.
|
||||
|
||||
---
|
||||
|
||||
## 8. Frontend contract
|
||||
|
||||
### 8.1 `lib/types.ts` additions
|
||||
|
||||
```ts
|
||||
export type LayoutType = "grid" | "pip"
|
||||
export interface SceneLayout {
|
||||
type: LayoutType
|
||||
columns?: number // grid only, 2..4
|
||||
slots: Array<string | null> // index = slot number; grid 1..6, pip length 2
|
||||
}
|
||||
export type OverlayItem =
|
||||
| { id: string; kind: "text"; text: string; x: number; y: number; fontSize: number; color: string; bold?: boolean }
|
||||
| { id: string; kind: "image"; imageId: string; x: number; y: number; width: number; height: number; opacity?: number }
|
||||
export type AudioRouting =
|
||||
| { mode: "program" }
|
||||
| { mode: "silent" }
|
||||
| { mode: "feed"; feedId: string }
|
||||
|
||||
export interface Scene {
|
||||
id: string
|
||||
roomId: string
|
||||
ownerId: string
|
||||
name: string
|
||||
active: boolean
|
||||
layout: SceneLayout
|
||||
overlays: OverlayItem[]
|
||||
createdAt: number
|
||||
updatedAt: number
|
||||
}
|
||||
|
||||
export interface SceneOutput {
|
||||
id: string
|
||||
roomId: string
|
||||
sceneId: string
|
||||
accountId: string
|
||||
enabled: boolean
|
||||
audio: AudioRouting
|
||||
account?: Account // masked by server
|
||||
runtime?: OutputRuntime
|
||||
}
|
||||
|
||||
export interface RoomImage {
|
||||
id: string
|
||||
name: string
|
||||
mime: string
|
||||
size: number
|
||||
url: string
|
||||
}
|
||||
|
||||
// Room gains:
|
||||
export interface Room {
|
||||
// ...existing keys...
|
||||
activeSceneId: string | null
|
||||
scenes: Scene[]
|
||||
sceneOutputs: SceneOutput[]
|
||||
images: RoomImage[]
|
||||
}
|
||||
```
|
||||
|
||||
### 8.2 `lib/api.ts` additions
|
||||
|
||||
```ts
|
||||
createScene: (roomId: string, body: { name: string; layout: SceneLayout }) => request<Scene>(`/api/rooms/${roomId}/scenes`, { method: "POST", body }),
|
||||
updateScene: (id: string, body: Partial<Pick<Scene, "name" | "layout" | "overlays" | "active">>) => request<Scene>(`/api/scenes/${id}`, { method: "PUT", body }),
|
||||
deleteScene: (id: string) => request<{ ok: boolean }>(`/api/scenes/${id}`, { method: "DELETE" }),
|
||||
activateScene: (id: string) => request<Scene>(`/api/scenes/${id}/activate`, { method: "POST" }),
|
||||
|
||||
createSceneOutput: (roomId: string, body: { sceneId: string; accountId: string; audio: AudioRouting; enabled?: boolean }) => request<SceneOutput>(`/api/rooms/${roomId}/scene-outputs`, { method: "POST", body }),
|
||||
updateSceneOutput: (id: string, body: Partial<Pick<SceneOutput, "accountId" | "enabled" | "audio">>) => request<SceneOutput>(`/api/scene-outputs/${id}`, { method: "PUT", body }),
|
||||
deleteSceneOutput: (id: string) => request<{ ok: boolean }>(`/api/scene-outputs/${id}`, { method: "DELETE" }),
|
||||
|
||||
uploadImage: (roomId: string, body: { name: string; data: string }) => request<RoomImage>(`/api/rooms/${roomId}/images`, { method: "POST", body }),
|
||||
deleteImage: (id: string) => request<{ ok: boolean }>(`/api/images/${id}`, { method: "DELETE" }),
|
||||
```
|
||||
|
||||
### 8.3 Component tree
|
||||
|
||||
New files under `web/src/components/`:
|
||||
|
||||
- `scene-panel.tsx` — `ScenePanel({ room, canEdit, onRefresh })`. A Card in the
|
||||
room center view: lists scenes (active badge), Activate, Edit, Delete
|
||||
(destructive confirms via `AlertDialog`), New scene button. Also lists the
|
||||
active scene's destinations (scene outputs) with their audio mode.
|
||||
- `scene-editor.tsx` — `SceneEditorDialog({ open, onOpenChange, scene, room, onSaved })`.
|
||||
Form built from `Field`/`FieldGroup`:
|
||||
- name `Field` (`Input`),
|
||||
- layout picker (`Select` grid/pip; grid columns `Select`; slot assignment:
|
||||
one feed `Select` per slot, options = room.feeds + "empty"),
|
||||
- overlay editor: list + add text/image; text form (`Input` text, x/y, fontSize,
|
||||
color `Input type="color"`, bold toggle); image form (image `Select`,
|
||||
x/y/width/height) — client-side validation matching §3.4,
|
||||
- audio routing editor: per scene output, `Select` mode (program/silent/feed;
|
||||
feed `Select` when mode=feed).
|
||||
- `image-upload.tsx` — `ImageUploader({ room, canEdit, onUploaded })`. File input;
|
||||
client-side pre-check (size ≤ 1 MiB, type ∈ png/jpeg/webp) before base64
|
||||
encoding and `api.uploadImage`.
|
||||
- `scene-sidebar.tsx` (or inline in `room-sidebar.tsx`) — active scene name +
|
||||
badge under the Outputs group.
|
||||
|
||||
Modified files:
|
||||
|
||||
- `room-view.tsx` — render `ScenePanel` above the existing Outputs card (or as a
|
||||
first tab); keep the passthrough Outputs card unchanged.
|
||||
- `room-sidebar.tsx` — show the active scene.
|
||||
- `App.tsx` — wire `SceneEditorDialog`/`ImageUploader` open-state like the
|
||||
existing dialogs; pass `canEdit = activeRoom.role !== "streamer"`.
|
||||
|
||||
### 8.4 State & data flow
|
||||
|
||||
- Reuse the existing 3-second `useApp` poll: `/api/state` now carries
|
||||
`scenes`/`sceneOutputs`/`images`, so no new polling is required.
|
||||
- Every mutation calls `api.*` then `onRefresh()` (the existing pattern).
|
||||
- Scene editor local state (name/layout/overlays/audio) is held in the dialog;
|
||||
it is assembled into the `PUT`/`POST` body on submit, then the editor closes
|
||||
and `onRefresh()` repaints.
|
||||
|
||||
### 8.5 Conventions (shadcn)
|
||||
|
||||
Use the existing primitives and patterns: `cn()` from `@/lib/utils`, lucide-react
|
||||
icons, `Field`/`FieldGroup`/`FieldLabel` for forms, `Dialog` for editors,
|
||||
`AlertDialog` for destructive confirms (delete scene/output/image, and overwrite
|
||||
warnings), `Select` for pickers, `Switch` for `enabled`, `Badge` for the active/
|
||||
live state, `toast` for success/error. No raw `confirm()` in new code (the
|
||||
existing output delete uses it, but new UI must use `AlertDialog`).
|
||||
|
||||
---
|
||||
|
||||
## 9. Config & image additions
|
||||
|
||||
`src/config.js` gains (all `intEnv`/string env with defaults):
|
||||
|
||||
| key | env | default |
|
||||
|-----|-----|---------|
|
||||
| `sceneWidth` | `SCENE_WIDTH` | 1920 |
|
||||
| `sceneHeight` | `SCENE_HEIGHT` | 1080 |
|
||||
| `sceneFps` | `SCENE_FPS` | 30 |
|
||||
| `sceneVideoBitrate` | `SCENE_VIDEO_BITRATE` | `4500k` |
|
||||
| `sceneAudioBitrate` | `SCENE_AUDIO_BITRATE` | `160k` |
|
||||
| `sceneFont` | `SCENE_FONT` | `/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf` |
|
||||
| `sceneFontBold` | `SCENE_FONT_BOLD` | `/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf` |
|
||||
| `sceneMaxImageBytes` | `SCENE_MAX_IMAGE_BYTES` | 1048576 |
|
||||
|
||||
`Dockerfile` must additionally `apk add --no-cache font-dejavu` so `drawtext` has
|
||||
a font (Composition requires a font; this is a hard runtime dependency of the
|
||||
text overlay feature). `ffmpeg` is already installed.
|
||||
|
||||
---
|
||||
|
||||
## 10. Contract coverage checklist
|
||||
|
||||
Downstream tasks must satisfy these; the spec is complete when each is
|
||||
implementable with no further design decisions.
|
||||
|
||||
**t2 (backend)** — data model (§3) persisted and exposed via `/api/state` (§6.4)
|
||||
without plaintext keys; routes (§6) with the role matrix (§6) and 404-on-non-member;
|
||||
`compose.js` filter graphs (§4) + per-output audio `-map` (§4.4) + `-c copy`
|
||||
passthrough otherwise (§5.4); live switch reconciliation (§5.4); URL redaction +
|
||||
text/image injection defenses (§4.5, §4.6, §7). Verify: `node --check` on
|
||||
`src/{index,fanout,server,store,compose,config,providers,grants,auth,oidc}.js`.
|
||||
|
||||
**t3 (frontend)** — types (§8.1), API client (§8.2), scene list/activate/edit UI
|
||||
with grid/PiP slot assignment (§8.3), text/image overlay editor with client-side
|
||||
size/type pre-check (§8.3), per-output audio routing UI (§8.3), shadcn
|
||||
conventions (§8.5), no key exposure (masked `account` in §6.2/§6.4). Verify:
|
||||
`cd services/multistreaming/web && npm run build`.
|
||||
|
||||
## 11. Decisions & trade-offs (locked)
|
||||
|
||||
1. **Per-destination re-encode**: one ffmpeg child per scene output, each
|
||||
re-encoding the composed video, so per-output audio `-map` can differ. Chosen
|
||||
over a shared `tee` muxer (which forces identical audio). Accepted cost.
|
||||
2. **One program at a time**: an active scene suspends the room's per-feed
|
||||
passthrough outputs; they resume on deactivation.
|
||||
3. **Live-slot layouts**: only live feeds are laid out; empty/non-live slots are
|
||||
skipped, and `n == 1` is promoted to fullscreen. Grid wraps
|
||||
`min(columns, n)` columns.
|
||||
4. **Audio**: composition assumes H.264 + AAC inputs (OBS defaults); `feed` mode
|
||||
uses `-map <j>:a?` for resilience; `program` mode requires ≥ 1 audio stream.
|
||||
5. **Images** are uploaded as base64 JSON (no new npm deps, no `multer`) and
|
||||
stored under `dataDir/uploads` with server-generated names.
|
||||
Loading…
Add table
Add a link
Reference in a new issue