a lot
This commit is contained in:
@@ -20,84 +20,114 @@ nix develop
|
||||
|
||||
No tests exist yet.
|
||||
|
||||
## Workspace layout
|
||||
|
||||
```
|
||||
crates/
|
||||
server/ — main binary (RTMP ingestion, HTTP API, WebRTC)
|
||||
entity/ — SeaORM entity definitions
|
||||
migration/ — SeaORM migrations
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
This is an RTMP-to-WHEP bridge: it accepts an RTMP video publish stream and re-streams it to browsers via WebRTC (using the WHEP signaling protocol).
|
||||
This is an RTMP-to-WHEP bridge: accepts an RTMP video publish stream and re-streams it to browsers via WebRTC (WHEP signaling protocol).
|
||||
|
||||
**Signal flow:**
|
||||
|
||||
```
|
||||
OBS/encoder → RTMP (port 8123) → H264Parser → async_broadcast channel
|
||||
↓
|
||||
Browser ← WebRTC/UDP ← str0m Rtc ← WHEP HTTP (port 5000)
|
||||
Browser ← WebRTC/UDP ← str0m Rtc ← WHEP HTTP (port 3000)
|
||||
```
|
||||
|
||||
### RTMP ingestion (port 8123)
|
||||
**Runtime:** tokio (not smol). The HTTP server runs as a tokio task using hyper.
|
||||
|
||||
Each incoming TCP connection is handled in a detached smol task:
|
||||
### RTMP ingestion (port 8123) — `crates/server/src/main.rs`
|
||||
|
||||
1. **Handshake** — reads C0+C1 (1537 bytes) using `rml_rtmp::Handshake`, sends S0+S1+S2, reads C2 (1536 bytes) to complete the handshake.
|
||||
2. **Session setup** — creates an `rml_rtmp::ServerSession` and writes its initial response bytes to the stream.
|
||||
3. **Event loop** — reads 4096-byte chunks and calls `rtmp_session.handle_input`. The library returns a mix of `OutboundResponse` packets (written back immediately) and `RaisedEvent` values:
|
||||
Each incoming TCP connection is handled in a detached tokio task:
|
||||
|
||||
1. **Handshake** — reads C0+C1 (1537 bytes) via `rml_rtmp::Handshake`, sends S0+S1+S2, reads C2 (1536 bytes).
|
||||
2. **Session setup** — creates an `rml_rtmp::ServerSession`, writes its initial response bytes.
|
||||
3. **Event loop** — reads 4096-byte chunks, calls `rtmp_session.handle_input`. Handles:
|
||||
- `ConnectionRequested` → accepted unconditionally.
|
||||
- `PublishStreamRequested` → accepted only if `stream_key == "test"`, rejected otherwise. On accept, a `StreamSession` holding an `async_broadcast::Sender<Arc<VideoFrame>>` is inserted into the `AppState` `DashMap`.
|
||||
- `VideoDataReceived` → checked for HEVC (bytes 1–4 == `hvc1`; connection dropped if true), then passed to `H264Parser::parse`. If a frame is returned it is broadcast on the channel.
|
||||
- `PublishStreamFinished` → the entry is removed from the `DashMap`.
|
||||
- `PublishStreamRequested` → accepted for any stream key; a `StreamSession` with an `async_broadcast::Sender<Arc<VideoFrame>>` is inserted into `AppState`.
|
||||
- `VideoDataReceived` → HEVC check (bytes 1–4 == `hvc1`; drops connection if true), then passed to `H264Parser::parse`. Parsed frames are broadcast on the channel.
|
||||
- `PublishStreamFinished` → entry removed from `AppState`.
|
||||
|
||||
### H.264 parsing (AVCC → Annex-B)
|
||||
`AppState` is a `Arc<Mutex<AppState>>` wrapping a `DashMap<String, StreamSession>`.
|
||||
|
||||
`H264Parser` processes the raw `VideoDataReceived` payload bytes:
|
||||
### HTTP API (port 3000) — `crates/server/src/http.rs`
|
||||
|
||||
- **Byte 0**: upper nibble = frame type (1 = keyframe), lower nibble = codec ID (7 = H.264; anything else is dropped).
|
||||
- **Byte 1**: AVC packet type — `0` = sequence header (AVCDecoderConfigurationRecord), `1` = NAL unit data.
|
||||
- **Bytes 2–4**: composition time offset (ignored).
|
||||
- **Bytes 5+**: payload.
|
||||
Async hyper server running in a tokio task. Routes:
|
||||
|
||||
For packet type `0` the parser walks the `AVCDecoderConfigurationRecord` to cache the raw SPS and PPS byte arrays.
|
||||
- `GET /api/catalog` — returns JSON `{ active_streams: [String] }` listing currently publishing stream keys.
|
||||
- `GET /api/meow` — returns `"meow"` (health check / placeholder).
|
||||
- WHEP signaling is not yet wired into this HTTP server (see webrtc.rs for the channel plumbing).
|
||||
|
||||
For packet type `1` the parser converts from AVCC (each NALU preceded by a 4-byte big-endian length) to Annex-B (each NALU preceded by the `00 00 00 01` start code). Before the first NALU of every keyframe it prepends the cached `SPS` and `PPS` in Annex-B form so that str0m's RTP packetizer can bundle them into a STAP-A alongside the IDR NALU.
|
||||
SDP offer/answer exchange uses `tokio::sync::mpsc` channels between HttpServer and the Webrtc task.
|
||||
|
||||
### WHEP signaling (port 5000)
|
||||
### WebRTC negotiation and media loop — `crates/server/src/webrtc.rs`
|
||||
|
||||
The browser opens a `RTCPeerConnection`, adds a `recvonly` video transceiver, calls `createOffer`, and POSTs the SDP to `POST /whep/test`.
|
||||
The `Webrtc` task receives `(stream_key, sdp_body)` tuples from `offer_rx`:
|
||||
|
||||
`tiny_http` (running in an OS thread) receives the request, sends the tuple `("", sdp_body)` on `offer_tx`, then **blocks** on `accept_rx` waiting for the answer SDP. The 201 response with `Content-Type: application/sdp` is sent once the answer arrives.
|
||||
|
||||
### WebRTC negotiation and media loop
|
||||
|
||||
The smol `Webrtc` task receives the offer from `offer_rx`:
|
||||
|
||||
1. Binds a UDP socket to `127.0.0.1:0` — this is the only ICE candidate advertised (host, UDP, loopback). Remote candidates coming from the browser are handled by str0m internally; the socket just needs to be reachable from the browser on the same machine.
|
||||
2. Builds an `Rtc` with H.264 explicitly configured for payload types 102, 104, and 106 (profiles `0x42e01f`, `0x4d001f`, `0x64001f`). The default H.264 support is disabled first so only these three PTs are offered.
|
||||
3. Adds a `SendOnly` video media track, then calls `changes.accept_offer(offer_sdp)` to produce the SDP answer.
|
||||
4. Sends the answer back on `accept_tx` (unblocking the HTTP thread), then detaches a per-connection async loop.
|
||||
1. Binds a UDP socket to `127.0.0.1:0` — only ICE candidate advertised (host, UDP, loopback).
|
||||
2. Builds `str0m::Rtc` with H.264 explicitly configured for PTs 102, 104, 106 (profiles `0x42e01f`, `0x4d001f`, `0x64001f`). Default H.264 support is disabled first.
|
||||
3. Adds a `SendOnly` video media track, calls `changes.accept_offer(offer_sdp)` to produce the SDP answer.
|
||||
4. Sends the answer back on `accept_tx`, then spawns a per-connection tokio task.
|
||||
|
||||
**Per-connection loop** (`Webrtc::detach_connection`):
|
||||
|
||||
- Calls `rtc.poll_output()` in a tight loop until it returns `Output::Timeout(deadline)`. Each iteration either sends a UDP datagram (`Output::Transmit`) or handles an event:
|
||||
- `Event::MediaAdded` — iterates the writer's payload params and picks the PT with the highest `profile_level_id`, storing it in `video_pt`.
|
||||
- `Event::Connected` — sets `connected = true`; media sending begins after this.
|
||||
- Once connected, on each iteration it lazily subscribes to the `async_broadcast` channel for stream key `"test"` (if not already subscribed), then drains all available frames with `try_recv` and writes each one via `writer.write(pt, now, rtp_time, frame.data)`. The `rtp_time` is constructed from `frame.timestamp_ms` as `MediaTime::from_millis`.
|
||||
- Then waits with `smol::future::or` for either the str0m deadline or a UDP datagram. Incoming datagrams are fed to `rtc.handle_input(Input::Receive(...))` for ICE/DTLS/RTCP processing; a timeout fires `rtc.handle_input(Input::Timeout(...))`.
|
||||
- Drains `rtc.poll_output()` until `Output::Timeout`. Each iteration sends UDP datagrams (`Output::Transmit`) or handles events:
|
||||
- `Event::MediaAdded` — picks the PT with the highest `profile_level_id`, stores in `video_pt`.
|
||||
- `Event::Connected` — sets `connected = true`; media sending begins.
|
||||
- When connected, lazily subscribes to the `async_broadcast` channel for the stream key, then drains up to 8 frames per iteration via `try_recv`, writing each with `writer.write(pt, now, rtp_time, frame.data)`. `rtp_time` is computed as `MediaTime::from_90khz(timestamp_ms * 90)`.
|
||||
- Waits (capped at 20 ms) with `tokio::select!` for either the str0m deadline or a UDP datagram. Incoming datagrams are fed to `rtc.handle_input(Input::Receive(...))`.
|
||||
|
||||
**Module breakdown:**
|
||||
### H.264 parsing — `crates/server/src/media.rs`
|
||||
|
||||
- `main.rs` — Entry point. Owns `AppState` (a `DashMap<String, StreamSession>`). Spawns the HTTP and WebRTC tasks, then loops accepting RTMP TCP connections. Each RTMP connection runs the `rml_rtmp` handshake and session, parses H.264 frames via `H264Parser`, and broadcasts them on a per-stream `async_broadcast` channel stored in `AppState`.
|
||||
`H264Parser` converts raw RTMP `VideoDataReceived` payloads (AVCC) to Annex-B:
|
||||
|
||||
- `src/http.rs` — Blocking `tiny_http` server on port 5000 running in its own OS thread. Handles CORS and WHEP `POST /whep/*` requests. Forwards the SDP offer to the WebRTC task via `smol::channel` and blocks waiting for the SDP answer before responding.
|
||||
- **Byte 0**: upper nibble = frame type (1 = keyframe), lower nibble = codec ID (7 = H.264; anything else dropped).
|
||||
- **Byte 1**: AVC packet type — `0` = sequence header, `1` = NAL unit data.
|
||||
- **Bytes 5+**: payload.
|
||||
|
||||
- `src/webrtc.rs` — Async task (smol) that receives SDP offers, builds a `str0m` `Rtc` instance with H.264 codec config, negotiates the answer, and detaches a per-connection loop. That loop polls `str0m` for output (packets to send), listens on a per-connection UDP socket for incoming DTLS/ICE, and drains the `async_broadcast` video channel to write Annex-B frames into the `str0m` writer.
|
||||
Packet type `0` walks `AVCDecoderConfigurationRecord` to cache SPS and PPS byte arrays.
|
||||
|
||||
- `src/media.rs` — `H264Parser` converts raw RTMP `VideoDataReceived` payloads (AVCC format) to Annex-B. Sequence-header packets (AVC packet type 0) update cached SPS/PPS; NAL unit packets (type 1) prepend SPS+PPS before each keyframe and convert length-prefixed NALUs to start-code NALUs.
|
||||
Packet type `1` converts AVCC (4-byte big-endian length prefix per NALU) to Annex-B (`00 00 00 01` start code). Before the first NALU of every keyframe, prepends SPS+PPS in Annex-B form.
|
||||
|
||||
- `src/rtmp.rs` — An early stub for a manual RTMP handshake implementation, unused in the current flow (the actual RTMP handling uses `rml_rtmp` directly in `main.rs`).
|
||||
### Unused stub — `crates/server/src/rtmp.rs`
|
||||
|
||||
**Key design choices:**
|
||||
Early manual RTMP handshake implementation, not used in the current flow.
|
||||
|
||||
- Only the hardcoded stream key `"test"` is accepted; other keys are rejected.
|
||||
- HEVC/H.265 is explicitly rejected at ingestion time.
|
||||
- `async_broadcast` channels are overflow-enabled (old frames are silently dropped if no subscriber drains fast enough).
|
||||
- The runtime is `smol` (not tokio). The HTTP server runs in a dedicated OS thread (`std::thread::spawn`) because `tiny_http` is synchronous.
|
||||
- `str0m` handles RTP packetization, DTLS, and ICE internally; the code only provides Annex-B video bytes and a UDP socket.
|
||||
## Database (SeaORM + SQLite)
|
||||
|
||||
DB file: `stream.db`. Migrations: `crates/migration/`. Entities: `crates/entity/`.
|
||||
|
||||
Run migrations: `Migrator::up(&db, None).await?` — idempotent, tracked in `seaql_migrations`.
|
||||
|
||||
### Entities
|
||||
|
||||
**`users`** (`crates/entity/src/users.rs`)
|
||||
- Fields: `id`, `username`, `hashed_password`
|
||||
- Relations: `has_many` → `stream_key`
|
||||
- `Entity::create(db, username, hashed_password)` — inserts a new user
|
||||
- `ActiveModel::update_username(db, username)` — updates username
|
||||
- `ActiveModel::update_password(db, hashed_password)` — updates password hash
|
||||
- Passwords must be hashed before being passed to these methods
|
||||
|
||||
**`stream_key`** (`crates/entity/src/stream_key.rs`)
|
||||
- Fields: `id`, `key_value` (unique), `user_id`, `label`, `is_active`, `is_unlisted`, `created_at`
|
||||
- Relations: `belongs_to` → `users`, `has_many` → `stream_session`
|
||||
|
||||
**`stream_session`** (`crates/entity/src/stream_session.rs`)
|
||||
- Fields: `id`, `stream_key_id`, `started_at`, `ended_at` (nullable)
|
||||
- Relations: `belongs_to` → `stream_key`
|
||||
|
||||
### SeaORM conventions
|
||||
|
||||
- Query methods go on `Entity` (e.g. `Entity::find_by_x`).
|
||||
- Mutation helpers that intercept save logic (e.g. setting timestamps, pre-save transforms) go on `ActiveModel`.
|
||||
- For destructive schema changes in prod, use expand-contract: add new structure → backfill → switch app code → drop old structure in a later migration.
|
||||
|
||||
**Test page:** `index.html` — open in a browser to view the stream via WHEP without any extra tooling.
|
||||
|
||||
Reference in New Issue
Block a user