rtmp -> whep, is working now
This commit is contained in:
@@ -0,0 +1,103 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Build
|
||||
cargo build
|
||||
|
||||
# Run
|
||||
cargo run
|
||||
|
||||
# Build (Nix)
|
||||
nix build
|
||||
|
||||
# Dev shell (provides clang + mold linker)
|
||||
nix develop
|
||||
```
|
||||
|
||||
No tests exist yet.
|
||||
|
||||
## Architecture
|
||||
|
||||
This is an RTMP-to-WHEP bridge: it accepts an RTMP video publish stream and re-streams it to browsers via WebRTC (using the WHEP signaling protocol).
|
||||
|
||||
**Signal flow:**
|
||||
|
||||
```
|
||||
OBS/encoder → RTMP (port 8123) → H264Parser → async_broadcast channel
|
||||
↓
|
||||
Browser ← WebRTC/UDP ← str0m Rtc ← WHEP HTTP (port 5000)
|
||||
```
|
||||
|
||||
### RTMP ingestion (port 8123)
|
||||
|
||||
Each incoming TCP connection is handled in a detached smol task:
|
||||
|
||||
1. **Handshake** — reads C0+C1 (1537 bytes) using `rml_rtmp::Handshake`, sends S0+S1+S2, reads C2 (1536 bytes) to complete the handshake.
|
||||
2. **Session setup** — creates an `rml_rtmp::ServerSession` and writes its initial response bytes to the stream.
|
||||
3. **Event loop** — reads 4096-byte chunks and calls `rtmp_session.handle_input`. The library returns a mix of `OutboundResponse` packets (written back immediately) and `RaisedEvent` values:
|
||||
- `ConnectionRequested` → accepted unconditionally.
|
||||
- `PublishStreamRequested` → accepted only if `stream_key == "test"`, rejected otherwise. On accept, a `StreamSession` holding an `async_broadcast::Sender<Arc<VideoFrame>>` is inserted into the `AppState` `DashMap`.
|
||||
- `VideoDataReceived` → checked for HEVC (bytes 1–4 == `hvc1`; connection dropped if true), then passed to `H264Parser::parse`. If a frame is returned it is broadcast on the channel.
|
||||
- `PublishStreamFinished` → the entry is removed from the `DashMap`.
|
||||
|
||||
### H.264 parsing (AVCC → Annex-B)
|
||||
|
||||
`H264Parser` processes the raw `VideoDataReceived` payload bytes:
|
||||
|
||||
- **Byte 0**: upper nibble = frame type (1 = keyframe), lower nibble = codec ID (7 = H.264; anything else is dropped).
|
||||
- **Byte 1**: AVC packet type — `0` = sequence header (AVCDecoderConfigurationRecord), `1` = NAL unit data.
|
||||
- **Bytes 2–4**: composition time offset (ignored).
|
||||
- **Bytes 5+**: payload.
|
||||
|
||||
For packet type `0` the parser walks the `AVCDecoderConfigurationRecord` to cache the raw SPS and PPS byte arrays.
|
||||
|
||||
For packet type `1` the parser converts from AVCC (each NALU preceded by a 4-byte big-endian length) to Annex-B (each NALU preceded by the `00 00 00 01` start code). Before the first NALU of every keyframe it prepends the cached `SPS` and `PPS` in Annex-B form so that str0m's RTP packetizer can bundle them into a STAP-A alongside the IDR NALU.
|
||||
|
||||
### WHEP signaling (port 5000)
|
||||
|
||||
The browser opens a `RTCPeerConnection`, adds a `recvonly` video transceiver, calls `createOffer`, and POSTs the SDP to `POST /whep/test`.
|
||||
|
||||
`tiny_http` (running in an OS thread) receives the request, sends the tuple `("", sdp_body)` on `offer_tx`, then **blocks** on `accept_rx` waiting for the answer SDP. The 201 response with `Content-Type: application/sdp` is sent once the answer arrives.
|
||||
|
||||
### WebRTC negotiation and media loop
|
||||
|
||||
The smol `Webrtc` task receives the offer from `offer_rx`:
|
||||
|
||||
1. Binds a UDP socket to `127.0.0.1:0` — this is the only ICE candidate advertised (host, UDP, loopback). Remote candidates coming from the browser are handled by str0m internally; the socket just needs to be reachable from the browser on the same machine.
|
||||
2. Builds an `Rtc` with H.264 explicitly configured for payload types 102, 104, and 106 (profiles `0x42e01f`, `0x4d001f`, `0x64001f`). The default H.264 support is disabled first so only these three PTs are offered.
|
||||
3. Adds a `SendOnly` video media track, then calls `changes.accept_offer(offer_sdp)` to produce the SDP answer.
|
||||
4. Sends the answer back on `accept_tx` (unblocking the HTTP thread), then detaches a per-connection async loop.
|
||||
|
||||
**Per-connection loop** (`Webrtc::detach_connection`):
|
||||
|
||||
- Calls `rtc.poll_output()` in a tight loop until it returns `Output::Timeout(deadline)`. Each iteration either sends a UDP datagram (`Output::Transmit`) or handles an event:
|
||||
- `Event::MediaAdded` — iterates the writer's payload params and picks the PT with the highest `profile_level_id`, storing it in `video_pt`.
|
||||
- `Event::Connected` — sets `connected = true`; media sending begins after this.
|
||||
- Once connected, on each iteration it lazily subscribes to the `async_broadcast` channel for stream key `"test"` (if not already subscribed), then drains all available frames with `try_recv` and writes each one via `writer.write(pt, now, rtp_time, frame.data)`. The `rtp_time` is constructed from `frame.timestamp_ms` as `MediaTime::from_millis`.
|
||||
- Then waits with `smol::future::or` for either the str0m deadline or a UDP datagram. Incoming datagrams are fed to `rtc.handle_input(Input::Receive(...))` for ICE/DTLS/RTCP processing; a timeout fires `rtc.handle_input(Input::Timeout(...))`.
|
||||
|
||||
**Module breakdown:**
|
||||
|
||||
- `main.rs` — Entry point. Owns `AppState` (a `DashMap<String, StreamSession>`). Spawns the HTTP and WebRTC tasks, then loops accepting RTMP TCP connections. Each RTMP connection runs the `rml_rtmp` handshake and session, parses H.264 frames via `H264Parser`, and broadcasts them on a per-stream `async_broadcast` channel stored in `AppState`.
|
||||
|
||||
- `src/http.rs` — Blocking `tiny_http` server on port 5000 running in its own OS thread. Handles CORS and WHEP `POST /whep/*` requests. Forwards the SDP offer to the WebRTC task via `smol::channel` and blocks waiting for the SDP answer before responding.
|
||||
|
||||
- `src/webrtc.rs` — Async task (smol) that receives SDP offers, builds a `str0m` `Rtc` instance with H.264 codec config, negotiates the answer, and detaches a per-connection loop. That loop polls `str0m` for output (packets to send), listens on a per-connection UDP socket for incoming DTLS/ICE, and drains the `async_broadcast` video channel to write Annex-B frames into the `str0m` writer.
|
||||
|
||||
- `src/media.rs` — `H264Parser` converts raw RTMP `VideoDataReceived` payloads (AVCC format) to Annex-B. Sequence-header packets (AVC packet type 0) update cached SPS/PPS; NAL unit packets (type 1) prepend SPS+PPS before each keyframe and convert length-prefixed NALUs to start-code NALUs.
|
||||
|
||||
- `src/rtmp.rs` — An early stub for a manual RTMP handshake implementation, unused in the current flow (the actual RTMP handling uses `rml_rtmp` directly in `main.rs`).
|
||||
|
||||
**Key design choices:**
|
||||
|
||||
- Only the hardcoded stream key `"test"` is accepted; other keys are rejected.
|
||||
- HEVC/H.265 is explicitly rejected at ingestion time.
|
||||
- `async_broadcast` channels are overflow-enabled (old frames are silently dropped if no subscriber drains fast enough).
|
||||
- The runtime is `smol` (not tokio). The HTTP server runs in a dedicated OS thread (`std::thread::spawn`) because `tiny_http` is synchronous.
|
||||
- `str0m` handles RTP packetization, DTLS, and ICE internally; the code only provides Annex-B video bytes and a UDP socket.
|
||||
|
||||
**Test page:** `index.html` — open in a browser to view the stream via WHEP without any extra tooling.
|
||||
Reference in New Issue
Block a user