rtmp -> whep, is working now

CLAUDE.md

@@ -0,0 +1,103 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Commands
+
+```bash
+# Build
+cargo build
+
+# Run
+cargo run
+
+# Build (Nix)
+nix build
+
+# Dev shell (provides clang + mold linker)
+nix develop
+```
+
+No tests exist yet.
+
+## Architecture
+
+This is an RTMP-to-WHEP bridge: it accepts an RTMP video publish stream and re-streams it to browsers via WebRTC (using the WHEP signaling protocol).
+
+**Signal flow:**
+
+```
+OBS/encoder  →  RTMP (port 8123)  →  H264Parser  →  async_broadcast channel
+                                                              ↓
+Browser  ←  WebRTC/UDP  ←  str0m Rtc  ←  WHEP HTTP (port 5000)
+```
+
+### RTMP ingestion (port 8123)
+
+Each incoming TCP connection is handled in a detached smol task:
+
+1. **Handshake** — reads C0+C1 (1537 bytes) using `rml_rtmp::Handshake`, sends S0+S1+S2, reads C2 (1536 bytes) to complete the handshake.
+2. **Session setup** — creates an `rml_rtmp::ServerSession` and writes its initial response bytes to the stream.
+3. **Event loop** — reads 4096-byte chunks and calls `rtmp_session.handle_input`. The library returns a mix of `OutboundResponse` packets (written back immediately) and `RaisedEvent` values:
+   - `ConnectionRequested` → accepted unconditionally.
+   - `PublishStreamRequested` → accepted only if `stream_key == "test"`, rejected otherwise. On accept, a `StreamSession` holding an `async_broadcast::Sender<Arc<VideoFrame>>` is inserted into the `AppState` `DashMap`.
+   - `VideoDataReceived` → checked for HEVC (bytes 1–4 == `hvc1`; connection dropped if true), then passed to `H264Parser::parse`. If a frame is returned it is broadcast on the channel.
+   - `PublishStreamFinished` → the entry is removed from the `DashMap`.
+
+### H.264 parsing (AVCC → Annex-B)
+
+`H264Parser` processes the raw `VideoDataReceived` payload bytes:
+
+- **Byte 0**: upper nibble = frame type (1 = keyframe), lower nibble = codec ID (7 = H.264; anything else is dropped).
+- **Byte 1**: AVC packet type — `0` = sequence header (AVCDecoderConfigurationRecord), `1` = NAL unit data.
+- **Bytes 2–4**: composition time offset (ignored).
+- **Bytes 5+**: payload.
+
+For packet type `0` the parser walks the `AVCDecoderConfigurationRecord` to cache the raw SPS and PPS byte arrays.
+
+For packet type `1` the parser converts from AVCC (each NALU preceded by a 4-byte big-endian length) to Annex-B (each NALU preceded by the `00 00 00 01` start code). Before the first NALU of every keyframe it prepends the cached `SPS` and `PPS` in Annex-B form so that str0m's RTP packetizer can bundle them into a STAP-A alongside the IDR NALU.
+
+### WHEP signaling (port 5000)
+
+The browser opens a `RTCPeerConnection`, adds a `recvonly` video transceiver, calls `createOffer`, and POSTs the SDP to `POST /whep/test`.
+
+`tiny_http` (running in an OS thread) receives the request, sends the tuple `("", sdp_body)` on `offer_tx`, then **blocks** on `accept_rx` waiting for the answer SDP. The 201 response with `Content-Type: application/sdp` is sent once the answer arrives.
+
+### WebRTC negotiation and media loop
+
+The smol `Webrtc` task receives the offer from `offer_rx`:
+
+1. Binds a UDP socket to `127.0.0.1:0` — this is the only ICE candidate advertised (host, UDP, loopback). Remote candidates coming from the browser are handled by str0m internally; the socket just needs to be reachable from the browser on the same machine.
+2. Builds an `Rtc` with H.264 explicitly configured for payload types 102, 104, and 106 (profiles `0x42e01f`, `0x4d001f`, `0x64001f`). The default H.264 support is disabled first so only these three PTs are offered.
+3. Adds a `SendOnly` video media track, then calls `changes.accept_offer(offer_sdp)` to produce the SDP answer.
+4. Sends the answer back on `accept_tx` (unblocking the HTTP thread), then detaches a per-connection async loop.
+
+**Per-connection loop** (`Webrtc::detach_connection`):
+
+- Calls `rtc.poll_output()` in a tight loop until it returns `Output::Timeout(deadline)`. Each iteration either sends a UDP datagram (`Output::Transmit`) or handles an event:
+  - `Event::MediaAdded` — iterates the writer's payload params and picks the PT with the highest `profile_level_id`, storing it in `video_pt`.
+  - `Event::Connected` — sets `connected = true`; media sending begins after this.
+- Once connected, on each iteration it lazily subscribes to the `async_broadcast` channel for stream key `"test"` (if not already subscribed), then drains all available frames with `try_recv` and writes each one via `writer.write(pt, now, rtp_time, frame.data)`. The `rtp_time` is constructed from `frame.timestamp_ms` as `MediaTime::from_millis`.
+- Then waits with `smol::future::or` for either the str0m deadline or a UDP datagram. Incoming datagrams are fed to `rtc.handle_input(Input::Receive(...))` for ICE/DTLS/RTCP processing; a timeout fires `rtc.handle_input(Input::Timeout(...))`.
+
+**Module breakdown:**
+
+- `main.rs` — Entry point. Owns `AppState` (a `DashMap<String, StreamSession>`). Spawns the HTTP and WebRTC tasks, then loops accepting RTMP TCP connections. Each RTMP connection runs the `rml_rtmp` handshake and session, parses H.264 frames via `H264Parser`, and broadcasts them on a per-stream `async_broadcast` channel stored in `AppState`.
+
+- `src/http.rs` — Blocking `tiny_http` server on port 5000 running in its own OS thread. Handles CORS and WHEP `POST /whep/*` requests. Forwards the SDP offer to the WebRTC task via `smol::channel` and blocks waiting for the SDP answer before responding.
+
+- `src/webrtc.rs` — Async task (smol) that receives SDP offers, builds a `str0m` `Rtc` instance with H.264 codec config, negotiates the answer, and detaches a per-connection loop. That loop polls `str0m` for output (packets to send), listens on a per-connection UDP socket for incoming DTLS/ICE, and drains the `async_broadcast` video channel to write Annex-B frames into the `str0m` writer.
+
+- `src/media.rs` — `H264Parser` converts raw RTMP `VideoDataReceived` payloads (AVCC format) to Annex-B. Sequence-header packets (AVC packet type 0) update cached SPS/PPS; NAL unit packets (type 1) prepend SPS+PPS before each keyframe and convert length-prefixed NALUs to start-code NALUs.
+
+- `src/rtmp.rs` — An early stub for a manual RTMP handshake implementation, unused in the current flow (the actual RTMP handling uses `rml_rtmp` directly in `main.rs`).
+
+**Key design choices:**
+
+- Only the hardcoded stream key `"test"` is accepted; other keys are rejected.
+- HEVC/H.265 is explicitly rejected at ingestion time.
+- `async_broadcast` channels are overflow-enabled (old frames are silently dropped if no subscriber drains fast enough).
+- The runtime is `smol` (not tokio). The HTTP server runs in a dedicated OS thread (`std::thread::spawn`) because `tiny_http` is synchronous.
+- `str0m` handles RTP packetization, DTLS, and ICE internally; the code only provides Annex-B video bytes and a UDP socket.
+
+**Test page:** `index.html` — open in a browser to view the stream via WHEP without any extra tooling.

Cargo.toml

@@ -4,6 +4,14 @@ version = "0.1.0"
 edition = "2024"
 
 [dependencies]
+async-broadcast = "0.7.2"
+bytes = "1.11.1"
+dashmap = "6.2.1"
 macro_rules_attribute = "0.2.2"
+rand = "0.10.1"
+rml_rtmp = "0.8.0"
+futures-lite = "2"
 smol = "2.0.2"
 smol-macros = "0.1.1"
+str0m = "0.20.0"
+tiny_http = "0.12.0"

index.html

@@ -0,0 +1,119 @@
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <title>WHEP Viewer</title>
+  <style>
+    body { margin: 0; background: #111; color: #eee; font-family: monospace; }
+    #video { display: block; width: 100%; max-width: 800px; }
+    #stats {
+      max-width: 800px;
+      padding: 8px 12px;
+      background: rgba(0,0,0,0.6);
+      font-size: 13px;
+      line-height: 1.8;
+    }
+    .label { color: #aaa; }
+    .value { color: #7ef; font-weight: bold; }
+    .warn  { color: #fa0; }
+    .bad   { color: #f44; }
+  </style>
+</head>
+<body>
+  <video id="video" autoplay muted playsinline></video>
+  <div id="stats">Waiting for stream…</div>
+<script>
+  const fmt = ms => ms < 0 ? '—' : ms.toFixed(1) + ' ms';
+
+  const colorClass = ms => ms < 0 ? '' : ms < 80 ? 'value' : ms < 200 ? 'warn' : 'bad';
+
+  let prevStats = null;
+
+  async function pollStats(pc) {
+    const reports = await pc.getStats();
+
+    let inbound = null;
+    let networkRttMs = -1;
+    reports.forEach(r => {
+      if (r.type === 'inbound-rtp' && r.kind === 'video') inbound = r;
+      // The nominated ICE candidate pair carries the active STUN ping RTT.
+      if (r.type === 'candidate-pair' && r.nominated && r.currentRoundTripTime != null) {
+        networkRttMs = r.currentRoundTripTime * 1000;
+      }
+    });
+
+    if (!inbound) return;
+
+    // Network one-way: ICE STUN ping RTT / 2.
+    const networkOneWayMs = networkRttMs >= 0 ? networkRttMs / 2 : -1;
+
+    // Jitter buffer latency: average time a packet waits before being emitted to decoder.
+    const jitterMs = inbound.jitterBufferEmittedCount > 0
+      ? (inbound.jitterBufferDelay / inbound.jitterBufferEmittedCount) * 1000
+      : -1;
+
+    // Decode latency: average time spent decoding each frame.
+    const decodeMs = inbound.framesDecoded > 0
+      ? (inbound.totalDecodeTime / inbound.framesDecoded) * 1000
+      : -1;
+
+    // Server → client: network transit + jitter buffer + decode.
+    const serverToClientMs =
+      (networkOneWayMs >= 0 && jitterMs >= 0 && decodeMs >= 0)
+        ? networkOneWayMs + jitterMs + decodeMs
+        : -1;
+
+    // Frames per second (received from network, before decode).
+    const fps = inbound.framesPerSecond ?? -1;
+
+    // Packets lost ratio.
+    const totalPkts = (inbound.packetsReceived || 0) + (inbound.packetsLost || 0);
+    const lossRatio = totalPkts > 0
+      ? ((inbound.packetsLost || 0) / totalPkts * 100).toFixed(1) + '%'
+      : '—';
+
+    const el = document.getElementById('stats');
+    const row = (label, val, cls) =>
+      `<span class="label">${label}:</span> <span class="${cls}">${val}</span>`;
+
+    el.innerHTML = [
+      row('Server → Client',  fmt(serverToClientMs), colorClass(serverToClientMs)),
+      row('  Network (1-way)', fmt(networkOneWayMs),  colorClass(networkOneWayMs)),
+      row('  Jitter buffer',   fmt(jitterMs),         colorClass(jitterMs)),
+      row('  Decode',          fmt(decodeMs),          colorClass(decodeMs)),
+      row('FPS',               fps >= 0 ? fps.toFixed(1) : '—', 'value'),
+      row('Packet loss',       lossRatio, 'value'),
+      row('Jitter',            fmt(inbound.jitter * 1000), colorClass(inbound.jitter * 1000)),
+    ].join('<br>');
+
+    prevStats = inbound;
+  }
+
+  const start = async () => {
+    window.pc = new RTCPeerConnection();
+    const pc = window.pc;
+    pc.addTransceiver('video', { direction: 'recvonly' });
+    pc.ontrack = (e) => {
+      const video = document.getElementById('video');
+      video.srcObject = new MediaStream([e.track]);
+      video.play().catch(err => console.error('play() failed:', err));
+      // Start polling once we have a track.
+      setInterval(() => pollStats(pc), 500);
+    };
+    pc.oniceconnectionstatechange = () => console.log('ice state:', pc.iceConnectionState);
+
+    const offer = await pc.createOffer();
+    await pc.setLocalDescription(offer);
+    const res = await fetch('http://localhost:5000/whep/test', {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/sdp' },
+      body: offer.sdp,
+    });
+    const answer = await res.text();
+    await pc.setRemoteDescription({ type: 'answer', sdp: answer });
+  };
+
+  start();
+</script>
+</body>
+</html>

src/http.rs

@@ -0,0 +1,60 @@
+use std::error::Error;
+
+use smol::channel::{Receiver, Sender};
+use tiny_http::{Header, Response, Server, StatusCode};
+
+pub struct HttpServer {
+    pub offer_tx: Sender<(String, String)>,
+    pub accept_rx: Receiver<(String, String)>,
+}
+
+impl HttpServer {
+    pub fn start(self) -> Result<(), Box<dyn Error>> {
+        std::thread::spawn(move || {
+            let server = Server::http("0.0.0.0:5000").unwrap();
+            for mut request in server.incoming_requests() {
+                let cors =
+                    Header::from_bytes(&b"Access-Control-Allow-Origin"[..], &b"*"[..]).unwrap();
+                let cors_methods =
+                    Header::from_bytes(&b"Access-Control-Allow-Methods"[..], &b"POST, OPTIONS"[..])
+                        .unwrap();
+                let cors_headers =
+                    Header::from_bytes(&b"Access-Control-Allow-Headers"[..], &b"Content-Type"[..])
+                        .unwrap();
+
+                if *request.method() == tiny_http::Method::Options {
+                    let reply = Response::from_data("")
+                        .with_status_code(StatusCode(204))
+                        .with_header(cors)
+                        .with_header(cors_methods)
+                        .with_header(cors_headers);
+                    request.respond(reply).unwrap();
+                    continue;
+                }
+
+                if request.url().contains("whep") && *request.method() == tiny_http::Method::Post {
+                    let mut body = String::new();
+                    request.as_reader().read_to_string(&mut body).unwrap();
+                    println!("HTTP: received offer, body length={}", body.len());
+                    smol::block_on(self.offer_tx.send(("".to_string(), body))).unwrap();
+                    let content_type =
+                        Header::from_bytes(&b"Content-Type"[..], &b"application/sdp"[..]).unwrap();
+
+                    let reply = match smol::block_on(self.accept_rx.recv()) {
+                        Ok(accept) => Response::from_data(accept.1)
+                            .with_status_code(StatusCode(201))
+                            .with_header(content_type)
+                            .with_header(cors)
+                            .with_header(cors_methods)
+                            .with_header(cors_headers),
+                        Err(_) => Response::from_data("")
+                            .with_status_code(StatusCode(500))
+                            .with_header(content_type),
+                    };
+                    request.respond(reply).unwrap();
+                }
+            }
+        });
+        Ok(())
+    }
+}

src/main.rs

@@ -1,23 +1,181 @@
-use std::error::Error;
+use std::{error::Error, sync::Arc};
 
+use async_broadcast::{Receiver, Sender, broadcast};
+use dashmap::DashMap;
 use macro_rules_attribute::apply;
-use smol::{io::AsyncReadExt, net::TcpListener, stream::StreamExt};
+use rml_rtmp::{
+    handshake::{Handshake, HandshakeProcessResult, PeerType},
+    sessions::{ServerSession, ServerSessionConfig, ServerSessionEvent, ServerSessionResult},
+};
+use smol::{
+    io::{AsyncReadExt, AsyncWriteExt},
+    lock::Mutex,
+    net::TcpListener,
+    stream::StreamExt,
+};
 use smol_macros::main;
 
+use crate::{
+    http::HttpServer,
+    media::{H264Parser, VideoFrame},
+};
+
+mod http;
+mod media;
+mod rtmp;
+mod webrtc;
+
+struct AppState {
+    stream_sessions: Arc<DashMap<String, StreamSession>>,
+}
+
+struct StreamSession {
+    stream_key: String,
+    frame_channel: async_broadcast::Sender<Arc<VideoFrame>>,
+}
+
 #[apply(main!)]
 async fn main() -> Result<(), Box<dyn Error>> {
     let listener = TcpListener::bind("0.0.0.0:8123").await?;
     let mut incoming = listener.incoming();
+    let mut appstate = Arc::new(Mutex::new(AppState {
+        stream_sessions: Arc::new(DashMap::new()),
+    }));
+    let (offer_tx, offer_rx) = smol::channel::bounded::<(String, String)>(32);
+    let (answer_tx, answer_rx) = smol::channel::bounded::<(String, String)>(32);
+
+    let http = HttpServer {
+        offer_tx,
+        accept_rx: answer_rx,
+    };
+
+    http.start()?;
+
+    let app = appstate.lock().await;
+
+    let webrtc = webrtc::Webrtc {
+        offer_rx,
+        accept_tx: answer_tx,
+        sessions_ref: app.stream_sessions.clone(),
+    };
+
+    webrtc.start()?;
+
+    drop(app);
 
     while let Some(connection) = incoming.next().await {
-        let mut stream = connection?;
+        let stream = connection?;
+        let appstate = appstate.clone();
+
         smol::spawn(async move {
-            let mut buf = vec![0; 1024];
+            let mut stream = stream;
-            stream.read(&mut buf).await.unwrap();
+            let mut server = Handshake::new(PeerType::Server);
-            print!("{:#?}", buf);
+            let appstate = appstate;
+
+            let mut c0_c1: [u8; 1537] = [0; 1537];
+            stream.read_exact(&mut c0_c1).await.unwrap();
+            let s0_s1_s2 = server.process_bytes(&c0_c1);
+            let s0_s1_s2 = match s0_s1_s2 {
+                Ok(HandshakeProcessResult::InProgress {
+                    response_bytes: bytes,
+                }) => bytes,
+                _ => panic!("handshake failed"),
+            };
+            stream.write_all(&s0_s1_s2).await.unwrap();
+            let mut c2 = vec![0u8; 1536];
+            stream.read_exact(&mut c2).await.unwrap();
+
+            match server.process_bytes(&c2[..]) {
+                Ok(HandshakeProcessResult::Completed { .. }) => {}
+                Ok(HandshakeProcessResult::InProgress {
+                    response_bytes: meow,
+                }) => stream.write_all(&meow).await.unwrap(),
+                x => panic!("Unexpected process_bytes response: {:?}", x),
+            }
+
+            let config = ServerSessionConfig::new();
+            let (mut rtmp_session, bytes) = ServerSession::new(config).unwrap();
+            for x in bytes {
+                if let ServerSessionResult::OutboundResponse(packet) = x {
+                    stream.write_all(&packet.bytes).await.unwrap();
+                }
+            }
+            let (mut video_channel, _video_rx) = broadcast::<Arc<VideoFrame>>(32);
+            video_channel.set_overflow(true);
+            let mut parser = H264Parser::new();
+
+            loop {
+                let mut buf: [u8; 4096] = [0; 4096];
+                let n = stream.read(&mut buf).await.unwrap();
+
+                let obs_events = rtmp_session.handle_input(&buf[..n]).unwrap();
+                for event in obs_events {
+                    match event {
+                        ServerSessionResult::OutboundResponse(packet) => {
+                            stream.write_all(&packet.bytes).await.unwrap();
+                        }
+                        ServerSessionResult::RaisedEvent(x) => match x {
+                            ServerSessionEvent::PublishStreamFinished { .. } => {
+                                appstate.lock().await.stream_sessions.remove("test");
+                            }
+                            ServerSessionEvent::PublishStreamRequested {
+                                request_id,
+                                app_name,
+                                stream_key,
+                                ..
+                            } => {
+                                let mut reply = rtmp_session.accept_request(request_id).unwrap();
+                                if stream_key != "test" {
+                                    reply =
+                                        rtmp_session.reject_request(request_id, "", "").unwrap();
+                                }
+
+                                let session = StreamSession {
+                                    stream_key: stream_key.clone(),
+                                    frame_channel: video_channel.clone(),
+                                };
+
+                                appstate
+                                    .lock()
+                                    .await
+                                    .stream_sessions
+                                    .insert(stream_key.clone(), session);
+
+                                for x in reply {
+                                    if let ServerSessionResult::OutboundResponse(y) = x {
+                                        stream.write_all(&y.bytes).await.unwrap();
+                                    }
+                                }
+                            }
+                            ServerSessionEvent::ConnectionRequested { request_id, .. } => {
+                                let reply = rtmp_session.accept_request(request_id).unwrap();
+                                for x in reply {
+                                    if let ServerSessionResult::OutboundResponse(y) = x {
+                                        stream.write_all(&y.bytes).await.unwrap();
+                                    }
+                                }
+                            }
+                            ServerSessionEvent::VideoDataReceived {
+                                data, timestamp, ..
+                            } => {
+                                if data.len() >= 5 && &data[1..5] == b"hvc1" {
+                                    println!("HEVC/H.265 not supported, closing connection");
+                                    return;
+                                }
+                                if let Some(parsed_frame) = parser.parse(&data, timestamp.value) {
+                                    video_channel.broadcast(Arc::new(parsed_frame)).await.ok();
+                                }
+                            }
+                            _ => {}
+                        },
+                        _ => {}
+                    }
+                }
+            }
         })
-        .await;
+        .detach();
     }
 
     Ok(())
 }
+

src/media.rs

@@ -0,0 +1,147 @@
+// Claude slop... im not skilled amount to do this bullshit
+
+use bytes::Bytes;
+
+pub struct VideoFrame {
+    pub data: Bytes,
+    pub is_keyframe: bool,
+    pub timestamp_ms: u32,
+}
+
+pub struct AudioFrame {
+    pub data: Bytes,
+    pub timestamp_ms: u32,
+}
+
+pub struct H264Parser {
+    sps: Option<Vec<u8>>,
+    pps: Option<Vec<u8>>,
+}
+
+impl H264Parser {
+    pub fn new() -> Self {
+        Self {
+            sps: None,
+            pps: None,
+        }
+    }
+
+    /// Parse an RTMP VideoDataReceived payload. Returns None for sequence
+    /// header packets (which carry SPS/PPS but no displayable frame).
+    pub fn parse(&mut self, bytes: &[u8], timestamp_ms: u32) -> Option<VideoFrame> {
+        if bytes.len() < 5 {
+            return None;
+        }
+
+        let frame_type = (bytes[0] >> 4) & 0x0F;
+        let codec_id = bytes[0] & 0x0F;
+
+        if codec_id != 7 {
+            return None; // not H.264
+        }
+
+        let avc_packet_type = bytes[1];
+        // bytes[2..5] are the composition time offset — not needed for sending
+        let payload = &bytes[5..];
+
+        match avc_packet_type {
+            0 => {
+                self.parse_sequence_header(payload);
+                None
+            }
+            1 => {
+                let is_keyframe = frame_type == 1;
+                let data = self.avcc_to_annexb(payload, is_keyframe)?;
+                Some(VideoFrame {
+                    data: Bytes::from(data),
+                    is_keyframe,
+                    timestamp_ms,
+                })
+            }
+            _ => None,
+        }
+    }
+
+    fn parse_sequence_header(&mut self, payload: &[u8]) {
+        // AVCDecoderConfigurationRecord layout:
+        // [0] configurationVersion
+        // [1] AVCProfileIndication
+        // [2] profile_compatibility
+        // [3] AVCLevelIndication
+        // [4] 0xFF (lower 2 bits = lengthSizeMinusOne, always 3 meaning 4-byte lengths)
+        // [5] 0xE0 | numSPS
+        // [6..] SPS entries: 2-byte length + bytes
+        // then: numPPS, PPS entries: 2-byte length + bytes
+        if payload.len() < 7 {
+            return;
+        }
+
+        let mut i = 5;
+
+        let num_sps = (payload[i] & 0x1F) as usize;
+        i += 1;
+
+        for _ in 0..num_sps {
+            if i + 2 > payload.len() {
+                return;
+            }
+            let len = u16::from_be_bytes([payload[i], payload[i + 1]]) as usize;
+            i += 2;
+            if i + len > payload.len() {
+                return;
+            }
+            self.sps = Some(payload[i..i + len].to_vec());
+            i += len;
+        }
+
+        if i >= payload.len() {
+            return;
+        }
+
+        let num_pps = payload[i] as usize;
+        i += 1;
+
+        for _ in 0..num_pps {
+            if i + 2 > payload.len() {
+                return;
+            }
+            let len = u16::from_be_bytes([payload[i], payload[i + 1]]) as usize;
+            i += 2;
+            if i + len > payload.len() {
+                return;
+            }
+            self.pps = Some(payload[i..i + len].to_vec());
+            i += len;
+        }
+    }
+
+    fn avcc_to_annexb(&self, payload: &[u8], is_keyframe: bool) -> Option<Vec<u8>> {
+        let mut out = Vec::new();
+
+        // Prepend SPS+PPS before every keyframe so str0m's packetizer
+        // can bundle them into a STAP-A alongside the IDR NALU.
+        if is_keyframe {
+            if let (Some(sps), Some(pps)) = (&self.sps, &self.pps) {
+                out.extend_from_slice(&[0, 0, 0, 1]);
+                out.extend_from_slice(sps);
+                out.extend_from_slice(&[0, 0, 0, 1]);
+                out.extend_from_slice(pps);
+            }
+        }
+
+        // Convert each length-prefixed NALU to an Annex B start-code NALU.
+        let mut i = 0;
+        while i + 4 <= payload.len() {
+            let nalu_len = u32::from_be_bytes(payload[i..i + 4].try_into().unwrap()) as usize;
+            i += 4;
+            if i + nalu_len > payload.len() {
+                break;
+            }
+            out.extend_from_slice(&[0, 0, 0, 1]);
+            out.extend_from_slice(&payload[i..i + nalu_len]);
+            i += nalu_len;
+        }
+
+        if out.is_empty() { None } else { Some(out) }
+    }
+}

src/rtmp.rs

@@ -0,0 +1,52 @@
+#[derive(Default, Debug)]
+pub struct RtmpSession {
+    client: RtmpClient,
+}
+
+#[derive(Debug)]
+struct RtmpClient {
+    version: u8,
+    timestamp: u32,
+    magic_bytes: [u8; 1536],
+}
+
+impl Default for RtmpClient {
+    fn default() -> Self {
+        Self {
+            magic_bytes: [0; 1536],
+            version: 0,
+            timestamp: 0,
+        }
+    }
+}
+
+impl RtmpSession {
+    pub fn consume_handshake(&mut self, bytes: &[u8]) -> Result<(), ()> {
+        let version = bytes[0];
+        let timestamp = u32::from_be_bytes(bytes[1..5].try_into().unwrap());
+        let mut random: [u8; 1536] = [0; 1536];
+        random.copy_from_slice(&bytes[1..1537]);
+
+        println!("size of rand: {}", random.len());
+
+        let client = RtmpClient {
+            version,
+            timestamp,
+            magic_bytes: random,
+        };
+
+        self.client = client;
+
+        Ok(())
+    }
+    pub fn response(&self) -> [u8; 1537] {
+        let mut reply: [u8; 1537] = [0; 1537];
+        reply[0] = 3;
+        // reply[1..1537].copy_from_slice(&self.client.magic_bytes);
+        let mut rand: [u8; 1536] = [0; 1536];
+        rand.fill(1);
+        reply[1..1537].copy_from_slice(&rand);
+
+        reply
+    }
+}

src/webrtc.rs

@@ -0,0 +1,209 @@
+use dashmap::DashMap;
+use smol::{
+    channel::{Receiver, Sender},
+    net::UdpSocket,
+};
+use std::{
+    error::Error,
+    sync::Arc,
+    time::{Duration, Instant},
+};
+
+use str0m::{
+    Candidate, Event, IceConnectionState, Input, Output, Rtc,
+    change::SdpOffer,
+    media::{MediaKind, MediaTime, Mid},
+    net::{Protocol, Receive},
+};
+
+use crate::{StreamSession, media::VideoFrame};
+
+pub struct Webrtc {
+    pub offer_rx: Receiver<(String, String)>,
+    pub accept_tx: Sender<(String, String)>,
+    pub sessions_ref: Arc<DashMap<String, StreamSession>>,
+}
+
+impl Webrtc {
+    pub fn start(self) -> Result<(), Box<dyn Error>> {
+        smol::spawn(async move {
+            while let Ok(offer) = self.offer_rx.recv().await {
+                let (stream_key, sdp_body) = offer;
+                let socket = UdpSocket::bind("127.0.0.1:0").await.unwrap();
+                let local_addr = socket.local_addr().unwrap();
+
+                let mut builder = Rtc::builder();
+                {
+                    let cc = builder.codec_config();
+                    cc.enable_h264(false);
+                    cc.add_h264(102.into(), None, true, 0x42e01f);
+                    cc.add_h264(104.into(), None, true, 0x4d001f);
+                    cc.add_h264(106.into(), None, true, 0x64001f);
+                }
+                let mut rtc = builder.build(Instant::now());
+
+                let candidate = Candidate::host(local_addr, Protocol::Udp).unwrap();
+                rtc.add_local_candidate(candidate);
+
+                let offer_sdp = SdpOffer::from_sdp_string(&sdp_body).unwrap();
+                let mut changes = rtc.sdp_api();
+                let mid = changes.add_media(
+                    MediaKind::Video,
+                    str0m::media::Direction::SendOnly,
+                    Some(stream_key.clone()),
+                    Some("video0".to_string()),
+                    None,
+                );
+                let offer_answer = match changes.accept_offer(offer_sdp) {
+                    Ok(a) => a,
+                    Err(e) => {
+                        println!("accept_offer failed: {:?}", e);
+                        continue;
+                    }
+                };
+                let answer_sdp = offer_answer.to_sdp_string();
+
+                self.accept_tx
+                    .send((stream_key.clone(), answer_sdp))
+                    .await
+                    .unwrap();
+
+                let sessions_ref = self.sessions_ref.clone();
+                smol::spawn(async move {
+                    Webrtc::detach_connection(socket, rtc, sessions_ref, mid).await;
+                })
+                .detach();
+            }
+        })
+        .detach();
+        Ok(())
+    }
+
+    async fn detach_connection(
+        socket: UdpSocket,
+        mut rtc: Rtc,
+        sessions_ref: Arc<DashMap<String, StreamSession>>,
+        _hint_mid: Mid,
+    ) {
+        let mut video_mid: Option<Mid> = None;
+        let mut video_pt = None;
+        let mut connected = false;
+        let mut video_stream: Option<async_broadcast::Receiver<Arc<VideoFrame>>> = None;
+
+        let mut recv_buf = vec![0u8; 65535];
+        let local_addr = socket.local_addr().unwrap();
+
+        loop {
+            let deadline = loop {
+                match rtc.poll_output() {
+                    Ok(Output::Timeout(t)) => break t,
+                    Ok(Output::Transmit(t)) => {
+                        if socket.send_to(&t.contents, t.destination).await.is_err() {
+                            return;
+                        }
+                    }
+                    Ok(Output::Event(e)) => match e {
+                        Event::MediaAdded(ma) => {
+                            if ma.kind == MediaKind::Video {
+                                if let Some(writer) = rtc.writer(ma.mid) {
+                                    let best = writer
+                                        .payload_params()
+                                        .max_by_key(|p| {
+                                            p.spec().format.profile_level_id.unwrap_or(0)
+                                        });
+                                    if let Some(params) = best {
+                                        println!("Selected PT {:?}", params.pt());
+                                        video_pt = Some(params.pt());
+                                        video_mid = Some(ma.mid);
+                                    }
+                                }
+                            }
+                        }
+                        Event::IceConnectionStateChange(state) => {
+                            println!("ICE state: {:?}", state);
+                        }
+                        Event::Connected => {
+                            println!("DTLS+ICE connected, ready for media");
+                            connected = true;
+                        }
+                        _ => {}
+                    },
+                    Err(_) => return,
+                }
+            };
+
+            if connected {
+                if video_stream.is_none() {
+                    if let Some(session) = sessions_ref.get("test") {
+                        video_stream = Some(session.frame_channel.new_receiver());
+                    }
+                }
+                if let Some(ref mut stream) = video_stream {
+                    // Drain at most 8 frames per loop tick so the UDP socket
+                    // (ICE keepalives, RTCP) is not starved by a backlog.
+                    for _ in 0..8 {
+                        match stream.try_recv() {
+                            Ok(frame) => {
+                                let now = Instant::now();
+                                // Explicit 90 kHz clock for H.264 RTP timestamps.
+                                let rtp_time = MediaTime::from_90khz(frame.timestamp_ms as u64 * 90);
+                                if let (Some(pt), Some(writer)) =
+                                    (video_pt, video_mid.and_then(|m| rtc.writer(m)))
+                                {
+                                    if let Err(e) =
+                                        writer.write(pt, now, rtp_time, frame.data.to_vec())
+                                    {
+                                        println!("write error: {:?}", e);
+                                    }
+                                }
+                            }
+                            Err(async_broadcast::TryRecvError::Empty) => break,
+                            Err(async_broadcast::TryRecvError::Closed) => return,
+                            // Overflow means some frames were dropped; the next
+                            // try_recv will give the oldest surviving frame, so
+                            // continue draining rather than breaking.
+                            Err(async_broadcast::TryRecvError::Overflowed(_)) => continue,
+                        }
+                    }
+                }
+            }
+
+            // Cap wait to 20 ms so frame delivery stays timely even when
+            // str0m's deadline is far out.
+            let wait_until = deadline.min(Instant::now() + Duration::from_millis(20)).max(Instant::now());
+
+            let input = smol::future::or(
+                async {
+                    smol::Timer::at(wait_until).await;
+                    None
+                },
+                async {
+                    let (n, from) = socket.recv_from(&mut recv_buf).await.ok()?;
+                    Some((n, from))
+                },
+            )
+            .await;
+
+            match input {
+                None => {
+                    rtc.handle_input(Input::Timeout(Instant::now())).ok();
+                }
+                Some((n, from)) => {
+                    let data = recv_buf[..n].to_vec();
+                    if let Ok(contents) = data.as_slice().try_into() {
+                        rtc.handle_input(Input::Receive(
+                            Instant::now(),
+                            Receive {
+                                proto: Protocol::Udp,
+                                source: from,
+                                destination: local_addr,
+                                contents,
+                            },
+                        ))
+                        .ok();
+                    }
+                }
+            }
+        }
+    }
+}