moq-video

What a browser gets from getUserMedia and WebCodecs, for native Rust: no ffmpeg, no GStreamer, no system codec to install.

Module	Does	Backends
`capture`	Camera, display, window, or application frames	AVFoundation + ScreenCaptureKit (macOS), V4L2 + X11/portal + PipeWire (Linux), Media Foundation + Windows.Graphics.Capture (Windows)
`encode`	Frames to H.264/H.265, published as a hang track	VideoToolbox, Media Foundation, NVENC, VAAPI, V4L2 M2M, MediaCodec (Android), openh264
`decode`	A subscribed track back to frames	VideoToolbox, Media Foundation/DXVA, NVDEC, VAAPI, V4L2 M2M, MediaCodec (Android), openh264
`render`	A frame as a `wgpu` texture	wgpu, with zero-copy Metal and Vulkan imports

Highlights:

Automatic backend selection, hardware first. Linux GPU libraries are dlopened at runtime, so one binary starts anywhere and warns when it falls back to software. openh264 (the default-on openh264 feature) is statically linked as the H.264 fallback; H.265 is hardware-only; AV1 decodes via NVDEC. The VAAPI encoder, decoder, and GPU resize share one render node: the first whose driver does all three, or the one the MOQ_VAAPI_DEVICE environment variable names (for example /dev/dri/renderD129).
Publish on demand. encode::publish_capture advertises the track up front and opens the camera only while someone subscribes.
GPU ownership where the platform allows. Matching codec backends consume their native GPU surfaces directly. The renderer imports CVPixelBuffer and supported DMA-BUF formats. Linux/NVIDIA producers can import dedicated Vulkan RGBA8 slots into CUDA with timeline-semaphore ordering and completion-driven slot return. Vulkan/CUDA surfaces deliberately have no CPU pixel fallback; other surfaces use the typed Surface::into_i420() and configured Surface::to_rgba(config) when needed.
Live bitrate control where the selected backend supports it, without forcing a keyframe. An unsupported backend keeps its opening rate.
Typed group structure. encode::Config::gop is a Gop enum (Keyframe { interval } today), so a later mode adds a variant instead of replacing the field. cut() opens a group at the next frame on both Encoder and Sink, and refuses with Error::CutUnsupported on a backend that cannot force one rather than letting the boundary silently slip to the interval.
Device enumeration for cameras, displays, windows, and apps, matching moq devices.

With capture enabled, capture::camera_modes lists a Linux camera's convertible sizes and exact rates before configuring it. Rates are moq_video::Rate, an exact rational that preserves 30000/1001: frames(duration) counts frames, rounded() gives the nearest whole frame rate for integer-only platform APIs, and as_f64() yields the catalog value. Sizes and rates shared by YUYV and MJPEG are combined; invalid I420 dimensions are excluded. A size range contributes its smallest and largest valid sizes aligned to the driver's step, and an empty rate list means no discrete intervals were reported. Device errors are returned rather than treated as an empty list. Other platforms return Error::Unsupported.

Windows display and window capture require Windows 10 2004 (build 19041) or newer and use Windows.Graphics.Capture without a Desktop Duplication or GDI fallback. Config::cursor controls cursor capture. Frames leave the capture pool as owned GPU NV12 textures; odd native edges are cropped to even sizes. The conversion declares BT.601 limited range through 576 lines and BT.709 limited range above it, preserving that color through resize and CPU download. Media Foundation encodes on the capture device; openh264 uses the download path. Display enumeration covers all adapters, and window enumeration excludes DWM-cloaked windows and reports visible frame bounds. Windows display:N selectors are enumeration indices, not persistent monitor identities. Switching from Desktop Duplication to WGC can change their mapping; enumerate displays again and reselect the intended monitor after upgrading. Enumeration does not start capture. A settled source resize ends the stream so callers can reopen; closing the captured item returns Error::SourceUnavailable. Dropping the stream releases its session even when no new frames arrive. If no usable first frame arrives within five seconds after capture starts, opening fails with Error::SourceUnavailable and releases the session.

Windows shows a capture border by default. On build 20348 or newer the backend requests borderless access; denial keeps the border and does not fail capture. Older supported builds keep the border. Application capture and system audio are not provided by this backend.

With pipewire enabled, capture::cameras also lists PipeWire camera nodes as pipewire:<node name> after the V4L2 devices. V4L2 lists only devices offering YUYV or MJPEG. A PipeWire V4L2 node is hidden only when its device path was already listed by V4L2, so identical webcams stay distinct, and PipeWire-only cameras and NV12- or RGB-only V4L2 devices remain visible through PipeWire. Explicit pipewire:<node name> selectors still open hidden nodes, and pipewire alone opens the camera with the highest session priority, the session manager's default. That reaches cameras V4L2 cannot: a Raspberry Pi CSI camera behind libcamera, and any camera from inside a Flatpak or Snap sandbox, where the default camera comes through the xdg-desktop-portal Camera interface. The mode is chosen from the node's own format list by the same rules as V4L2 (below) and offered exactly, across YUY2, NV12, RGB, and MJPEG, and camera_modes lists the same modes.

capture::Config::framerate is an Option<Rate> request in the same exact type; the stream reports the rate the device accepted, or None when the driver reported none. V4L2 and PipeWire cameras choose the closest geometry, then the format whose accepted rate is nearest the request, then the cheaper conversion when both match equally well.

rust

let mut video = moq_video::decode::Consumer::new(&broadcast, &rendition, "video", Default::default()).await?;
let mut renderer = moq_video::render::Renderer::new(&device, &queue, Default::default())?;
while let Some(frame) = video.read().await? {
    let texture = renderer.render(&frame)?;
}

bash

cargo add moq-video                      # nvidia, mediacodec, openh264 on by default
cargo add moq-video --features capture   # camera + screen capture, no system build deps
cargo add moq-video --features render    # wgpu rendering
cargo add moq-video --features v4l2      # Linux V4L2 M2M codecs, no system build deps
cargo add moq-video --features vaapi     # Linux VAAPI codecs (bindgen needs libclang)
cargo add moq-video --features pipewire  # Wayland screen + PipeWire cameras (links libpipewire)
cargo add moq-video --no-default-features --features openh264  # software H.264 only
cargo add moq-video --no-default-features --features nvidia    # Linux NVIDIA only, no C++ or wgpu

The language bindings use the codec-only shape, which is what keeps their Android floor at API 24 instead of MediaCodec's API 26 entry points.

API: docs.rs/moq-video. Pair with moq-audio.

decode::Config::output chooses where decoded pictures live. Output::Native, the default, hands back whatever the backend decoded into: a CVPixelBuffer, a Direct3D11 texture, a CUDA buffer, a VAAPI DMA-BUF for zero-copy rendering, or CPU pixels from a software decoder. Output::Cpu delivers every picture as Surface::I420, decoded straight to system memory where the backend can. Native surfaces still answer Surface::into_i420(), which returns an I420 with geometry and color metadata intact; call I420::into_data() only when packed bytes are required. decode::Config::scale_hint is best effort and only a decoder with a hardware scaler honors it; Frame::resize is the exact-size operation. decode::Consumer takes decode::Options, which carries the subscription's start and max_age beside the decoder config.

Linux/NVIDIA applications with a native Vulkan producer use frame::vulkan::Importer. Each reusable image is a dedicated, optimal-tiling VK_FORMAT_R8G8B8A8_UNORM allocation exported with an opaque memory FD, plus an opaque-FD timeline semaphore and the physical-device UUID. Publishing consumes the producer-owned slot; awaiting its completion returns that slot only after CUDA readers finish. Import capacity bounds retained images. Unsupported devices, formats, layouts, and synchronization are errors, with no CPU mapping or staging fallback. A non-exportable application image needs one Vulkan GPU copy into an exportable slot. The image is VK_FORMAT_B8G8R8A8_UNORM when imported through Image::bgra8 instead.

frame::cuda::Converter turns a published Vulkan frame into the NV12 Surface::Cuda NVENC encodes in place, on the GPU, in one declared color space (matrix and range) with 4:2:0 chroma averaged per 2x2 block and no transfer function applied. Its buffers come from a pool sized at construction, and cuda::Frame::resize scales a converted frame for a smaller rendition from the same pool, so one captured frame feeding HD and SD holds a fixed number of buffers and a producer that outruns its encoder gets an error instead of unbounded device memory. Open the encoder with encode::Kind::Named("nvenc") and the same encode::Config::color: Kind::Auto could fall back to a software encoder that reads the frame back, and the portable Surface::resize downloads when the GPU scaler fails. Everything under frame::cuda and frame::vulkan runs on the device or returns an error.

just rs vulkan-cuda runs the opt-in native Vulkan/CUDA/NVENC hardware exercise, including a three-view 1280x720 workload that reports per-stage latency and CPU time.

moq-video ​

moq-video