---
url: https://doc.moq.dev/concept/hang.md
description: The media layer, a WebCodecs-shaped catalog plus timestamped frames
---

# hang

hang is the media format on top of [moq-lite](/concept/moq-lite): a catalog
track that describes the media tracks, and a container that gives each frame a
timestamp. It is modeled on [WebCodecs](https://www.w3.org/TR/webcodecs/) so a
browser can decode it directly. The spec is
[draft-lcurley-moq-hang](/draft/moq-hang). Plaintext broadcast names end in
`.hang` so a player knows which catalog to expect. End-to-end encrypted
broadcasts live under `<opaque>/<epoch>`, where `<opaque>` is derived from the
credential and a semantic name such as `foo.hang`, and one epoch identifies
each publisher run. The path exposes no format or protection marker; the
payloads follow [moq-e2ee](/draft/moq-e2ee).

## Catalog

`catalog.json` is a JSON track listing the renditions of each media kind and
the decoder config for each. It updates live as tracks come and go, and a
compressed twin (`catalog.json.z`) is published alongside it. Catalog readers refuse an
update with more than 64 video, audio, and text renditions combined, before
opening media subscriptions. The refusal covers the whole update.

```json
{
  "video": {
    "renditions": {
      "hd": {
        "codec": "avc1.64001f",
        "description": "0164001f...",
        "codedWidth": 1280,
        "codedHeight": 720,
        "container": { "kind": "legacy" }
      }
    }
  },
  "audio": {
    "renditions": {
      "en": { "codec": "opus", "sampleRate": 48000, "numberOfChannels": 2, "container": { "kind": "legacy" } }
    }
  }
}
```

Each rendition extends the WebCodecs `VideoDecoderConfig` or
`AudioDecoderConfig`. Every WebCodecs codec is fair game; H.264, H.265, VP8,
VP9, AV1, AAC, and Opus are what the tools produce today. Properties shared by
every video rendition (display size, rotation, flip) sit at the section root.

A few things the catalog can express beyond decoder config:

* **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the track name used to subscribe, so labels need not be unique and renaming one doesn't rename the track.
* **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected.
* **Jitter.** A rendition can say how far its frames fell behind the media clock before the publisher flushed them, in whole milliseconds rounded up. Encoders report the spread of lateness above each rendition's own recent minimum, so a constant encoder delay is not jitter; container imports estimate batch spans without counting ingest delay. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent.
* **Delay.** A rendition can also say how far its frames reach the transport behind the broadcast's earliest rendition, measured the same way from each rendition's minimum lateness, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's.
* **Stalled renditions.** A publisher can flag a rendition as temporarily bad so players prefer another one without the track disappearing. First-party video publishers set this flag after more than three frame intervals of source silence or encoding lag while subscribed, and clear it after three on-time completed frames or when idle. Browser and native capture poll while waiting; FLV and MPEG-TS importers observe video silence as container data arrives. The shared detector is `hang::catalog::stalled::Detector` in Rust and `Catalog.Stalled.Detector` in JavaScript. It is a playback diagnostic, not an authorization or routing signal.
* **Archive.** A broadcast may advertise an `archive` entry naming its timeline track (a small index of each complete aligned segment) and, if recorded, the replay MoQ path, object-store URL, and format version. The timeline is what lets the [HLS gateway](/bin/hls) build playlists without subscribing to media.
* **Clock.** The optional root `clock` maps PTS zero to wall time so every media track and the archive index share one fixed epoch after timescale conversion. It is independent of `archive`, so a live-only publisher can expose wall-clock timing without creating a segment index.
* **Extensions.** The root is a loose object. Applications add their own sections (`scte35`, for example) next to the ones hang defines, optionally naming a track that carries the data. Every library exposes a way to write your section without clobbering the built-in ones, and readers ignore what they don't know.

## Text

Captions and subtitles are their own tracks in a `text` section, not part of
the video bitstream, so the relay stays media-agnostic and a viewer downloads
only the language it picked. Renditions work like audio: usually one per
language, with `lang` and `label` driving the picker.

There is no WebCodecs decoder for text, so a consumer parses each cue itself.
`format` says how (`vtt`, `ttml`, or `utf8`) and `role` is `subtitle` (dialogue)
or `caption` (all audio). Each frame's timestamp is the cue's start time on the
same media clock, so cues schedule against the same playhead.

## Data tracks

Not everything in a broadcast is media: a chat log, a telemetry feed, a
thumbnail, a serialized game state. The `json` and `binary` sections list these
as plain tracks, split by whether a generic consumer can parse the payload.

```json
{
  "json": {
    "tracks": {
      "chat": { "mode": "stream", "compression": "deflate" },
      "status": { "mode": "snapshot" }
    }
  },
  "binary": {
    "tracks": {
      "thumbnail": { "mode": "snapshot", "mime": "image/jpeg" }
    }
  }
}
```

Unlike the media sections these are not renditions: each entry is a distinct
track, not an alternative to choose between. `mode` is required and says how
groups compose the frames, because reading an append log as a latest-value
document would silently discard everything but the last payload:

* `snapshot` is lossy. Each group supersedes the previous one, so a consumer reads only the newest. A JSON track may follow the first frame with merge-patch deltas.
* `stream` is an ordered log: one payload per frame, all in a single group that is never rolled. Retention is still bounded by the group cache, and a consumer that falls behind fails the read rather than silently resuming mid-log.

The rest is descriptive: `compression` (`deflate`, the same group-scoped
`deflate-raw` the catalog uses), `schema` on a JSON track, `mime` on a binary
one, `bitrate`, `jitter`, and `delay` with the same meaning as for media, plus
the optional `broadcast` reference. A publisher measures `jitter` and `delay`
only from payloads stamped with their capture time on the broadcast clock. A
consumer that doesn't recognize a `mode` or `compression` ignores that track and
round-trips it verbatim.

In Rust the catalog owns the lifetime: `catalog.json_stream(track, config)` (or
`json_snapshot` / `binary_snapshot` / `binary_stream`) writes the entry and
retracts it when the producer drops. Read the config from `catalog.json.tracks`
or `catalog.binary.tracks`, then pair its name and config with
`moq_mux::catalog::Entry::new` to subscribe. In C, `moq_publish_json_*` and
`moq_publish_binary_*` do the same, retracting on `_finish`. In the browser, read the same map,
subscribe by name, and hand the track to `@moq/json` or `@moq/binary`.

An application with its own per-track fields can list a data track in its own
root section instead, flattening the JSON or binary entry beside those fields
so there is one entry per track. Name the section with a namespaced key such as
`com.example.mavlink`. A generic consumer only finds tracks in `json` and
`binary`.

## Container

The `container.kind` on each rendition says how frames are framed:

| Kind | Frame layout | Use |
| --- | --- | --- |
| `legacy` | varint microsecond timestamp + codec payload | The default. Cheapest. |
| `cmaf` | `moof` + `mdat` | fMP4 passthrough for HLS/DASH interop; ~100 bytes per frame. |
| `loc` | small property block + payload | The IETF [LOC](/concept/standard#loc) container. |

A consumer skips renditions with a kind it doesn't recognize and carries them
through when republishing the catalog.

A Legacy or LOC video publisher can close the last frame's duration with an empty
codec payload whose timestamp is that frame's exclusive end. Consumers treat it
as metadata and never pass it to a decoder. This lets a group close immediately
without waiting for the next frame. Audio has codec-defined durations, and CMAF
carries sample durations directly, so neither needs per-group duration markers.
Audio retains its separate terminal marker before codec drain packets, allowing
consumers to discard encoder padding beyond the source endpoint. Empty payloads
on data tracks remain data, including empty text cues.

## Groups and keyframes

A video group is a GoP: it begins with a keyframe and holds the frames that
depend on it. That alignment is what makes MoQ's congestion behavior safe. A
relay can drop a whole group, a viewer can join at any group boundary, and the
decoder never sees a frame whose reference is missing. Audio groups are
independent too and typically hold about a second.

The `description` field carries out-of-band codec setup (an `avcC` box for
H.264). When it is absent, the parameter sets ride inline before each keyframe,
which is what `avc3`/`hev1` tracks do. Decoders should handle both.

## Your own format

hang is a convention, not a requirement. If you control both ends, publish
whatever frames you like on raw tracks; the relay never looks inside them. The
[MoQ Boy](/bin/demo) demo mixes hang media tracks with JSON status and command
tracks on the same broadcast.