hang

hang is the media format on top of moq-lite: a catalog track that describes the media tracks, and a container that gives each frame a timestamp. It is modeled on WebCodecs so a browser can decode it directly. The spec is draft-lcurley-moq-hang. Plaintext broadcast names end in .hang so a player knows which catalog to expect. End-to-end encrypted broadcasts live under <opaque>/<epoch>, where <opaque> is derived from the credential and a semantic name such as foo.hang, and one epoch identifies each publisher run. The path exposes no format or protection marker; the payloads follow moq-e2ee.

Catalog

catalog.json is a JSON track listing the renditions of each media kind and the decoder config for each. It updates live as tracks come and go, and a compressed twin (catalog.json.z) is published alongside it. Catalog readers refuse an update with more than 64 video, audio, and text renditions combined, before opening media subscriptions. The refusal covers the whole update.

{
  "video": {
    "renditions": {
      "hd": {
        "codec": "avc1.64001f",
        "description": "0164001f...",
        "codedWidth": 1280,
        "codedHeight": 720,
        "container": { "kind": "legacy" }
      }
    }
  },
  "audio": {
    "renditions": {
      "en": { "codec": "opus", "sampleRate": 48000, "numberOfChannels": 2, "container": { "kind": "legacy" } }
    }
  }
}

Each rendition extends the WebCodecs VideoDecoderConfig or AudioDecoderConfig. Every WebCodecs codec is fair game; H.264, H.265, VP8, VP9, AV1, AAC, and Opus are what the tools produce today. Properties shared by every video rendition (display size, rotation, flip) sit at the section root.

A few things the catalog can express beyond decoder config:

• Labels. Any rendition may carry a human-readable label for a track picker. The map key stays the track name used to subscribe, so labels need not be unique and renaming one doesn't rename the track.
• Renditions in another broadcast. A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected.
• Jitter. A rendition can say how far its frames fell behind the media clock before the publisher flushed them, in whole milliseconds rounded up. Encoders report the spread of lateness above each rendition's own recent minimum, so a constant encoder delay is not jitter; container imports estimate batch spans without counting ingest delay. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A 0 is read as absent.
• Delay. A rendition can also say how far its frames reach the transport behind the broadcast's earliest rendition, measured the same way from each rendition's minimum lateness, so a video encoder running 200 ms behind audio advertises delay: 200 on video. It follows the same rules as jitter. A player holds the largest delay + jitter among the renditions it subscribes to, and never subtracts one rendition's delay from another's.
• Stalled renditions. A publisher can flag a rendition as temporarily bad so players prefer another one without the track disappearing. First-party video publishers set this flag after more than three frame intervals of source silence or encoding lag while subscribed, and clear it after three on-time completed frames or when idle. Browser and native capture poll while waiting; FLV and MPEG-TS importers observe video silence as container data arrives. The shared detector is hang::catalog::stalled::Detector in Rust and Catalog.Stalled.Detector in JavaScript. It is a playback diagnostic, not an authorization or routing signal.
• Archive. A broadcast may advertise an archive entry naming its timeline track (a small index of each complete aligned segment) and, if recorded, the replay MoQ path, object-store URL, and format version. The timeline is what lets the HLS gateway build playlists without subscribing to media.
• Clock. The optional root clock maps PTS zero to wall time so every media track and the archive index share one fixed epoch after timescale conversion. It is independent of archive, so a live-only publisher can expose wall-clock timing without creating a segment index.
• Extensions. The root is a loose object. Applications add their own sections (scte35, for example) next to the ones hang defines, optionally naming a track that carries the data. Every library exposes a way to write your section without clobbering the built-in ones, and readers ignore what they don't know.

Text

Captions and subtitles are their own tracks in a text section, not part of the video bitstream, so the relay stays media-agnostic and a viewer downloads only the language it picked. Renditions work like audio: usually one per language, with lang and label driving the picker.

There is no WebCodecs decoder for text, so a consumer parses each cue itself. format says how (vtt, ttml, or utf8) and role is subtitle (dialogue) or caption (all audio). Each frame's timestamp is the cue's start time on the same media clock, so cues schedule against the same playhead.

Data tracks

Not everything in a broadcast is media: a chat log, a telemetry feed, a thumbnail, a serialized game state. The json and binary sections list these as plain tracks, split by whether a generic consumer can parse the payload.

{
  "json": {
    "tracks": {
      "chat": { "mode": "stream", "compression": "deflate" },
      "status": { "mode": "snapshot" }
    }
  },
  "binary": {
    "tracks": {
      "thumbnail": { "mode": "snapshot", "mime": "image/jpeg" }
    }
  }
}

Unlike the media sections these are not renditions: each entry is a distinct track, not an alternative to choose between. mode is required and says how groups compose the frames, because reading an append log as a latest-value document would silently discard everything but the last payload:

• snapshot is lossy. Each group supersedes the previous one, so a consumer reads only the newest. A JSON track may follow the first frame with merge-patch deltas.
• stream is an ordered log: one payload per frame, all in a single group that is never rolled. Retention is still bounded by the group cache, and a consumer that falls behind fails the read rather than silently resuming mid-log.

The rest is descriptive: compression (deflate, the same group-scoped deflate-raw the catalog uses), schema on a JSON track, mime on a binary one, bitrate, jitter, and delay with the same meaning as for media, plus the optional broadcast reference. A publisher measures jitter and delay only from payloads stamped with their capture time on the broadcast clock. A consumer that doesn't recognize a mode or compression ignores that track and round-trips it verbatim.

In Rust the catalog owns the lifetime: catalog.json_stream(track, config) (or json_snapshot / binary_snapshot / binary_stream) writes the entry and retracts it when the producer drops. Read the config from catalog.json.tracks or catalog.binary.tracks, then pair its name and config with moq_mux::catalog::Entry::new to subscribe. In C, moq_publish_json_* and moq_publish_binary_* do the same, retracting on _finish. In the browser, read the same map, subscribe by name, and hand the track to @moq/json or @moq/binary.

An application with its own per-track fields can list a data track in its own root section instead, flattening the JSON or binary entry beside those fields so there is one entry per track. Name the section with a namespaced key such as com.example.mavlink. A generic consumer only finds tracks in json and binary.

Container

The container.kind on each rendition says how frames are framed:

Kind	Frame layout	Use
legacy	varint microsecond timestamp + codec payload	The default. Cheapest.
cmaf	moof + mdat	fMP4 passthrough for HLS/DASH interop; ~100 bytes per frame.
loc	small property block + payload	The IETF LOC container.

A consumer skips renditions with a kind it doesn't recognize and carries them through when republishing the catalog.

A Legacy or LOC video publisher can close the last frame's duration with an empty codec payload whose timestamp is that frame's exclusive end. Consumers treat it as metadata and never pass it to a decoder. This lets a group close immediately without waiting for the next frame. Audio has codec-defined durations, and CMAF carries sample durations directly, so neither needs per-group duration markers. Audio retains its separate terminal marker before codec drain packets, allowing consumers to discard encoder padding beyond the source endpoint. Empty payloads on data tracks remain data, including empty text cues.

Groups and keyframes

A video group is a GoP: it begins with a keyframe and holds the frames that depend on it. That alignment is what makes MoQ's congestion behavior safe. A relay can drop a whole group, a viewer can join at any group boundary, and the decoder never sees a frame whose reference is missing. Audio groups are independent too and typically hold about a second.

The description field carries out-of-band codec setup (an avcC box for H.264). When it is absent, the parameter sets ride inline before each keyframe, which is what avc3/hev1 tracks do. Decoders should handle both.

Your own format

hang is a convention, not a requirement. If you control both ends, publish whatever frames you like on raw tracks; the relay never looks inside them. The MoQ Boy demo mixes hang media tracks with JSON status and command tracks on the same broadcast.