Skip to main content

Module moq_event

Module moq_event 

Source
Expand description

moq-lite event bus plane — replaces the ZMQ XPUB/XSUB ProxyService (#167).

§Architecture

The event bus is a process-global MoqEventOrigin rooted at local/events. Each service registers its own broadcast under local/events/{source} with a single events track. Each published frame carries the full dot-separated topic ({source}.{entity}.{event}) and the raw payload, packed as:

  [4 bytes topic_len BE][topic UTF-8][payload bytes]

Subscribers connect to the origin, discover source broadcasts via OriginConsumer::announced(), subscribe to their events track, and do in-memory topic-prefix filtering (matching ZMQ’s prefix-filter semantics).

This is the “Live” preset for the event bus: fan-out, unbounded, at-most-once. No chained HMAC or policy axes — events are best-effort lifecycle signals, not auditable streams. crate::events::EventPublisher / crate::events::EventSubscriber (the canonical broadcast type — see crate::events) layer group-key encryption on top of this transport, selectable via crate::crypto::event_crypto::EventPrivacy.

§Two origins: moq_event vs moq_stream

Broadcast publish/subscribe (fan-out, N subscribers per source/topic) is consolidated exclusively on THIS origin (moq_event / MoqEventOrigin). The sibling moq_stream origin (crate::moq_stream) is reserved for point-to-point token streaming (one producer, one consumer per stream, DH-derived StreamKeys) — it is not a second broadcast plane and new fan-out use cases should not be added to it. Anything that needs “N interested parties learn about an event” belongs on moq_event; anything that is a private request/response or token stream between two parties belongs on moq_stream.

§QoS (#606)

MoqEventSubscriber reuses the EXISTING StreamOpt contract from streaming.capnp / crate::stream_info (#213/#273) rather than a parallel event-specific QoS type — see crate::stream_info::EventLive (default) and crate::stream_info::EventReliable (at-least-once + retained, for events that must not be silently dropped, e.g. model.loaded). Only delivery and overflow_policy are enforced client-side today (ordering/completion are reserved for a future chained-integrity layer, same as the flat at-most-once default already implies Ordering::Ordered is not currently checked).

§#393 — per-OID broadcast paths

The original design (above) is a flat single-track fan-out: every event for every model OID lands on the same events track, and subscribers filter post-read. That is the firehose problem — O(whole-network) reads. The fix mirrors the streaming plane (moq_stream: {tenant}/{service}/{topic}/{instance}): each model OID gets its OWN broadcast path, local/events/publications/{oid_hash}, so moq’s scope(&[Path]) makes wire-level selectivity automatic (one QUIC uni-stream per group per subscribed track). A node tracking N of M OIDs reads N tracks, not M.

The flat local/events/{source} track is RETAINED as a transition fallback: publishers that mirror to both paths keep legacy subscribers working while new subscribers read the selective per-OID track. No capnp change — the event payload format is unchanged; only the track naming/routing changes.

§Late-join retention (decision A: firehose-backfill)

moq’s per-track cache evicts groups older than MAX_GROUP_AGE (5s) — too short for a scheduler reconstructing publication history. On first subscription to an OID’s track, BackfillMode checks whether history is available via the optional atproto firehose (sh.tangled.git.refUpdate) / registry; if so it replays that history before switching to live MoQ. If the firehose is unavailable the subscriber starts live-only (graceful degradation) — the firehose is the cold-start path, MoQ is the live path.

Structs§

MoqEventOrigin
Shared moq origin for the event bus plane.
MoqEventPublisher
In-process moq event publisher.
MoqEventSubscriber
In-process moq event subscriber.

Enums§

BackfillMode
Late-join retention mode for a per-OID (#393) subscriber.

Constants§

EVENT_PREFIX
Broadcast path prefix (the local/events root under which per-source broadcasts are registered).
EVENT_TRACK
Track name for the event fanout track within each source broadcast.
OID_HASH_LEN
Number of hex characters of the SHA-256 OID hash used in the broadcast path. 16 hex chars (64 bits) keeps paths short while making collisions negligible across any realistic OID namespace (birthday bound ≈ 2^32 models).
PUBLICATIONS_PREFIX
Full prefix for per-OID publication broadcasts.
PUBLICATIONS_SEGMENT
Broadcast path segment under which per-OID (#393) publication broadcasts live: local/events/publications/{oid_hash}. Mirrors the streaming plane’s {tenant}/{service}/{topic}/{instance} shape so scope(&[Path]) gives wire-level selectivity — a node tracking N of M OIDs reads N tracks, not M.

Traits§

BackfillSource
A source of historical publication events for firehose-backfill late-join (#393).

Functions§

connect_event_moq_uds_background
Connect a local event origin to the event service’s UDS plane (#275).
ensure_event_client_origin
Ensure this process can publish/subscribe events, wiring a CLIENT-mode event origin connected to the event service’s UDS plane if needed (#275).
global_moq_event_origin
Borrow the process-global moq event bus origin, if initialized.
init_global_moq_event_origin
Register the process-global moq event bus origin.
install_event_network_client_origin
Install a client origin without opening any local socket.
oid_hash
Stable, filesystem/moq-safe identifier for a model OID.
publication_broadcast_path
Build the per-OID publication broadcast path: local/events/publications/{oid_hash}.
run_event_network_link
One authenticated network link. Target identity must accompany a fresh checkpoint-resolved Iroh transport; callers re-resolve before reconnecting.
serve_event_moq_uds_background
Serve the event-bus origin over a UDS socket so cross-process publishers and subscribers reach the same tree (#275).