Expand description
moq-lite event bus plane — replaces the ZMQ XPUB/XSUB ProxyService (#167).
§Architecture
The event bus is a process-global MoqEventOrigin rooted at local/events.
Each service registers its own broadcast under local/events/{source} with a
single events track. Each published frame carries the full dot-separated topic
({source}.{entity}.{event}) and the raw payload, packed as:
[4 bytes topic_len BE][topic UTF-8][payload bytes]Subscribers connect to the origin, discover source broadcasts via
OriginConsumer::announced(), subscribe to their events track, and do
in-memory topic-prefix filtering (matching ZMQ’s prefix-filter semantics).
This is the “Live” preset for the event bus: fan-out, unbounded, at-most-once.
No chained HMAC or policy axes — events are best-effort lifecycle signals, not
auditable streams. crate::events::EventPublisher / crate::events::EventSubscriber
(the canonical broadcast type — see crate::events) layer group-key encryption
on top of this transport, selectable via crate::crypto::event_crypto::EventPrivacy.
§Two origins: moq_event vs moq_stream
Broadcast publish/subscribe (fan-out, N subscribers per source/topic) is
consolidated exclusively on THIS origin (moq_event / MoqEventOrigin).
The sibling moq_stream origin (crate::moq_stream) is reserved for
point-to-point token streaming (one producer, one consumer per stream,
DH-derived StreamKeys) — it is not a second broadcast plane and new
fan-out use cases should not be added to it. Anything that needs “N
interested parties learn about an event” belongs on moq_event; anything
that is a private request/response or token stream between two parties
belongs on moq_stream.
§QoS (#606)
MoqEventSubscriber reuses the EXISTING StreamOpt contract from
streaming.capnp / crate::stream_info (#213/#273) rather than a parallel
event-specific QoS type — see crate::stream_info::EventLive (default) and
crate::stream_info::EventReliable (at-least-once + retained, for events
that must not be silently dropped, e.g. model.loaded). Only delivery and
overflow_policy are enforced client-side today (ordering/completion are
reserved for a future chained-integrity layer, same as the flat at-most-once
default already implies Ordering::Ordered is not currently checked).
§#393 — per-OID broadcast paths
The original design (above) is a flat single-track fan-out: every event for
every model OID lands on the same events track, and subscribers filter
post-read. That is the firehose problem — O(whole-network) reads. The fix
mirrors the streaming plane (moq_stream: {tenant}/{service}/{topic}/{instance}):
each model OID gets its OWN broadcast path, local/events/publications/{oid_hash},
so moq’s scope(&[Path]) makes wire-level selectivity automatic (one QUIC
uni-stream per group per subscribed track). A node tracking N of M OIDs reads
N tracks, not M.
The flat local/events/{source} track is RETAINED as a transition fallback:
publishers that mirror to both paths keep legacy subscribers working while
new subscribers read the selective per-OID track. No capnp change — the event
payload format is unchanged; only the track naming/routing changes.
§Late-join retention (decision A: firehose-backfill)
moq’s per-track cache evicts groups older than MAX_GROUP_AGE (5s) — too
short for a scheduler reconstructing publication history. On first
subscription to an OID’s track, BackfillMode checks whether history is
available via the optional atproto firehose (sh.tangled.git.refUpdate) /
registry; if so it replays that history before switching to live MoQ. If the
firehose is unavailable the subscriber starts live-only (graceful
degradation) — the firehose is the cold-start path, MoQ is the live path.
Structs§
- MoqEvent
Origin - Shared moq origin for the event bus plane.
- MoqEvent
Publisher - In-process moq event publisher.
- MoqEvent
Subscriber - In-process moq event subscriber.
Enums§
- Backfill
Mode - Late-join retention mode for a per-OID (#393) subscriber.
Constants§
- EVENT_
PREFIX - Broadcast path prefix (the
local/eventsroot under which per-source broadcasts are registered). - EVENT_
TRACK - Track name for the event fanout track within each source broadcast.
- OID_
HASH_ LEN - Number of hex characters of the SHA-256 OID hash used in the broadcast path. 16 hex chars (64 bits) keeps paths short while making collisions negligible across any realistic OID namespace (birthday bound ≈ 2^32 models).
- PUBLICATIONS_
PREFIX - Full prefix for per-OID publication broadcasts.
- PUBLICATIONS_
SEGMENT - Broadcast path segment under which per-OID (#393) publication broadcasts
live:
local/events/publications/{oid_hash}. Mirrors the streaming plane’s{tenant}/{service}/{topic}/{instance}shape soscope(&[Path])gives wire-level selectivity — a node tracking N of M OIDs reads N tracks, not M.
Traits§
- Backfill
Source - A source of historical publication events for firehose-backfill late-join (#393).
Functions§
- connect_
event_ moq_ uds_ background - Connect a local event origin to the event service’s UDS plane (#275).
- ensure_
event_ client_ origin - Ensure this process can publish/subscribe events, wiring a CLIENT-mode event origin connected to the event service’s UDS plane if needed (#275).
- global_
moq_ event_ origin - Borrow the process-global moq event bus origin, if initialized.
- init_
global_ moq_ event_ origin - Register the process-global moq event bus origin.
- install_
event_ network_ client_ origin - Install a client origin without opening any local socket.
- oid_
hash - Stable, filesystem/moq-safe identifier for a model OID.
- publication_
broadcast_ path - Build the per-OID publication broadcast path:
local/events/publications/{oid_hash}. - run_
event_ network_ link - One authenticated network link. Target identity must accompany a fresh checkpoint-resolved Iroh transport; callers re-resolve before reconnecting.
- serve_
event_ moq_ uds_ background - Serve the event-bus origin over a UDS socket so cross-process publishers and subscribers reach the same tree (#275).