Skip to main content

Module inference_client

Module inference_client 

Source

Structs§

ChatMessage
Generated from Cap’n Proto struct ChatMessage
ChatTemplateRequest
Generated from Cap’n Proto struct ChatTemplateRequest
DeltaStatusResult
Generated from Cap’n Proto struct DeltaStatusResult
EmbedImagesRequest
Generated from Cap’n Proto struct EmbedImagesRequest
EmbedImagesResponse
Generated from Cap’n Proto struct EmbedImagesResponse
ExportPeftRequest
Generated from Cap’n Proto struct ExportPeftRequest
ExportPeftResult
Generated from Cap’n Proto struct ExportPeftResult
GenerationRequest
Generated from Cap’n Proto struct GenerationRequest
HealthStatus
Generated from Cap’n Proto struct HealthStatus
InferenceClient
Auto-generated client for the Inference service.
LayerProfileResult
Generated from Cap’n Proto struct LayerProfileResult
LoraConfig
Generated from Cap’n Proto struct LoraConfig
MergeLoraRequest
Generated from Cap’n Proto struct MergeLoraRequest
MethodSchema
Schema metadata for a single RPC method.
ModelInfo
Generated from Cap’n Proto struct ModelInfo
ModuleNormRatio
Generated from Cap’n Proto struct ModuleNormRatio
ParamSchema
Schema metadata for a single RPC method parameter.
SaveAdaptationRequest
Generated from Cap’n Proto struct SaveAdaptationRequest
SaveAdaptationResult
Generated from Cap’n Proto struct SaveAdaptationResult
SnapshotDeltaResult
Generated from Cap’n Proto struct SnapshotDeltaResult
ToolCall
Generated from Cap’n Proto struct ToolCall
ToolCallFunction
Generated from Cap’n Proto struct ToolCallFunction
TrainStepRequest
Generated from Cap’n Proto struct TrainStepRequest
TrainStepResult
Generated from Cap’n Proto struct TrainStepResult

Enums§

AdaptationStrategy
Generated from Cap’n Proto enum AdaptationStrategy
DispatchResult
Result of a dispatch call — either a JSON response or a streaming setup.
FinishReason
Generated from Cap’n Proto enum FinishReason
InferenceResponseVariant
Response variants for the InferenceResponseVariant service.
Role
Generated from Cap’n Proto enum Role

Traits§

InferenceRpc
Generated RPC trait for the Inference service.

Functions§

dispatch
Transport-agnostic JSON dispatch for this service.
render_doc
Render documentation for this service.
schema_metadata
Schema metadata for the Inference service.
scoped_client_tree
Returns the scoped client tree metadata for dynamic CLI/MCP dispatch.
vfs_nodes
Generated 9P/VFS node table for the inference service (epic #539).

Type Aliases§

Completion
Delivery
Destination
ErrorInfo
IrohReach
MoqlServerIdentity
Ordering
OverflowPolicy
QuicReach
Retention
StreamInfo
StreamOpt
TransportConfig