Skip to main content

Module janus

Module janus 

Source
Expand description

Janus multimodal vision-language model implementation

Janus combines vision and language understanding with optional image generation. It uses a composition pattern to reuse existing language models (typically Llama) while adding vision encoders and cross-modal alignment.

NOTE: This module contains vision transformer components that are planned for future multimodal support. They are currently unused but retained for the upcoming Janus VLM integration.

Structs§

CLIPVisionTower
CLIP vision encoder (placeholder - not yet implemented with new architecture)
GenerationConfig
Generation pipeline configuration
GenerationPipeline
Generation pipeline for image generation
JanusConfig
Janus model configuration
JanusModel
Main Janus multimodal model
JanusPlaceholderConfig
Configuration for Janus image placeholder replacement
JanusPlaceholderReplacement
Result of Janus placeholder token replacement
MlpProjector
MLP projector for vision-language alignment
ProjectorConfig
MLP projector configuration (for aligners)
SigLIPVisionTower
SigLIP vision encoder (wrapper around siglip module)
VQModel
VQ-VAE model for image generation
VQModelConfig
VQ-VAE model configuration
VisionEncoderConfig
Vision encoder configuration
VisionHead
Vision head for generating image tokens
VisionHeadConfig
Vision head configuration for image generation

Enums§

VisionEncoder
Vision encoder implementations

Functions§

replace_janus_placeholders
Replace Janus image placeholder tokens with proper token sequence