Expand description
Janus multimodal vision-language model implementation
Janus combines vision and language understanding with optional image generation. It uses a composition pattern to reuse existing language models (typically Llama) while adding vision encoders and cross-modal alignment.
NOTE: This module contains vision transformer components that are planned for future multimodal support. They are currently unused but retained for the upcoming Janus VLM integration.
Structs§
- CLIP
Vision Tower - CLIP vision encoder (placeholder - not yet implemented with new architecture)
- Generation
Config - Generation pipeline configuration
- Generation
Pipeline - Generation pipeline for image generation
- Janus
Config - Janus model configuration
- Janus
Model - Main Janus multimodal model
- Janus
Placeholder Config - Configuration for Janus image placeholder replacement
- Janus
Placeholder Replacement - Result of Janus placeholder token replacement
- MlpProjector
- MLP projector for vision-language alignment
- Projector
Config - MLP projector configuration (for aligners)
- SigLIP
Vision Tower - SigLIP vision encoder (wrapper around siglip module)
- VQModel
- VQ-VAE model for image generation
- VQModel
Config - VQ-VAE model configuration
- Vision
Encoder Config - Vision encoder configuration
- Vision
Head - Vision head for generating image tokens
- Vision
Head Config - Vision head configuration for image generation
Enums§
- Vision
Encoder - Vision encoder implementations
Functions§
- replace_
janus_ placeholders - Replace Janus image placeholder tokens with proper token sequence