Skip to main content

Module qwen3_5

Module qwen3_5 

Source
Expand description

Qwen3.5 hybrid SSM/full-attention model implementation

Qwen3.5 interleaves Gated DeltaNet (linear SSM attention) and standard GQA layers. Every 4th layer (by default) is full attention; the rest are GatedDeltaNet. Supports both dense (qwen3_5_text) and MoE (qwen3_5_moe) variants.

Structsยง

Qwen3_5Model
Qwen3_5TextConfig
Qwen3.5 text backbone configuration