Expand description
Qwen3.5 hybrid SSM/full-attention model implementation
Qwen3.5 interleaves Gated DeltaNet (linear SSM attention) and standard GQA layers. Every 4th layer (by default) is full attention; the rest are GatedDeltaNet. Supports both dense (qwen3_5_text) and MoE (qwen3_5_moe) variants.
Structsยง
- Qwen3_
5Model - Qwen3_
5Text Config - Qwen3.5 text backbone configuration