Release 5.17.0

What moved

Release v5.17.0 New Model additions HYV4 Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every token to 8 of them. The context window is 1M tokens. The architecture combines four features: Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent ( kv_lora_rank ) that kv_b_proj expands back to one key/value per query head.

Why it matters

Release v5.17.0 New Model additions HYV4 Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. That is a public launch file from Transformers, dated 2026-09-10. Tagged LLM.

On the record

• Filed from the Transformers official RSS on 2026-09-10.
• Primary source host: github.com.
• Release v5.17.0 New Model additions HYV4 Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token.