Qwen3.8-Flash-Next: Alibaba previews the Qwen4 architecture

Alibaba's Qwen team released Qwen3.8-Flash-Next on August 26, 2026, an open-weight multimodal mixture-of-experts model that MarkTechPost describes as built “for cost per token.” The checkpoint carries a 125-billion-parameter backbone, but only 6 billion parameters activate for any given token — a ratio the Qwen team positions as an early preview of the architecture that will underpin Qwen4, the same role Qwen3-Next played ahead of Qwen3.5, according to MarkTechPost.

Total parameter count on disk reaches 180 billion once a 51-billion-parameter N-gram embedding table and a 4-billion-parameter multi-token prediction module are added to the main backbone, MarkTechPost reports. The Decoder describes the N-gram layer as a kind of “phrase dictionary” that stores common word groups as standalone entries and can sit in ordinary system RAM instead of GPU memory, at what Qwen calls “relatively low additional cost.”

Four changes carry the release

MarkTechPost identifies four architectural changes behind Qwen3.8-Flash-Next. The first is a hybrid attention scheme: three of every four layers run Gated DeltaNet, a linear-attention mechanism that compresses history into a fixed-size recurrent state, while the fourth layer runs Qwen Sparse Attention (QSA), which selects context at micro-block granularity through a lightweight indexer. Across the model's 48 layers, the pattern repeats as 12 blocks of three Gated DeltaNet layers followed by one QSA layer, with a QSA budget capped at 512 blocks or 2,048 tokens.

The second change, Gated Residual, widens the residual stream into four parallel branches governed by an element-wise read gate and a per-branch write gate at bottleneck rank 320. The third is the N-gram embedding described above. The fourth is a training recipe that applies the Muon optimizer alongside AdamW to specific weight categories, eliminates batch-size warmup and refits the model's scaling laws, MarkTechPost reports. The mixture-of-experts layer itself routes among 512 experts, activating 10 routed experts plus one shared expert per token, at an expert intermediate dimension of 640.

• 125 billion total parameters in the main backbone, only 6 billion active per token
• 51-billion-parameter N-gram embedding table plus a 4-billion-parameter multi-token prediction module — 180 billion parameters on disk in total
• 512 experts in the MoE layer, with 10 routed plus 1 shared active per token
• 262,144-token native context window, extensible to 1 million tokens via YaRN
• FP8 checkpoint: 172.78 GiB; BF16 checkpoint: 335.28 GiB

Benchmarks put it ahead of larger rivals — mostly

On agentic coding, Qwen reports Flash-Next scoring 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro, both ahead of DeepSeek-V4-Flash and Claude Opus 4.6 (Max), according to The Decoder. On office and professional-workflow tasks the gap widens further: Flash-Next posts 73.9 on CoWorkBench against 45.1 for DeepSeek-V4-Flash, and 55.7 on JobBench, nearly double Qwen3.7-Plus's 27.6, The Decoder reports. Qwen3.7-Plus itself carries 397 billion total parameters with 17 billion active per token — nearly three times Flash-Next's active count — while DeepSeek-V4-Flash carries 284 billion parameters with 13 billion active, per The Decoder's account of Alibaba's own benchmark comparisons.

The model does not win everywhere. MarkTechPost notes that Claude Opus 4.6 (Max) leads on Humanity's Last Exam, 40.0 against Qwen's 35.9, and that DeepSeek-V4-Flash-0731 leads NL2Repo-Bench, 54.2 versus 48.1. The Decoder adds that Claude Opus 4.6 is, by comparison, an “older” Anthropic model dating to February 2026, and cautions that benchmark scores and real-world performance can diverge.

A ninth of the training cost, a fraction of the price

Qwen states that training Flash-Next cost roughly one-ninth what Qwen3.7-Plus cost, according to MarkTechPost. On serving speed, the two sources diverge on the exact numbers: MarkTechPost reports Qwen's own announcement citing QSA kernel speedups of up to 7.6 times on prefill and 4.9 times on decode at 1 million tokens, while separate SGLang and vLLM deployment recipes cited in the same MarkTechPost piece put the improvement at 10.2 times and 6.6 times respectively — a discrepancy MarkTechPost itself flags, urging readers to “treat the range as vendor-reported until independently measured.” Qwen also claims 8.6 times the prefill throughput of Qwen3.7-Plus at a 90% prefix-cache hit rate, per MarkTechPost.

On pricing, The Decoder reports that the production version, Qwen3.8-Flash, is available through QwenCloud at $0.16 per million input tokens and $0.47 per million output tokens, with the API expected to go live shortly. That places it roughly one-twelfth the price of Alibaba's current flagship, Qwen3.8-Max, released in early August and positioned against Claude Opus 4.8, Gemini 3.1 Pro and GPT-5.6 Sol, The Decoder notes — even though Flash-Next performs just below that flagship on Alibaba's own benchmarks.

Deployable, but not on a workstation

Despite its efficient active-parameter count, Flash-Next is not a model an individual developer can run casually. MarkTechPost reports the FP8 checkpoint weighs 172.78 GiB and the BF16 version 335.28 GiB; per vLLM's own deployment recipes, the minimum validated FP8 configuration needs two GPUs in tensor-parallel mode on Nvidia's GB300, with four recommended, while an 8×H200 node requires a mixed tensor-expert-parallel setup because standard 8-way tensor parallelism is incompatible with the checkpoint's 128-wide quantization blocks. Sparse activation cuts compute, MarkTechPost notes, but not storage. The model ships under Alibaba's own qwen-community-1.0 license rather than Apache 2.0, meaning commercial users need to check the terms before deployment, MarkTechPost adds.

The bet Alibaba is making with Flash-Next is structural, not just numerical: keeping active parameters low while total capacity grows through mixture-of-experts routing and auxiliary lookup tables lets a model approach the reasoning quality of far larger, more expensive systems while serving it at a fraction of the compute cost per query. It is why Qwen frames this release not as a finished product but as an architecture preview — a public rehearsal, on a smaller model, of ideas Alibaba plans to scale into the full Qwen4 line.

For companies weighing whether to build on Western proprietary APIs such as Claude or GPT, or on Chinese open-weight alternatives, Qwen3.8-Flash-Next sharpens that choice: an open-weight model priced at a fraction of frontier proprietary rates, matching or beating far larger systems on coding and office-automation benchmarks, is now available to self-host or rent through QwenCloud — though it still requires a multi-GPU server, not a laptop, and ships under a non-Apache license worth reading closely before any commercial deployment.