Alibaba ships the Qwen4 architecture as open weights before the flagship exists
Qwen3.8-Flash-Next puts 36 Gated DeltaNet layers and 12 sparse-attention layers on Hugging Face, which means the retrieval budget Qwen4 will inherit is something you can measure against your own traces now.
The Engineer · Build desk
What happened
- Alibaba opened the weights of Qwen3.8-Flash-Next, a multimodal Mixture-of-Experts model its release blog calls an early preview of the architecture that will be used in Qwen4.
- The open checkpoint is the research build on Hugging Face and ModelScope, while Qwen3.8-Flash is the production sibling on the QwenCloud API with a 1M-token default context and built-in tools.
- NVIDIA published best-effort Day 0 functional support through SGLang, vLLM and TensorRT LLM, plus inference validation on GB300 NVL72 and post-training recipes in NeMo AutoModel and NeMo RL.
- Simon Willison reported running Unsloth quantizations of the model, at 72.5GB and 78.9GB, on a DGX Spark on 26 August 2026.
Why it matters
- What's now possible capability Alibaba says this checkpoint plays the role Qwen3-Next played for Qwen3.5, whose attention pairing then shipped across four subsequent series, so a failure found and filed now has a path into Qwen4 rather than into a later errata note.
- What doesn't add up contradiction Capacity planning splits on which published count you believe, 125B in Alibaba's framing or 176B in NVIDIA's, and the 51B of difference is a single n-gram table that may or may not be resident on your accelerators.
- What it costs cost At 4.8% of parameters active per token, per-token compute for coding work is cheap while residency sets the hardware bill, so the saving only reaches teams who can hold the whole thing or offload the embedding table.
- What it limits constraint Anyone repeating the 8.6x prefill throughput figure inherits its test conditions, a 1M-token context with a 90% prefix-cache hit rate, which describes agent traffic with heavy prefix reuse rather than general serving.
Next story · Build · 3d ago
Qwen3 opens eight models with thinking and non-thinking modes, six dense ones under Apache 2.0Confirmed2 publishers
The layer table is the part worth reading. The model runs 48 layers as a repeating block, written in the config as 12 x (3 x (Gated DeltaNet -> MoE) -> 1 x (Qwen Sparse Attention -> MoE)) [4]. That gives 36 Gated DeltaNet layers and 12 sparse-attention layers [15]. NVIDIA's description of why is the clearest one published: the GDN layers compress history into a fixed-size recurrent state, so KV cache stops growing as the sequence gets longer, and the one layer in four running QSA does the precise retrieval [5].
Now the budget line. QSA is configured with a budget of 512 blocks or 2048 tokens, and its indexer is MQA with four query heads and one shared key head [16]. Divide those two numbers and each micro-block is four tokens wide [30]. So exact retrieval, at a native 262,144-token window or a million with YaRN [2], resolves to a block-level importance estimate choosing 512 of those micro-blocks [6]. That is where I would spend the evaluation budget: long-context recall on your own corpus, at your own context lengths, because the indexer's ranking is the single point on which the design's whole efficiency argument rests.
The efficiency numbers are Alibaba's, and they carry their conditions with them. The attention kernel is reported at up to 7.6x prefill and 4.9x decode against full attention on 1M-token work [19], and the serving figure of 8.6x prefill throughput over Qwen3.7-Plus was measured at 1M context with a 90% prefix-cache hit rate [20]. A coding agent replaying a large shared repository prefix might get near that. A document pipeline with cold, unique inputs will not, and should price the GDN layers alone.
On the coding table, the SWE-bench Pro margin over Claude-Opus-4.6 (Max) is 9.1 points [17], but the footnote matters more than the gap: every model there was run in the Claude Code harness at temp 1.0 and top_p 0.95, except Opus, for which Alibaba reports the officially published score [8]. The comparison is a harness-matched run against a vendor number. The margin over Alibaba's own Qwen3.8-27B, measured the same way, is 0.8 points [18]. On DeepSWE 1.1 the reported figure is the best of two harnesses, and Alibaba notes this model does better on mini-SWE-agent [9], which is a fair disclosure and also a reminder that harness choice is a tuning knob.
Then the parameter accounting. Alibaba's framing is 125B with 6B activated, plus 51B of n-gram embedding and 4B of MTP [31]; NVIDIA calls the same checkpoint 176B total [32], which is the 125B and the 51B added together [33]. The 51B is 20 million bigrams and trigrams indexed at layer 2 [12], and Alibaba's stated reason for scaling there is that embeddings are cheaper to compute and easier to offload than MoE experts [11]. Whether your stack actually offloads it decides which number goes in your capacity plan. Simon Willison's quantized runs at 72.5GB and 78.9GB [24] are the honest lower bound for local work.
Adoption cost is low: the repo ships Transformers-format weights that load in vLLM, SGLang and TokenSpeed [25]. A model numbered 3.8 previewing a model numbered 4 is a naming scheme I will not attempt to defend, but the layer table is legible, and that is the thing you can test.
What to watch
- An independent SWE-bench Pro run that puts every model, Claude-Opus-4.6 (Max) included, through the same harness.
- Whether inference stacks expose the 51B n-gram table as an offloadable tensor rather than resident weights.
- Whether the shipped Qwen4 family keeps the 3:1 GDN-to-QSA layer ratio and the 512-block retrieval budget.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence58
- Adoption52
- Hype gap+32
- Incentives76
- Confidence71
Perspective Coverage
5 publishers- Builder
- Builder 52%
- Operator
- Operator 28%
- Investor
- Investor 20%
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Alibaba stated in its release blog: "In this release, we are opening the weights of Qwen3.8-Flash-Next, a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4."
- [2]
The model natively supports 262,144 tokens of context, extensible to 1,000,000 tokens with YaRN.
- [3]
Qwen3.8-Flash-Next is the open-weight research-frontier model available on Hugging Face and Alibaba's ModelScope; Qwen3.8-Flash is the production version built on the same architecture, offered via the QwenCloud API with 1 million tokens by default and official built-in tools.
- [4]
The model has 48 layers with a hidden layout of 12 x (3 x (Gated DeltaNet -> MoE) -> 1 x (Qwen Sparse Attention -> MoE)), and a hidden dimension of 2560.
- [5]
Three out of every four layers use Gated DeltaNet to continuously compress historical context into a fixed-size recurrent state, eliminating KV cache growth as sequences lengthen; the remaining layer uses QSA for precise retrieval across the full context.
- [6]
QSA aggregates the sequence into micro-blocks, estimates their importance at the block level, and selects only the most relevant regions, in contrast to previous sparse-attention approaches that rely on token-level indexers whose cost grows with context length.
- [7]
On SWE-bench Pro, Qwen3.8-Flash-Next scores 62.5, against 61.7 for Qwen3.8-27B, 55 for Qwen3.7-Plus, 56.0 for DeepSeek-V4-Flash-0731 and 53.4 for Claude-Opus-4.6 (Max).
- [8]
Model card note on SWE-bench Pro: except for Claude-Opus-4.6 (Max), for which the officially published score is reported, all models were evaluated with the Claude Code harness at temp=1.0, top_p=0.95 and 256K context.
- [9]
For DeepSWE 1.1, Alibaba evaluated with the Claude Code and mini-SWE-agent harnesses and reports the highest score across the two, noting Qwen3.8-Flash-Next performs best on mini-SWE-agent.
- [10]
Alibaba says Qwen3.8-Flash requires around one-ninth of the training resources while delivering superior performance, and significantly reduces training and inference costs compared with Qwen3.7-Plus, a model three times its size.
- [11]
Alibaba states that embeddings provide a parameter-scaling axis that requires less computation and is more amenable to offloading than Mixture-of-Experts, making scaling efficient for memory-constrained accelerators.
- [12]
The n-gram embedding covers 20,000,000 bigrams and trigrams, indexed at layer 2.
- [13]
Alibaba claims the model has "superior capabilities in coding and office tasks" and an optimal balance among capability, latency and cost.
- [14]
6B activated out of a 125B main model is 4.8% of parameters active per token.
- [15]
The 12-times repeated block yields 36 Gated DeltaNet layers and 12 Qwen Sparse Attention layers.
- [16]
Qwen Sparse Attention config: 24 query heads, 2 KV heads, head dimension 256, rotary position embedding dimension 64, indexer structure MQA with 4 query heads and 1 shared key head, indexer head dimension 128, budget of 512 blocks or 2048 tokens.
- [17]
The reported SWE-bench Pro margin over Claude-Opus-4.6 (Max) is 9.1 points.
- [18]
The reported SWE-bench Pro margin over Qwen3.8-27B is 0.8 points.
- [19]
Alibaba's published benchmarks report the QSA attention kernel delivering speedups of up to 7.6x during prefill and 4.9x during decoding compared with full attention on 1M-token workloads.
- [20]
In a cache-heavy online serving test at 1M-token context length with a 90% prefix-cache hit rate, Qwen3.8-Flash-Next achieved 8.6x the prefill throughput of Qwen3.7-Plus.
- [21]
NVIDIA provides best-effort Day 0 functional support through SGLang, vLLM and TensorRT LLM, validation across GB300 NVL72 for inference, and post-training recipes from NeMo AutoModel and NeMo RL.
- [22]
GB300 NVL72 integrates 72 Blackwell Ultra GPUs in one NVLink domain with 130 TB/s all-to-all communication, and NVIDIA reports over 16K tokens per second per GPU and over 200 tokens per second per user.
- [23]
The model also runs on local NVIDIA hardware including DGX Station, DGX Spark clusters, and workstations with four RTX PRO 6000 Blackwell Workstation Edition GPUs.
- [24]
Simon Willison reported on 26 August 2026 that he was trying the model on a DGX Spark using Unsloth quantized builds, specifically the 72.5GB UD-IQ1_S and the 78.9GB UD-Q2_K_XL.
- [25]
The repository ships model weights and configuration files in Hugging Face Transformers format, described as compatible with Transformers, vLLM, SGLang and TokenSpeed.
- [26]
The MoE configuration is 512 experts with 10 routed plus 1 shared activated, and an expert intermediate dimension of 640.
- [27]
Alibaba says Qwen3.8-Flash plays the same role Qwen3-Next played for Qwen3.5, introducing the hybrid Gated DeltaNet plus Gated Attention design that was then used across the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series.
- [28]
Gated Residual modulates information through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate, with 4 branches and a bottleneck rank of 320.
- [29]
The training recipe applies Muon and AdamW to specific weight categories and eliminates batch-size warmups, starting directly at the target batch size to reduce total optimizer steps.
- [30]
A budget of 2048 tokens across 512 blocks works out to four tokens per micro-block.
- [31]
Model card figures: 125B parameters with 6B activated, plus 51B n-gram embedding and 4B MTP.
- [32]
NVIDIA describes Qwen3.8-Flash-Next as a multimodal MoE model with 176B total parameters, including 51B N-gram embedding parameters, activating 6B parameters per token.
- [33]
The two published headline counts differ by the n-gram embedding table: 125B plus 51B equals the 176B total NVIDIA cites.
Sources
5 independent publishers whose own reporting we read for this story.
- developer.nvidia.comExperiment with Qwen3.8-Flash-Next 176B Model on NVIDIA GB300 NVL72 for Agentic Coding
1 article · August 26, 2026
- huggingface.coQwen/Qwen3.8-Flash-Next · Hugging Face
1 article · August 26, 2026
- simonwillison.netQwen3.8-Flash-Next
2 articles · August 26, 2026
- the-decoder.comAlibaba releases Qwen3.8-Flash-Next, targeting "ultimate cost efficiency"
2 articles · August 26, 2026
- thenewstack.ioAlibaba just released Qwen3.8-Flash: “An early preview of the architecture in Qwen4”
1 article · August 28, 2026
Topics and entities
Topics
- Long-context inference
- Local Model Serving
- Open-Weight Model Publishing
- Agentic Coding Benchmarks
- Sparse Attention
- Mixture of Experts Architectures