Qwen3.8-2.4T-A95B: Benchmarks, Architecture, and the Real Hardware Bill for Self-Hosting

On August 12, Qwen published the weights of Qwen3.8-2.4T-A95B, the open checkpoint of the Qwen3.8-Max model it launched through an API on August 3. It arrives two weeks after Kimi K3's weights, which makes two open models above 2 trillion parameters in one month.

The model name tells you most of what you need: 2.4 trillion total parameters, 95 billion active.

The question I care about is the one the name hides. What does it take to run this thing? So I worked it out from the published file sizes and configuration. The calculations below are mine and are labelled as such. I have not run this model myself.

Qwen3.8-2.4T-A95B specifications

Specification	Qwen3.8-2.4T-A95B
Total parameters	2,446,182,725,504
Active parameters	95B
Layers	92: 23 blocks of 3 Gated DeltaNet and 1 Gated Attention, each followed by MoE
Experts	512, with 10 routed and 1 shared per token
Context	262,144 tokens natively, extensible to 1,010,000
Input	Text only
Thinking	Cannot be disabled
Serving	SGLang, vLLM, TokenSpeed

Source: model card.

Some listings give the context as 1 million tokens. The card says 262,144 natively, so plan around that number unless you have tested the extended mode.

What the benchmarks do and do not tell you

Qwen's official table is labelled Qwen3.8-Max, and the same figures appeared in the August 3 launch post, nine days before the open weights.

Benchmark	Score	Configuration
Terminal-Bench 2.1	86.6	Claude Code, average of 10, 5-hour timeout
SWE-bench Pro	67.7	Claude Code, temperature 1.0, 256K, tasks corrected
DeepSWE 1.1	56.6	Better of two harnesses
GPQA Diamond	92.6	Effort not stated
Humanity's Last Exam	43.6	Effort not stated

Source: Qwen, August 3, 2026. All vendor-reported.

Three cautions follow.

Is this the same model? I can find no statement that the table was run on the open checkpoint. It is probably closely related, but the published scores are for a named API model.
Harnesses differ. Rivals' Terminal-Bench results come from other harnesses. K3's 88.3 was achieved in Kimi Code, so a comparison with Qwen's 86.6 in Claude Code is not like for like.
Independent results are for the API. The Decoder reports an Artificial Analysis Intelligence Index of 56 for the API, after a first score of 53 was withdrawn because of endpoint issues. I found no score on the open checkpoint as of today. I would not read anything into a one-point gap with K3 either.

Why a 95B-active model needs 2.4T of memory

People often see "95B active" and picture a 95B model. Active parameters describe compute per token. Routing means any expert could be needed for the next token, so all of them have to be loaded.

The config lets us see how lopsided this is. Here is my calculation, which matches the published counts:

One expert has 8,192 × 2,048 × 3 = 50,331,648 parameters.
Multiplying by 512 experts and 93 layers, counting one extra prediction layer, gives 2,396,591,751,168.
That is 97.97% of all the weights, and it matches the FP8 checkpoint's 8-bit tensor count exactly.
Active parameters: 10 routed experts across 92 layers give 46.3B, plus about 49.6B of non-routed parameters, for about 95.9B. That agrees with the published 95B.

So for each token, you touch 2% of a very large object that all has to sit in fast memory. That is the deal with this kind of architecture. You get cheap compute per token, and you pay in memory.

Why the 4-bit checkpoint is 1.45 TB

Format	Size on disk
BF16	4.892 TB
FP8	2.496 TB
NVFP4	1.447 TB
GGUF Q8_0	2.60 TB
GGUF IQ4_XS	1.31 TB
GGUF Q1_0	397 GB

Sources: FP8, NVFP4, GGUF repositories.

A naive estimate says 2.4T parameters at 4 bits is 1.2 TB, and one recipe says about that. The repository is 1.447 TB. The gap can be explained, and this breakdown is my calculation, with the last item inferred:

Experts at 4 bits: 1,198.3 GB.
Non-routed parameters kept at 16 bits: 99.2 GB.
Scale factors, at about one byte per 16 weights: about 149.8 GB.
Total: about 1,447 GB, which matches the repository.

Budget from the bytes in the repository and treat any round number with suspicion.

The real hardware bill for Qwen3.8

Using 288 GB per B300, and counting weights alone, I get roughly 17 GPUs for BF16, 8.7 for FP8 and 5 for NVFP4. Real deployments need more, and the published recipes reflect that.

Setup	Source	Status
FP8 on 16x GB300, over 4 nodes	SGLang, NVIDIA Dynamo	Vendor recipe. First launch can take up to 4 hours
BF16 on 32x GB300, over 8 nodes	SGLang	Marked not verified
At least two B300 or MI355X nodes	vLLM	Vendor floor
NVFP4 on one B300 node	SGLang	Vendor recipe

Context memory comes on top of the weights. By my estimate the attention layers' KV cache needs 47,104 values per token (23 blocks × 4 heads × 256 × 2). At FP8 that is about 12.3 GB for one session at the full 262,144 tokens. SGLang's notes add that Gated DeltaNet state usually caps concurrency, so treat this as a lower bound. A long-context product needs real capacity planning, beyond fitting the weights.

As for performance, NVIDIA claims over 4,000 tokens per second per GPU and 350 per user on GB300 NVL72 with FP8, without stating concurrency or effort. vLLM reports a quality gap from quantisation, with AIME25 at 87.78 on FP8 and 92.22 on NVFP4 under a raised reasoning budget. That is a surprising direction, and I would verify it on your own workloads before picking a format.

I have left dollar figures out. GPU rental varies too widely and I do not have a good source for it. Use your own quote against the GPU counts above, and then compare that with your actual API bill.

The licence is close to MIT, with two conditions

Products above 100 million monthly active users or $20 million in monthly revenue must show the model name in the interface.
Model-as-a-Service or AI Work Assistant businesses above $50 million over 12 months need a separate licence.
Internal use is exempt.

The repository's Apache 2.0 notice covers the code. The LICENSE file governs the weights, and it is more generous to large service providers than K3's $20 million threshold. "AI Work Assistant" is a broad term, so read that clause carefully if your product might fit it.

Should you self-host Qwen3.8?

Most teams should not. At this size you are running a small GPU cluster to serve one model, with thinking that cannot be turned off.

The sensible first step is to compare it through an API with your current model, on your own tasks. On Auto-RAG I built automated experimentation across retrieval strategies and prompt templates, so tuning became a measured run rather than an argument. A model decision of this size deserves the same treatment. Measure accepted results per dollar, then ask what self-hosting would have to cost to beat that.

The cases where self-hosting makes sense are the same as for K3: data that cannot leave, sustained high volume, or a requirement to control the weights. If that is you, start with NVFP4 on a single node, test quality against FP8 yourself, and measure how many sessions fit beside the weights.

A final practical note. A 2.5 TB download takes a while, and this field is not standing still. When yours finishes, check whether a newer checkpoint has appeared.

Sources

Written by Eslam Hamed, a Technical Lead and AI Solutions Architect working on enterprise RAG, agentic AI, and AI infrastructure.