Meta's Muse Spark Open is an 816-billion-parameter mixture-of-experts model that activates 42 billion parameters per token, routes each token to eight of 256 experts, and accepts one million tokens of context. The figures come from an NVIDIA technical blog post dated October 15, written by NVIDIA product staff, which presents the model as tuned for agentic coding on a GB300 NVL72 rack. Those four numbers describe the model's shape precisely. What the post leaves out — the baseline behind "higher first-attempt accuracy," the error introduced by NVFP4 weights, the prefix-cache hit rate that decides whether long sessions are affordable, the token cost of a screen recording — is what determines whether the shape survives contact with a production loop.
The arithmetic is the least ambiguous part. Forty-two billion of 816 billion is 5.1 percent: for every token, roughly one part in twenty of the weight tensor is read for compute, while the whole tensor must still be resident in memory. That split is why the checkpoint ships in NVFP4. At four bits per weight plus block scales, 816 billion parameters occupy roughly 0.5 to 0.6 bytes each, or some 410 to 460 gigabytes — small enough to fit inside a single eight-GPU node's aggregate high-bandwidth memory, and small enough that a rack has memory left over for KV cache. The ratio buys serving cost; the precision buys residency. Both are capacity arguments before they are speed arguments.
The speed claim is where the post gets loose. It reports "up to 50K tokens per second per GPU and up to 600 tokens per second per user" for agentic workloads on a GB300 NVL72. The caption beneath that sentence attributes the same result to "four NVIDIA Blackwell GPUs with speculative decode." Both statements can be true if the four-GPU result is a sub-configuration of the rack, but they are not the same measurement, and only one of them is a rack-scale claim. Dividing the two figures gives roughly 83 concurrent streams per GPU, or about 6,000 across a 72-GPU rack if batching is perfect — a derived number, not a published one, and one that assumes every stream sustains the ceiling simultaneously.
The per-user figure also depends on speculative decoding, whose multiplier is set by the draft model's acceptance rate. That rate is not reported, and it varies with workload: predictable boilerplate accepts readily, while a model rewriting an unfamiliar code path emits higher-entropy tokens that get rejected. A 600 tokens-per-second per-user ceiling measured with speculation is not the same quantity as a steady-state decode rate, and only the second predicts how an agent loop feels while generating a 200-line diff.
Then there is prefill. A million-token window is a memory problem before it is an attention problem. Take a configuration common to models of this size — around 60 layers, eight KV heads, 128 dimensions per head, KV cache held in FP8 — and each token of context costs roughly 120 kilobytes, so a full-window session holds about 120 gigabytes of cache. That figure is an estimate built on assumed dimensions, not a published one, but the order of magnitude explains why the post promises "larger KV cache for concurrent long context sessions" without saying how many sessions. On a GPU with 288 gigabytes of HBM, a handful of full-window conversations consume the device.
The economics of an agent loop then turn on how much of that context is recomputed each turn. Prefix caching, and Dynamo's cache-aware routing that sends a request to the worker already holding the prefix, are the right mechanisms, and the post names both. What it does not give is a hit rate on a realistic multi-turn repository task. Without that, the cost of a 40-turn session against a large codebase is unquantified — and the difference between a 90 percent and a 40 percent hit rate is several-fold in both latency and dollars.
A million tokens of repository is also not a million tokens of signal. Vendored dependencies, generated code, lockfiles and near-duplicate files fill a window quickly, and long-context coding failures tend to be retrieval failures — the wrong file chosen among similarly named ones — rather than capacity failures. The post mentions goal conditioning and context compaction, which reads as an implicit admission that the window is not free. Compaction is a lossy summarisation step, and its trigger, granularity and retention policy are not described. If it fires at 700,000 tokens, the effective context is whatever survives it, not one million.
The multimodal design is the most interesting engineering bet in the write-up. A single 1.8-billion-parameter perception encoder handles interleaved text, images, video frames and audio in one pass, replacing the usual stack of per-modality encoders and the alignment glue between them. That simplifies the pipeline and keeps everything inside a single attention budget. It also compresses: dedicated speech encoders alone are often around 1.5 billion parameters, so sharing 1.8 billion across four modalities implies a tight token budget per input. How many tokens a 30-second screen recording consumes — and therefore how much of the repository it displaces — is not stated. The demo diagnoses a bug from narrated video; the checkable version of that claim is word error rate on the narration plus file-selection accuracy, neither of which appears.
NVFP4 deserves its own note, because the throughput story is partly a precision story. Four-bit weights with block scaling generally hold up at this scale, but two components are sensitive. The router makes a discrete decision — top eight of 256 — and small perturbations can flip expert assignment, which changes the computation rather than merely rounding it. Long-context attention accumulates error across a million positions. The post reports no post-quantization evaluation, so "higher first-attempt accuracy" does not state at which precision it was measured, and the throughput figures are quoted at a precision whose quality cost is left unstated.
None of this demands exotic evidence. A named harness with pass@1 across seeds and its variance; the baseline model, version and quantization; per-task token cost and wall-clock alongside accuracy; a malformed-tool-call rate behind "reliable tool calling"; a prefix-cache hit rate on a multi-turn repository task; and an evaluation re-run on the NVFP4 checkpoint that actually ships. Those are the minimum for a serving claim to be reproducible rather than merely reported.
The architecture itself is legible, and the direction is coherent: sparse activation to hold serving cost near a mid-size dense model, four-bit weights to make 816 billion parameters resident, rack-scale interconnect because top-8 routing over 256 experts is an all-to-all problem, and disaggregated prefill because prompt processing and token generation have different bottlenecks. The open question is narrower than the headline. It is whether 50,000 tokens per second per GPU holds on an independent harness at the shipped precision, and whether a million-token window delivers retrieval precision rather than merely capacity. Until both are measured outside the vendor's four-GPU configuration, the numbers describe a ceiling, not a deployment.
Sources
NVIDIA Technical Blog, "Scale Agentic Coding Workflows with Meta's Muse Spark Open on NVIDIA GB300 NVL72," October 15, 2026 — https://developer.nvidia.com/blog/scale-agentic-coding-workflows-with-metas-muse-spark-open-on-nvidia-gb300-nvl72/
Meta's Muse Spark Open NVFP4 weights on Hugging Face, and NVIDIA Dynamo deployment recipes for SGLang, vLLM and TensorRT LLM, both referenced in the same post.