At GTC Berlin, Nvidia announced official support for eight-node DGX Spark clusters, taking unified memory from 128 GB on a single node to 512 GB on four and 1 TB on eight. The company says the larger pool is enough to load the full GLM 5.3 checkpoint (753B parameters) locally, and that a GLM 5.3 Flash configuration measured roughly 80 tokens per second in generation across eight nodes. In its concurrency table, prefill stays at 1,600 tokens per second in total while decode rises from 31 tokens per second at one concurrent request to 51 at four and 67 at eight. Nvidia also describes RoCEnante, a Spark-tuned collective backend that performs an all-to-all exchange followed by local reduction: it reached 3.06x decode speedup at four nodes and 4.43x at eight, against 2.56x and 3.05x without it, on Qwen3.8-27B-NVFP4 with speculative decoding disabled. Nvidia notes the approach moves more data, and that NCCL remains the better choice for high concurrency or larger clusters. Eight-node support in NVIDIA Sync is due by the end of the year.
Our takeThe scaling story splits in two. Prefill holds flat at 1,600 tokens per second as nodes are added, while decode only goes from 31 to 67 tokens per second - about 2.2x on eight nodes - so the near-linear claim describes training throughput, not interactive latency. RoCEnante's edge is also size-dependent: all-to-all trades fewer communication steps for more traffic, which is why Nvidia points users back to NCCL as rank counts grow. These are single-vendor numbers on one model family, with no comparison against an equivalent switch fabric.