AWS published a sample that fine-tunes Qwen2.5-1.5B with PyTorch FSDP across eight NVIDIA L4 GPUs using the published Ray Train deep learning container, with no image build at any stage. The cluster is one EKS control plane, an m7i.xlarge system node running the KubeRay operator and a Ray head that advertises no GPUs, and two EFA-enabled g6.12xlarge nodes in a single Availability Zone. Each g6.12xlarge carries four L4 GPUs, 48 vCPUs, 192 GiB of memory, 40 Gbps of networking and one EFA interface; each runs one Ray worker pod claiming all four GPUs and the EFA device, giving ranks 0-3 and 4-7 across the two pods. The container pins CUDA 13.0.2 and NCCL 2.29.7 and ships the AWS OFI NCCL plugin, so collectives use EFA as soon as the pods request vpc.amazonaws.com/efa; without that request NCCL falls back to sockets silently. FSDP shards roughly 12 GB of bf16 weights, gradients and optimizer state to about 1.5 GB per L4, at the cost of gathering the full weights and reduce-scattering gradients every step. Setup is five idempotent scripts taking 20 to 30 minutes on a fresh Region, and the post puts the running stack at about $9.50 an hour for both GPU instances, the control plane and the NAT gateway. The sample trains on random token IDs, so it verifies that collectives crossed both nodes rather than measuring throughput.
Our takeThe useful part is the failure catalogue: a missing nvidia.com/gpu.present label leaves the device plugin running on zero nodes without an error, and an absent EFA resource request drops NCCL to sockets while the job still completes. Both are silent-degradation paths that a green CI run would hide, so inspecting which transport NCCL actually used after the run is the right gate. Because the sample trains on random tensors it says nothing about scaling efficiency; for FSDP the per-step all-gather across two nodes is the number worth measuring, along with step-time variance.