FarmGPU retweeted
The NVIDIA Vera Rubin NVL72 scale-up fabric, built on the sixth-generation NVIDIA NVLink.
The NVLink 6 Switch trays connect 72 Rubin GPUs in an all-to-all configuration.
The NVLink spine—over 5,000 copper cables make up the vertical NVLink traverse. It securely moves more data than the entire bandwidth of the internet.
The single-wide NVIDIA MGX rack design unlocks a cable-free, horizontal NVLink scale-up traverse, running directly on PCB for resiliency at scale.
This is how NVIDIA Vera Rubin NVL72 works as one, designed to deliver the lowest token cost and 10x more performance per watt than the previous generation.
#NVIDIAVeraRubin
FarmGPU retweeted
NVIDIA's GPU fault remediation tool went production-ready this week. It's called NVSentinel, it's open-source, and I think it's worth understanding how it works because it draws a clean line between two very different layers of the infrastructure stack.
NVSentinel sits inside Kubernetes. It reads GPU health signals from the NVIDIA driver via DCGM, catches things like ECC errors and driver crashes, and then takes automated action at the scheduler level: cordon the node, drain workloads, hand off to a repair workflow. For teams running NVIDIA GPUs on Kubernetes, that automation is real and useful.
What caught my attention is where it stops. NVSentinel operates above the OS. It can tell you a GPU is failing and pull it out of rotation, but it doesn't go below the driver layer to diagnose root cause, it doesn't talk to BMCs, and it doesn't correlate failures across a fleet. The repair workflow it triggers is a handoff to an external system that has to actually figure out what happened and fix it. That lower layer, actual hardware-level diagnostics and recovery, is what we're focusing on at Cosmic.
I like that NVIDIA is raising the baseline here. It makes the conversation about where the real complexity lives a lot easier to have.
What does your GPU fault response look like today? Are you handling it at the Kubernetes layer, the hardware layer, or both?




