NVIDIA’s NV-Raw2Insights-US: Teaching Ultrasound to Actually Listen

Ultrasound has always been the workhorse of medical imaging—safe, real-time, portable, cheap. But the images you see on the screen? They’re not what the probe actually hears. They’re a reconstruction, a compressed interpretation built on assumptions like “sound travels at the same speed through all tissue.” Which it doesn’t. Never has.

NVIDIA, in collaboration with Siemens Healthineers, just released a model called NV-Raw2Insights-US that tries to fix this at the root. Instead of working from finished images, it learns directly from the raw channel data—the actual echoes bouncing back from inside a patient. The idea is to stop treating ultrasound as a camera and start treating it as a microphone that actually understands what it hears.

What Raw2Insights Actually Means

The name is a bit mouthful, but the concept is refreshingly direct. Traditional ultrasound beamforming throws away a lot of information to produce that familiar grainy video. NV-Raw2Insights-US goes the other direction—it takes the full, unprocessed signal and estimates a personalized map of sound speed for each patient. Then it uses that map to refocus the image in real time.

This is the kind of thing that used to require iterative computation, multiple passes, and a lot of time. Now it’s a single AI inference pass on a Blackwell-class GPU. The shift from “process this image” to “understand this patient’s physics” is what makes this interesting, not just another model that makes ultrasound look slightly less noisy.

The Deployment Side is Actually Practical

One thing that caught my attention: they didn’t just build the model and call it a day. They also built the data pipeline to get raw ultrasound data out of a clinical scanner in the first place. That’s usually the hard part—clinical machines don’t expose raw channel data because the bandwidth is enormous.

Their solution is called Holoscan Sensor Bridge, an open-source FPGA IP that streams data over DisplayPort from an ACUSON Sequoia scanner, packetizes it, and sends it over Ethernet to an NVIDIA IGX system for inference. It’s a clever way to retrofit existing hardware without redesigning the scanner. The result is streamed back to the ultrasound machine to adjust focus live.

This is higher bandwidth and lower latency than I expected for a prototype. They’re using an Altera Agilex-7 FPGA dev kit paired with HSB, and it works. The whole thing runs on NVIDIA Holoscan, their edge AI platform, with inference on Blackwell GPUs.

What This Unlocks Beyond Better Images

The immediate benefit is clearer, better-focused ultrasound images—especially in challenging patients where tissue heterogeneity messes with traditional assumptions. But the architecture matters more than the first application.

Because raw channel data is now in GPU memory, you can plug in additional AI models without changing the hardware. That means software-defined ultrasound: new capabilities delivered via updates, not new machines. NVIDIA calls this “modular expansion,” and it’s a phrase I usually roll my eyes at, but here it actually fits.

Some downsides? This is still investigational. The model is released as a research artifact—GitHub, model weights, dataset all available. But clinical validation and regulatory clearance are separate beasts. Also, the hardware requirements (Blackwell GPU, IGX system, FPGA dev kit) aren’t trivial. This isn’t something you’ll see in a community clinic next year.

The Bottom Line

NV-Raw2Insights-US is a genuine step toward AI-native imaging, not just AI applied to images. It learns from physics, not pixels. That distinction matters. The collaboration with Siemens Healthineers (shoutout to Ismayil Guracar and Rickard Loftman) shows this isn’t just NVIDIA talking to itself—there’s real clinical engineering behind it.

If you want to play with it, the code and weights are up on GitHub. The dataset is there too. It’s worth a look, even if just to understand where ultrasound is heading. Because the era of treating sound like a constant is ending.

References:

• “Ultrasound Autofocusing: Common Midpoint Phase Error Optimization via Differentiable Beamforming,” IEEE Trans. Medical Imaging, Feb. 2026.
• “Investigating Pulse-Echo Sound Speed Estimation in Breast Ultrasound with Deep Learning,” arXiv:2302.03064.
• NVIDIA Holoscan SDK Documentation.