Pramaana Labs just scored $27M to make AI stop hallucinating in high-stakes fields

Pramaana Labs just closed a $27 million seed round led by Khosla Ventures, and the pitch is refreshingly direct: they want to make AI actually trustworthy in fields where mistakes aren’t just embarrassing, they’re catastrophic.

The startup is going after formal verification for AI models. That’s the kind of rigorous, mathematical proof we use to verify hardware and critical software systems — think aerospace or chip design — now applied to neural networks. It’s not a new idea academically, but it’s one that’s been notoriously hard to scale.

What caught my attention is the target markets. Pramaana isn’t trying to verify your chatbot’s recipe suggestions. They’re going after law, drug discovery, and tax preparation. Three verticals where an AI hallucination could mean a malpractice suit, a failed clinical trial, or an IRS audit that ruins someone’s year.

In legal, for instance, we’ve already seen lawyers get sanctioned for submitting briefs containing fake citations generated by ChatGPT. Formal verification won’t fix every problem, but it could catch the kind of structural inconsistencies that lead to those disasters. Drug discovery is even more high-stakes — a false positive from an AI model could send researchers down a dead-end path for months.

Tax prep is the one that really made me pause. The US tax code is famously Byzantine, and there’s been a wave of AI-powered tax tools that promise to simplify filing. But if the model misses a deduction or misclassifies income, the customer — not the AI company — is on the hook. Formal verification could at least guarantee that the model’s reasoning is logically sound, even if the tax law itself is a mess.

The $27M seed round is unusually large for this stage, but Khosla Ventures has a history of betting big on infrastructure plays. They were early on OpenAI and Stripe, so they’re not exactly throwing darts blindfolded.

Still, I’m cautious about how far formal verification can go with today’s large language models. These things are essentially black boxes with billions of parameters. Proving anything about their behavior is computationally expensive, and the techniques that work for small models don’t always scale. Pramaana will need to show they can verify real production models, not just toy examples.

What I do like is the focus. Instead of trying to verify everything, they’re targeting narrow, high-value domains where the cost of error is high enough to justify the overhead. That’s a smarter strategy than trying to solve AI safety in general, which is a quagmire that’s swallowed many well-funded startups before.

The timing also makes sense. We’re past the point where AI companies could just say “trust us” and get away with it. Regulators are circling, and enterprise buyers are getting savvier about demanding guarantees. Formal verification won’t be the only answer, but it’s a serious one.

I’ll be watching to see how they handle the compute costs. Verification is expensive, and if they pass those costs to customers, it could limit adoption to only the biggest firms. But for a law firm facing a potential billion-dollar malpractice claim, the math might work out fine.

Bottom line: Pramaana is tackling a real problem in a way that’s focused and practical. The execution is the hard part, but the seed round buys them time to figure it out.