Booz Allen Switches to Nvidia’s Nemotron

AI Summary

Government contractor Booz Allen Hamilton has shifted much of its AI software use from Anthropic models to Nvidia’s Nemotron, citing cost savings in cybersecurity and defense work. The article also describes growing adoption of Nemotron through Palantir and developer platforms, while noting that Nvidia’s models remain far behind leading frontier models in overall usage.

Nvidia is managing the worst hit to its reputation in its three-decade history from the revelations that its CEO Jensen Huang had an affair with a woman reportedly working for China’s spy agencies (read our latest on that here). But in one bright spot for the AI chip giant, there are early signs of headway in its efforts to build the world’s best open-source models. Booz Allen Hamilton—the $10 billion-market capitalization government contractor—several months ago switched from primarily using Anthropic’s models to run its software for cybersecurity and defense to using Nvidia’s Nemotron models, CTO Bill Vass told me.  That move has resulted in significant cost savings, he said, though the company declined to share specifics. (Booz Allen spends tens of millions of dollars annually on AI, according to a person familiar with the company.) “What we've been doing is the opposite of tokenmaxxing,” the term for using as much AI as possible, Vass said in an interview. When it comes to actually building, rather than running, its software, Booz Allen developers still mostly use Anthropic’s Claude Code unless its customers ask them to use other coding tools, Vass said. That’s likely because Claude Code is still lauded as the best tool for software engineering. However, Vass said he is interested in having the firm’s developers use Nemotron models for coding as their capabilities improve. Nemotron has been making inroads with contractors and government agencies in part thanks to its relationship with Palantir. The software firm and Nvidia have partnered more closely lately, as both companies have positioned themselves as suppliers of AI that can help enterprises avoid relying on frontier labs Anthropic and OpenAI. Nvidia recently signed a big deal to use Palantir’s software internally to manage its supply chain.  And Palantir in late June launched a product aimed at helping U.S. government agencies and enterprises use Nvidia open source models through its software. Some government customers have already switched from closed-source frontier AI models to Nemotron, Palantir CEO Alex Karp told me this summer.  It’s not just big customers trying Nemotron. Open-source coding software platform Kilo Code says Nvidia’s Nemotron models accounted for nearly 17% of the roughly 50 trillion tokens processed through its software in October, up from 8% the previous month. Individual developers as well as those from small and large enterprises primarily use Kilo Code to access open source models for coding work.  Despite that progress, Nvidia’s models are still eclipsed by frontier models. Nemotron models accounted for just about 4% of tokens processed through OpenRouter over the past month, down slightly from a peak of 7% one week in August. OpenRouter, the Stripe-owned software that lets businesses autonomously switch between different models from one place, processes a lot more tokens than KiloCode (about 700 trillion monthly) and for a broader set of tasks than coding. So far in November, Anthropic accounted for a higher share (31%) of OpenRouter customers’ spending than any other AI provider, while Nvidia accounted for less than 1% of spending. Tokens as Benefits Those OpenRouter statistics show that most companies are still spending the lion’s share of their AI budgets on Anthropic’s models. As those firms look to moderate spending, they’re diverging on whether to dictate firm token caps to their software engineers, as I laid out in this story today with Phoebe and Catherine. That’s becoming a sticking point for engineers. Maria Colacurcio, the CEO of human resources software startup Syndio, said talent acquisition leaders at Syndio’s large enterprise clients have begun receiving requests from new job candidates to get higher token budgets as part of their employment packages. The requests come after the candidates are offered the job for higher or unlimited token budgets and access to the latest OpenAI and Anthropic models. “So there's all of these sort of side addendums being created where candidates are getting basically agreements to these carve outs,” she explained.  Right now, these aren’t formal benefits. “Nobody‘s writing it down. It’s not really part of policy,” she said.  If AI prices stay high and enterprises further restrict spending—as companies such as Apple, Microsoft, and Disney have—tokens could potentially become a formalized, coveted employee benefit. New From Our Reporters Google Grapples With AI’s Hit to Its Ads Juggernaut By Alix Coutures Exclusive How Jensen Huang’s Yearslong Ties to Alice Huang Enabled $7 Billion-Plus Nvidia Chip Sales to China By The Information Staff Exclusive Meet the Nvidia Board Members Who Will Decide Jensen Huang’s Fate By Abram Brown Upcoming Events Wednesday, November 11 — The Information’s 50 Most Promising Startups This Wednesday, unpack the 50 early-stage startups positioned to define tech’s next era. Get an insider look as The Information’s reporters reveal the companies