Nvidia’s First Groq 3 LPX Benchmark Turns a Best-Case Demo Into a Scaling Question

Nvidia said Monday that its Groq 3 LPX inference accelerator has entered full production, and for the first time let an outside benchmark firm attach real numbers to the hardware it licensed from Groq for roughly $20 billion in December 2025. The results, published by Artificial Analysis and detailed in a technical breakdown from The Register, show the new hardware generating tokens far faster than Nvidia’s own GPUs. They also show, once the fine print is read, exactly how narrow a case Nvidia chose to demonstrate.

3,400 Tokens a Second, Under Ideal Conditions

In Artificial Analysis’s independent test, an Nvidia Groq 3 LPX rack generated 3,400 output tokens per second running Google’s Gemma 4 31B model with a 100,000-token input context, a figure Nvidia calls the fastest ever recorded for that model. Nvidia says that makes the system four times faster than “the nearest alternative platform.” The Register worked out who that alternative is: Cerebras, which still turned in a respectable 882 tokens per second under the same conditions on Artificial Analysis’s public leaderboard.

The first production customer is Nebius, a Netherlands-based cloud provider that SiliconANGLE reports will fold Groq 3 LPX into Nebius Token Factory, its production inference platform, without requiring customers to migrate to new APIs. “Generation is the phase of inference that determines how responsive an AI system actually is, and that’s exactly what NVIDIA Groq 3 LPX is built to accelerate,” Nebius CTO Danila Shtan said in Nvidia’s announcement. Jensen Huang framed the release more broadly: “Inference is the growth engine of AI,” he said, adding that “Vera Rubin extends that vision with workload-optimized AI factory configurations designed for the era of agentic AI, advancing the performance frontier with LPX for ultrafast token generation.”

Why a Chip With Less Memory Can Be Faster

Groq’s LPUs, short for language processing units, trade memory capacity for memory speed. Instead of the high-bandwidth DRAM that Nvidia’s GPUs use, mainly GDDR7 and HBM4, Groq’s chips run entirely on a large pool of on-die SRAM: memory built directly into the processor instead of connected to it over a separate bus. The third generation of the chips, which launched as part of the Vera Rubin platform earlier this year, delivers 150 TB/s of memory bandwidth per chip, well beyond even HBM4’s roughly 2.75 TB/s per memory stack. Because inference speed is usually limited by how fast a chip can read its own memory rather than how fast it can compute, that bandwidth advantage translates directly into faster token generation.

The cost is capacity. Each Groq 3 LPU carries just 500 MB of SRAM, about 1/576th the 288 GB of memory on a single Rubin GPU. That is nowhere near enough to hold a model like Gemma 4 31B on its own, so Nvidia’s architecture splits the model across many chips connected by Ethernet. A fully loaded LPX rack holds up to 256 LPUs for a combined 128 GB of high-bandwidth SRAM, and multiple racks can be linked together to serve larger models.

The Benchmark Picked the Easy Case

Gemma 4 31B is a favorable model to showcase. At 31 billion parameters, it fits inside a single LPX rack regardless of how Nvidia stores the weights, and at the FP8 precision Nvidia used, the model needs a little over 31 GB, or by The Register’s estimate, just under 64 of a rack’s 256 LPUs. Nvidia did not explain exactly how it distributes the model across chips when The Register asked; the outlet’s best guess is a mix of pipeline and data parallelism, which would leave enough spare capacity for roughly four concurrent copies of the model per rack.

Gemma 4 31B is also a dense model, meaning every one of its 31 billion parameters activates on every token, a total similar to the 37 billion active parameters in DeepSeek’s 671-billion-parameter V3, a mixture-of-experts model that activates only a fraction of its full parameter count per token and carries routing overhead that a dense-model benchmark does not capture. DeepSeek V3’s full parameter count still matters for memory, though: The Register calculates that serving it on Groq hardware would require 1,342 LPUs, just over five LPX racks, a very different proposition from the single-rack test Nvidia published.

A Comparison Without a Chip Count

The Cerebras comparison has a similar gap. Nvidia’s claim of a 4x speed advantage over Cerebras’s 882 tokens per second is accurate as far as the Artificial Analysis numbers go, but neither company has disclosed how many chips it used to hit those figures, at least not in the material published so far. A speed comparison that leaves out hardware count and cost is not necessarily misleading, but it is not a complete efficiency picture either.

The Real Bet Is Pairing GPUs With LPUs

Nvidia frames Monday’s numbers as a preview rather than the full pitch. Its larger plan is a heterogeneous architecture that splits inference into two phases: a compute-heavy “prefill” phase, where GPUs process the incoming prompt and build the model’s working memory, and a memory-bandwidth-bound “decode” phase, where LPUs generate the response token by token. Nvidia’s Vera Rubin platform, which sxz.io covered in July, was already built around a similar division of labor between chip types. The pitch is that combining GPUs and LPUs will let Nvidia sustain a large fraction of that 3,400-tokens-per-second figure across many simultaneous users, not just a single best-case stream.

That bet matters because of what fast token generation buys an AI agent: more turns of reasoning, more tool calls, more work completed in the same span of wall-clock time. Nvidia and its customers are wagering that in an agentic AI market, that speed is worth a hardware premium.

From Rival Chipmaker to Licensed Technology

Groq spent nearly a decade building the LPU as a challenger to Nvidia’s GPUs before that competition ended in December 2025, when Nvidia signed a non-exclusive licensing agreement for Groq’s technology worth roughly $20 billion and hired away founder Jonathan Ross, president Sunny Madra, and other engineers. Nvidia unveiled the resulting Groq 3 LPX hardware at its GTC conference in March, and the accelerator has since become a rack-mountable component of the Vera Rubin platform. Groq itself, no longer in the chip-design business, has raised more than $1 billion since to rebuild as an Nvidia-powered “neocloud,” sxz.io reported earlier this month.

Monday’s disclosure is the first outside look at whether the hardware behind that $20 billion bet performs as advertised. On the numbers Nvidia chose to publish, it does. Whether that holds once customers run larger, sparser models across many concurrent users, rather than a single 31-billion-parameter model built to fit neatly in one rack, is the question Nvidia has yet to answer.