OpenAI’s Jalapeño: A First-Gen Inference Chip That Actually Competes With Nvidia

A Credible First-Generation Challenge to the GPU Incumbency

For a decade, running a large AI model in production has meant renting Nvidia GPUs and paying the margin that comes with a near-monopoly. On June 24, 2026, OpenAI and Broadcom put the first serious dent in that assumption. According to Tom’s Hardware, the two companies unveiled Jalapeño — OpenAI’s first chip — as a massive reticle-sized ASIC purpose-built for inference. The program was first announced last October, and the design reached tape-out in just nine months, which OpenAI describes as the fastest such development cycle it believes has been achieved for a high-performance advanced semiconductor.

What makes the result notable is not that a large AI company built a chip — several have chip programs — but that a first-generation part posted numbers competitive with the market leader. As SemiAnalysis put it in its independent benchmark write-up, and as The Decoder relayed, CEO Dylan Patel summarized it bluntly: “Usually first generation chips aren’t competitive, but OpenAI is beating Nvidia Blackwell and even Rubin.” That is a strong claim from an analyst house that does not hand out compliments lightly — but it comes with caveats that matter as much as the headline, and this article gives them equal weight.

Jalapeño is an inference chip only. It runs models but does not train them, and it is not a general-purpose GPU. CNBC reported that the chip is being developed with Broadcom and will be deployed inside OpenAI’s own compute infrastructure, framed as a “threat” to Nvidia margins as custom silicon gains ground. The strategic logic is straightforward: inference — not training — is where the recurring, high-volume compute cost lives once a model is in production, so a purpose-built inference part is where a large operator can most plausibly claw back margin from its GPU supplier.

What the Benchmarks Actually Show

The independent numbers come from SemiAnalysis running its public InferenceX benchmark across three models. The Decoder’s summary lists the tested set as GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5. On GPT-OSS, Jalapeño hit about 1,400 tokens per second per user; on DeepSeek R1, it topped 700 tokens per second on a single concurrent request. On the headline performance-per-watt comparison, SemiAnalysis found that Jalapeño beats Nvidia’s Blackwell across almost all scenarios.

OpenAI’s own stated figures, which SemiAnalysis reproduced, are specific. The company says Jalapeño delivers 1.5x to 1.9x more AI work per watt at peak throughput across all three tested models, with 1.7x to 3.6x lower end-to-end latency than the best commercially available systems, and 2.1x to 4.1x higher performance for interactive workloads. OpenAI and Broadcom additionally claim roughly 50% lower cost per token versus Nvidia GPUs — a figure that, if it holds in production, reshapes the economics of serving a model at scale.

1. Read the perf-per-watt win as real but scenario-dependent

The performance-per-watt advantage is the most defensible part of the story, because efficiency is where custom silicon tuned to a narrow workload structurally beats a general-purpose GPU. The 1.5x–1.9x range OpenAI quotes is a per-watt figure at peak throughput, and SemiAnalysis’s independent read agrees that Jalapeño wins on efficiency across nearly all the scenarios it tested. For anyone modeling long-run inference cost, watts are the line item that compounds — a datacenter’s power envelope, not its rack count, is usually the binding constraint. A durable perf-per-watt edge is therefore the claim worth tracking, more than any single tokens-per-second peak.

2. Treat the “beats Nvidia” framing as competitive, not decisive

Here is the caveat the headlines bury. The comparison against Nvidia’s Blackwell is somewhat incomplete, because Jalapeño uses newer HBM4 memory. The fairer like-for-like is Nvidia’s newer Vera Rubin platform, which also uses HBM4. Against Vera Rubin, Jalapeño still squeezes out somewhat more output tokens per megawatt — but on total cost of ownership per token, the two come out roughly even, producing almost the same output tokens per dollar. “Competitive with Nvidia’s HBM4-class Rubin, roughly cost-even” is the accurate framing — not “destroys Nvidia.”

Three further caveats keep the result honest. First, SemiAnalysis did not run its full preferred test suite — there are no long-context, multi-turn agentic results yet, and larger models such as DeepSeek V4 Pro and Kimi K3 have not been tested on Jalapeño at all. Second, these are engineering samples, not production silicon, whereas Nvidia’s Rubin systems are already shipping to customers. Third, Jalapeño posted its numbers without using multi-token prediction or speculative decoding, while some of the comparison systems did rely on those optimizations — which cuts both ways: it means Jalapeño has headroom to improve, but also that the current comparison is not perfectly apples-to-apples.

3. Watch the deployment ramp before drawing conclusions

Benchmarks are a promise; deployed volume is the proof. TechCrunch reported that Jalapeño is expected to deploy at the end of 2026 “in very small volumes,” with a more significant ramp coming in 2027. That timeline matters: a chip that benchmarks well but ships in trivial quantities does not move the market. The signal to watch through 2027 is whether OpenAI’s own inference workloads migrate onto Jalapeño in volume, because that is the clearest evidence that the perf-per-watt and cost claims survive contact with production.

Why AI-Designed Silicon and Custom Kernels Matter Beyond OpenAI

Part of what made the nine-month cycle possible is that OpenAI used its own models to accelerate parts of the design. Tom’s Hardware notes that the companies optimized the architecture around the kernels, memory movement, networking, and serving patterns that matter most for frontier inference, and OpenAI has said its Codex system automatically wrote optimized compute kernels for the chip. The die measures roughly 840 mm², very close to the reticle-size limit of EUV lithography systems, which is why “reticle-sized” is the accurate description rather than marketing.

The broader signal is a structural shift, not a single product. When a model company can co-design an inference ASIC in nine months — partly with AI writing the low-level kernels — the barrier to purpose-built inference silicon drops for the whole industry. CNBC framed the moment as custom silicon gaining ground against Nvidia’s margins, noting a wave of custom AI-chip deals earlier in 2026. The competitive threat to the GPU incumbency is not that one chip wins one benchmark; it is that the method — fast, AI-accelerated ASIC design tuned to a narrow inference workload — becomes repeatable.

For markets that do not own the compute — most of the emerging world included — this is the part of the story with the longest tail. If custom inference silicon and AI-written kernels push the cost per token down and diversify supply beyond a single vendor, the price of running frontier models eventually falls for everyone downstream who rents rather than builds. That does not happen in 2026, and it does not happen because of one chip. But the direction of travel is what a first-generation part competing with Nvidia at all actually signals.

The Honest Scorecard

Strip away the framing on both sides and the result sits in a narrow, defensible band. Jalapeño is a genuine engineering achievement: a first-generation inference ASIC that posts real, independently benchmarked performance-per-watt gains against Nvidia’s Blackwell, delivered in a development cycle that would have been implausible a few years ago. The efficiency win is credible and the cost claim is plausible. That is enough to make it the most serious challenge to the CUDA-and-GPU incumbency the market has seen.

It is not, however, a Nvidia-killer, and pretending otherwise sets up the wrong expectations. Against Nvidia’s HBM4-class Vera Rubin — the correct comparison — Jalapeño is roughly cost-even, not dominant. It is still an engineering sample against a shipping product. The full agentic and large-model test suite has not been run. The right posture for anyone downstream is to treat this as the opening move in a multi-year re-balancing of inference economics, monitor the 2027 deployment ramp, and update as the production numbers arrive — not to declare the GPU era over on the strength of a first-generation benchmark.

Frequently Asked Questions

Sources & Further Reading

• Broadcom and OpenAI unveil custom-built Jalapeño inference processor — Tom’s Hardware
• OpenAI’s first custom chip Jalapeño reportedly beats Nvidia’s Blackwell and Rubin in inference benchmarks — The Decoder
• OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show — TechCrunch
• OpenAI’s Jalapeño AI chip is a ‘threat’ to Nvidia margins as custom silicon gains ground — CNBC
• OpenAI Jalapeño: Better Than Nvidia — SemiAnalysis
• OpenAI × Broadcom: Jalapeño inference chip — OpenAI Newsroom