OpenAI's Jalapeño chip outpaces Nvidia in first benchmarks

OpenAI used the Hot Chips conference on Tuesday, 25 August 2026, to share its most detailed look yet at Jalapeño, the inference chip it has been developing in-house, publishing the first round of benchmark results for the new system. Tested on SemiAnalysis's InferenceX benchmark, Jalapeño registered both more tokens served per user and higher throughput per kilowatt than the best inference processors currently available — the first time OpenAI has put public numbers behind its silicon strategy.

The bottom line is that the results show a very, very significant performance advance over state of the art. Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It's very efficient to serve a lot of customers, but it can also be very low latency.

How Jalapeño compares with Nvidia's Blackwell

The comparison OpenAI published was made against a system built on Nvidia's Blackwell architecture, currently the industry's reference point for large-scale AI inference. Richard Ho, OpenAI's vice president of hardware, told reporters that Jalapeño offers what he called "the best of both worlds" — lower latency and higher throughput at the same time — when AI systems usually have to trade one for the other. OpenAI itself has acknowledged the comparison is a snapshot: by the time Jalapeño reaches full deployment, competing hardware will likely have advanced significantly as well.

• 1.5x to 1.9x more AI work delivered per watt than the comparison Nvidia GB200 and GB300 systems, across the GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T models
• 1.7x to 3.6x lower end-to-end latency across those same three models
• Benchmarked on InferenceX, a testing platform built by SemiAnalysis to measure how AI systems handle inference workloads
• Compared against the best publicly recorded results at the time of testing, using Nvidia's GB200 and GB300 superchips

Built with Broadcom, and with OpenAI's own models

Jalapeño was developed by OpenAI in close partnership with Broadcom, and OpenAI has said its own models were used in the chip's development process. The company frames Jalapeño as the start of a multi-generation platform, in which AI products, models, chips and memory are designed together, rather than the chip being adapted after the fact to whatever model needs to run on it. That full-stack approach, OpenAI said, let its engineers target specific phases of the inference process that often create friction — chiefly "prefill", where a system processes and interprets everything a user has sent it, and the communication step that follows, which frequently becomes a bottleneck as workloads scale.

We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.

A small first deployment, not a wholesale switch away from Nvidia

Ho said Jalapeño is expected to reach deployment in "very small volumes" by the end of 2026, with a more meaningful rollout following in 2027; OpenAI has not said how many chips it plans to deploy next year. Despite the benchmark results, Ho was clear that OpenAI has no plan to replace its entire compute fleet with Jalapeño. The company's overall compute strategy, he said, still includes "very good partners", Nvidia among them, and OpenAI intends to keep developing second- and third-generation versions of the new chip alongside continued purchases from established suppliers.

Jalapeño itself was first disclosed publicly earlier in the year — accounts of the exact date differ, with some placing the first announcement in October 2025 and others in June 2026 — before Tuesday's event supplied the first hard performance numbers to back up the original pitch. OpenAI intends the platform to keep evolving generation over generation, with hardware, memory and networking tuned together rather than treated as separate purchases.

What this changes for businesses running AI in production

For any company whose product runs on rented AI inference capacity, Jalapeño matters less as a chip to buy — OpenAI is not selling it to outside customers — and more as a signal of where inference costs and latency are heading over the next two to three years. Cloud providers and AI labs racing to cut the cost of serving tokens tend to pass efficiency gains through to customers over time, through cheaper API pricing or faster responses at the same price. Businesses currently paying for GPU-based inference capacity, and weighing whether to lock in long-term commitments with Nvidia-based cloud providers, now have one more concrete data point suggesting that today's pricing and latency floor is not fixed — and that dependence on a single supplier for inference hardware may look different by 2027 than it does today.