OpenAI Unveils Custom Inference Chip Jalapeño With Efficiency Gains

OpenAI on Tuesday unveiled its first custom inference chip, codenamed Jalapeño, presenting benchmark results at the Hot Chips conference that claim significant performance advantages over Nvidia's current systems. The company said the chip, developed in collaboration with Broadcom, is designed exclusively for running AI models and aims to deliver more AI work per watt while reducing response times.

In tests using the public InferenceX benchmark from SemiAnalysis, OpenAI reported that Jalapeño achieved 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency compared to Nvidia's GB200 and GB300 systems. For highly interactive workloads, the performance gap widened to between 2.1 and 4.1 times higher. The chip was evaluated across three open models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. On the largest model tested, Kimi K2.5 1T, Jalapeño delivered about 1.5 times higher peak performance per watt and 3.4 times lower latency than the comparison system.

Richard Ho, OpenAI's head of hardware, described the results as a "very, very significant performance advance over state of the art." He noted that the chip's architecture is...