DeepSeek launches DSpark to speed AI responses up to 85%

A Chinese artificial intelligence company has released a new software framework that it claims can dramatically accelerate the speed at which AI models generate responses, potentially lowering the cost of operating large language models. DeepSeek announced DSpark, a method it says can increase per-user response speeds by 60 to 85 percent.

The framework addresses a core inefficiency in current AI systems: most large language models generate text one word, or token, at a time. This sequential process often leaves powerful graphics processing units underutilized, especially for long responses, leading to longer wait times for users. DSpark employs a technique called speculative decoding to overcome this bottleneck.

Instead of generating tokens individually, the system uses a small, lightweight "draft" model to propose candidate responses. These proposals are then verified in batches by a larger, more powerful model. This approach allows more computational work to be done in parallel. DSpark further refines the process with a semi-autoregressive generation method that produces small groups of tokens at once. A confidence-based scheduling system dynamically adjusts the depth of...