DeepSeek V3.2Exp uses sparse attention to cut long-context costs

Chinese AI developer DeepSeek on Monday released an experimental version of its V3 foundation model, DeepSeekV3.2Exp, saying the update reduces inference costs for long-context tasks and is available as an open‑weight release for developers. The Hangzhou-based company published the model and accompanying technical materials on developer platforms and said the release introduces a sparse attention mechanism intended to make processing long sequences more efficient.

DeepSeek described the new mechanism, called DeepSeek Sparse Attention, as a two-stage system that first uses a lightning indexer to prioritize relevant excerpts from a large context window and then applies a fine-grained token selection step to choose tokens from those excerpts for detailed attention. The company said this approach lets the model operate over extended inputs with lower computational load and that internal tests showed API costs for long-context calls could fall by roughly 50 percent compared with the previous V3.1Terminus release.

To evaluate the impact of sparse attention, DeepSeek said it aligned V3.2Exp training configurations with V3.1Terminus and ran public benchmarks. The company reported...