DeepSeek Introduces mHC for Efficient LLM Training

Chinese artificial intelligence startup DeepSeek has introduced a novel training method for large language models, aimed at enhancing efficiency and scalability amid global competition in AI development. The framework, known as Manifold-Constrained HyperConnections or mHC, was detailed in a research paper published on January 1 through open platforms arXiv and Hugging Face.

The method builds on foundational AI architectures, specifically improving upon residual connections introduced in 2015 and HyperConnections developed by ByteDance in 2024. Residual connections allow gradients—signals that guide model improvements during training—to bypass layers in neural networks, reducing errors in processing user prompts through stacked software components. DeepSeek's mHC refines this by incorporating mathematical structures called manifolds to stabilize gradient flow, addressing limitations in prior approaches such as training instability and high memory demands.

A team of 19 researchers, including company founder and CEO Liang Wenfeng, tested mHC on models with 3 billion, 9 billion, and 27 billion parameters. Compared to HyperConnections, the new technique delivered superior performance...