1
Reproducing DeepSeek's MHC: When Residual Connections Explode
DeepSeek replaced standard residuals with massive Hyper-Connections, and then watched them explode. I reproduced the 9x signal amplification to understand their fix: the Sinkhorn-Knopp algorithm. Part 1 of 2.