These notes are my instrument to learn and share things I find beautiful. They're mostly for myself, but it would be great if people find them useful.https://nastya236.github.io/https://nastya236.github.io/blog/rmsnorm-backward-persistent/https://nastya236.github.io/blog/rmsnorm-backward-persistent/The RMSNorm backward pass is memory-bandwidth-bound, and the naive weight gradient doubles the traffic. Here is how to get rid of it.Sun, 09 Aug 2026 00:00:00 GMT