EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
Quick summary
arXiv:2511.10333v2 Announce Type: replace-cross Abstract: Training large language models (LLMs) at scale incurs substantial communication overhead, while static gradient compression cannot adapt to gradient evolution and may degrade model quality. We propose EDGC, an entropy-driven dynamic gradient compression framework that adapts compression ranks to gradient entropy during training. EDGC combines efficient entropy estimation through gradient sampling, a theoretical model relating entropy to compression rank under a bounded-error constraint, and window-based rank adjustment across pipeline s
Key takeaways
- arXiv:2511.10333v2 Announce Type: replace-cross Abstract: Training large language models (LLMs) at scale incurs substantial communication overhead, while static gradient compression cannot adapt to gradient evolution and may degrade model quality.
- We propose EDGC, an entropy-driven dynamic gradient compression framework that adapts compression ranks to gradient entropy during training.
- EDGC combines efficient entropy estimation through gradient sampling, a theoretical model relating entropy to compression rank under a bounded-error constraint, and window-based rank adjustment across pipeline s
Why it matters
“EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments