Large-Small Model Collaboration for Enhancing Edge-Deployed Small Models
Quick summary
arXiv:2503.10367v2 Announce Type: replace-cross Abstract: Edge devices host domain-specific small language models (SLMs) with limited resources, while private clouds offer larger LLMs. We propose G-Boost, an adaptive edge-cloud framework that improves a deployed SLM's task performance without parameter updates. It formulates reasoning as a tree search, choosing at each step between SLM-only inference and SLM-LLM logit fusion---which transfers domain knowledge from the SLM's adapted version to the cloud LLM without exposing private data. A process reward model guides Monte Carlo tree search to
Key takeaways
- arXiv:2503.10367v2 Announce Type: replace-cross Abstract: Edge devices host domain-specific small language models (SLMs) with limited resources, while private clouds offer larger LLMs.
- We propose G-Boost, an adaptive edge-cloud framework that improves a deployed SLM's task performance without parameter updates.
- It formulates reasoning as a tree search, choosing at each step between SLM-only inference and SLM-LLM logit fusion---which transfers domain knowledge from the SLM's adapted version to the cloud LLM without exposing private data.
Why it matters
“Large-Small Model Collaboration for Enhancing Edge-Deployed Small Models” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments