arXiv Artificial Intelligence

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

7 days ago 03.09.2026 07:00 👁 1 views
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

Quick summary

arXiv:2609.02685v1 Announce Type: cross Abstract: RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations. Finetuning methods such as RAFT and PA-RAG enhance RAG by injecting new knowledge into the model's parameters, but require generating a massive amount of synthetic QA that covers the entire corpus. Extended Pre-Training (EPT) on the text corpus avoids the need for comprehensive s

Key takeaways

  • arXiv:2609.02685v1 Announce Type: cross Abstract: RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM).
  • Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations.
  • Finetuning methods such as RAFT and PA-RAG enhance RAG by injecting new knowledge into the model's parameters, but require generating a massive amount of synthetic QA that covers the entire corpus.

Why it matters

“DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.

Kaynak sitede devamını oku: arXiv Artificial Intelligence ↗