Learning New Facts with QLoRA: An Acquisition-Retention Frontier
Quick summary
arXiv:2608.25677v2 Announce Type: replace-cross Abstract: Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends strongly on adapter capacity. We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must acquire anonymized geographic associations while retaining unrelated capabilities. Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition--reten
Key takeaways
- arXiv:2608.25677v2 Announce Type: replace-cross Abstract: Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters.
- We show that this assumption depends strongly on adapter capacity.
- We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must acquire anonymized geographic associations while retaining unrelated capabilities.
Why it matters
This is more than a company headline: it shows who controls infrastructure, users and data in the AI value chain. The practical effect will appear in product integration, pricing and delivered capacity.
Member comments