Probing Factual Knowledge Transfer with Training Data Interventions
Quick summary
arXiv:2609.01341v1 Announce Type: cross Abstract: Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language data? To answer this question more reliably, we propose an intervention-based framework: starting from an English-pretrained model, we continue pretraining on Persian data from which specific facts have been systematically removed at varying levels of granularity. We construct SIFT, a resource of 500 triples across 20 relations, stratified by the cultural origin of eac
Key takeaways
- arXiv:2609.01341v1 Announce Type: cross Abstract: Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language data?
- To answer this question more reliably, we propose an intervention-based framework: starting from an English-pretrained model, we continue pretraining on Persian data from which specific facts have been systematically removed at varying levels of granularity.
- We construct SIFT, a resource of 500 triples across 20 relations, stratified by the cultural origin of eac
Why it matters
“Probing Factual Knowledge Transfer with Training Data Interventions” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments