Do Language Models Reason Across Languages?
Quick summary
arXiv:2601.06644v2 Announce Type: replace-cross Abstract: The real-world information sources are inherently multilingual, which naturally raises a question about whether language models can synthesize information across languages. In this paper, we introduce a simple two-hop question answering setting, where answering a question requires making inferences over two multilingual documents. We find that language models are more sensitive to language variation in answer-span documents than in those providing bridging information, despite the equal importance of both documents for answering a quest
Key takeaways
- arXiv:2601.06644v2 Announce Type: replace-cross Abstract: The real-world information sources are inherently multilingual, which naturally raises a question about whether language models can synthesize information across languages.
- In this paper, we introduce a simple two-hop question answering setting, where answering a question requires making inferences over two multilingual documents.
- We find that language models are more sensitive to language variation in answer-span documents than in those providing bridging information, despite the equal importance of both documents for answering a quest
Why it matters
“Do Language Models Reason Across Languages?” highlights the need for repeatable measurement rather than a single impressive demonstration. Independent validation across datasets and clearly stated limitations determine whether a result can guide product decisions.
Member comments