Evaluating Name-Only Directory Routing for One-Shot Code Search
Quick summary
arXiv:2609.35918v1 Announce Type: cross Abstract: Finding the right files is an early challenge for coding agents. We test whether a language model can follow directory and file names to find annotated code files missed by fixed lexical queries. Across 82 audited issues from 11 repositories at pinned pre-fix commits, name-only directory routing recovered 0.465 of gold files within eight candidates, compared with 0.352 for FTS5 and 0.245 for a fixed full-issue rg query. The paired gain over FTS5 was 0.113 (95% repository-cluster bootstrap interval, 0.053 to 0.168). Under a shared 16K-token cont
Key takeaways
- arXiv:2609.35918v1 Announce Type: cross Abstract: Finding the right files is an early challenge for coding agents.
- We test whether a language model can follow directory and file names to find annotated code files missed by fixed lexical queries.
- Across 82 audited issues from 11 repositories at pinned pre-fix commits, name-only directory routing recovered 0.465 of gold files within eight candidates, compared with 0.352 for FTS5 and 0.245 for a fixed full-issue rg query.
Why it matters
This model development creates a new option for users and a new testing obligation for developers. A fixed evaluation set comparing quality, cost and failure behavior is more useful than launch claims.
Member comments