Aperture: Merge-Consistent Rotary States for Compressed Tokens
Quick summary
arXiv:2609.36781v1 Announce Type: new Abstract: Token compression combines content from several positions, yet rotary position embeddings usually assign the merged token one coordinate. We ask what positional information must survive later merges. Aperture stores Fourier moments of the token's weighted support at the model's rotary frequencies. We prove that these moments have minimal real dimension among continuous states sufficient for the selected expected rotary interactions. Represented mass makes updates additive; attention normalisation remains a separate readout choice. Uniform interva
Key takeaways
- arXiv:2609.36781v1 Announce Type: new Abstract: Token compression combines content from several positions, yet rotary position embeddings usually assign the merged token one coordinate.
- We ask what positional information must survive later merges.
- Aperture stores Fourier moments of the token's weighted support at the model's rotary frequencies.
Why it matters
This model development creates a new option for users and a new testing obligation for developers. A fixed evaluation set comparing quality, cost and failure behavior is more useful than launch claims.
Member comments